Home | Forums | Couchbase | Couchbase Server 1.8.x

Setting new key value pairs fails after exhausting RAM quota

4 replies [Last post]
Thu, 05/17/2012 - 16:56
calvin21
Offline
Joined: 05/17/2012
Groups: None

Hi,

I'm doing some performance testing for CouchBase and as part of my test, I want to force read requests to be served up from disk rather than from the RAM.

So I setup the RAM quota to just 512 MB and after creating a few thousand records, I start getting these errors..

"Set failed: 0?4"

Any ideas what might be going on? There's nothing in the event log (seen from the admin console).

Perhaps this is something basic but do I need to add any config to allow data to be flushed to disk seamlessly?

Thanks!

Top
Wed, 05/23/2012 - 10:45
mikew
Offline
Joined: 03/14/2011
Groups:

What SDK are you using?

Top
Wed, 05/23/2012 - 14:45
calvin21
Offline
Joined: 05/17/2012
Groups: None

I'm using couchbase-client-1.1-dp.jar.

I think this was happening due to CouchBase running out of RAM. I was expecting CouchBase to start flushing to disk when this happened but I read somewhere that all keys alongwith some metadata are always stored in memory for performance reasons (which caused subsequent writes to fail when there was no more memory left). This would mean that the amount of RAM needed is in proportion to the size of my data set which is a bit concerning to me.

It would be nice to get some more insights about how CouchBase manages memory.

Top
Wed, 05/23/2012 - 15:08
mikew
Offline
Joined: 03/14/2011
Groups:

The meta data is currently 72 bytes plus the length of the key. This memory is permanently stored in memory unless of course you delete the key. From a client perspective you can get two different types of out of memory errors. One type happens when you are writing to the server faster than it can persist data to disk. At some point your memory will become full and the server will send a TMP_OOM error. This means that the server doesn't have any memory, but if you try again later (about 1ms later) the server will be able to handle your request. At some point if you load enough data you will get an OOM which means that the server is permanently out of memory. This is really bad and the Couchbase alerting system will tell you when you are getting close to this. As a system admin it is your responsibility to add more capacity when you reach this point.

Also, can you do this in your java client to get the message from the server sent back for your set request. It will tell you if you have a TMP_OOM or OOM error.

System.out.println(client.set("key", 0, "value").getStatus().getMessage());

Top
Wed, 05/23/2012 - 15:10
calvin21
Offline
Joined: 05/17/2012
Groups: None

Thanks, that's very helpful!

Top