Home | Forums | Membase | Membase Server 1.7.x

Membase memory usage

2 replies [Last post]
Sun, 11/06/2011 - 19:54
adaboost
Offline
Joined: 11/06/2011
Groups: None

Hello sirs,

We are using Membase 1.7.1.1 and have been running into some issues in terms of memory scalability. Our use case consists of ~20-30 byte keys with only counter values (incr, decr mutations only). We currently have 20 million keys per 16GB membase server. These counters are used in random order and do not exceed the max value of an integer.

The servers are fine until they start swapping and going to disk because they've hit their limits. 20 million keys for 16GB of ram seems a bit inefficient. Is the use case given not amenable to membase? Or is there some settings that should be tweaked/dealt with in order to optimize for a key heavy dataset.

Thanks!
--young

Top
Mon, 11/07/2011 - 10:46
adaboost
Offline
Joined: 11/06/2011
Groups: None

To be more fair the cluster seemed to be scaling well at 20 million keys. At that point it was handling 3 million keys per gigabyte of memory. At 60 million keys this ratio drops to 1.25 million keys per gigabyte.

The build is membase-server-community_x86_64_1.7.1.1 on 64-bit ubuntu lucid boxes. The dataset is a single bucket allocated with all available memory with replication set to 1.

Top
Mon, 11/07/2011 - 19:42
adaboost
Offline
Joined: 11/06/2011
Groups: None

As a comparison, redis on the same dataset stores 34 million keys @ 3.5 gb of ram. This is pretty much a consistent 10 million keys per 1 GB ram. Membase is seemingly VERY inefficient at scale for key heavy datasets. Any pointers to make it more comparable?

Top