Home | Forums | Membase | Membase Server 1.6.x

Membase tuning

4 replies [Last post]
Tue, 05/31/2011 - 12:23
abhihome
Offline
Joined: 03/14/2011
Groups: None

Hi,

We are migrating to Membase from EHCache, and we are trying to address some performance concerns with Membase.
The application is a content website, where data needs to be pulled against thousand's of keys.
EHCache worked well, because the cache is within the same same server - it is not distributed.

With Membase, since the requests are going out to the Membase server to get the data, it is causing latency.
We have addressed a most of it by doing a multi-get using asyncGetBulk(), but this is still taking time when there are large number of keys.

Questions:
1. I am using 20 net.spy.memcached.MemcachedClient instances, and running the application on two servers, so a total of 40 MemcachedClient's. Is there a scale to decide the number of clients to be used? If yes, can you please explain?

2. What is the advisable number of keys an asyncGetBulk() can effectively handle?

3. Is there any other tuning strategies I can apply to speed up the Cache-Gets?

Server: Membase 1.6.5
Client: memcached-2.6.jar (Spymemcached)
Number of Membase Servers: 2 (4GB each)
Local Moxi: None
Data buckets : One (default)

Constructor used to create clients:
MemcachedClient(ConnectionFactory cf, List<InetSocketAddress> addrs)
The Multi-Get method:

Future<Map<String, Object>> f;
f = getCache().asyncGetBulk(keys,serializingTranscoder);
Object o = f.get(2, TimeUnit.SECONDS);

Please let me know if I am missing any information, which you are looking for to respond to the questions.

Thanks!
Abhilash

Top
Tue, 05/31/2011 - 12:48
dan
Offline
Joined: 01/05/2011
Groups: None

Slight hijack, which exact version of the spymemcached client are using? The pre-releases that have been posted here do not work for me with any of the async get requests, I get back null values for keys that actually exist. (I submitted a bug back in April about this).

Edit, or are you connecting to the moxi instance on each server?

Top
Tue, 05/31/2011 - 13:04
abhihome
Offline
Joined: 03/14/2011
Groups: None

Hi Dan,
I am using memcached-2.6.jar as my spymemcached client.
I do not have a local Moxi on each server.

Thanks
Abhilash

Top
Wed, 06/01/2011 - 16:49
mikew
Offline
Joined: 03/14/2011
Groups:

First off, Spy 2.6 doesn't provide support for a vBucket map so one thing that is slowing you down is that some of the requests in the getBulk that you are sending will have to take two hops to get to the server that the actual key is on. There are two things you can do to fix this. The first is to setup client side moxi. Having moxi on the client side will allow moxi to resolve which keys should be sent to which server before making the first hop. This will ensure that all requests get to the correct server and will cut down on latency. The alternative to this is to wait for the 2.7 release of spymemcached which will have vBucket map support. This version of spymemcached will allow you to completely bypass moxi altogether and will result in the lowest possible latency. This release should be out sometime next week.

The other thing that you can do is try to use 16 MemcachedClient connection instead of 20. For personal experience I have found that 16 connections per machine gives the best throughput/latency trade-off. With that said this number should really be based on what else is running on your server and is application specific. 16 connections might work great for you, but you might find that doing some experimenting yourself is the best way to optimize. Just remember more threads leads to more context switches and higher perceived latency between the server and client.

Top
Wed, 06/01/2011 - 20:20
abhihome
Offline
Joined: 03/14/2011
Groups: None

Thanks Mike, this is helpful.

Top