At the moment I’m doing some tests with couchbase 3.0.1 community edition and the java sdk 2.0.1. In full metadata ejection mode the performance drops after approximately 100 million inserts. In contrast, the value ejection mode performs 200 million inserts without problems. Could it be a bug?
I use 2 Clients with 32 Threads for inserting and the Cluster has 2 Nodes.
The hardware specs for clients and cluster nodes are identical:
Dell PowerEdge C62202 with
Interesting, I’d suspect you are hitting some resource exhaustion at some point.
Are you performing only inserts, or are you also retrieving the data back… At this 100m doc mark, can you see something odd happening? For example the disk write queue is growing too large and data keeps backing off…
Can you share the client code so we can try to reproduce it? Also, it would be great if you could run a cbcollect_info so we can take a look at server characteristics.
To rule out anything odd, do you see different client behaviour when the modes are changed? For example garbage collection taking out the box or OOM issues? Something in the logs on the client can be found?
Also, can you share the code somewhere to look at (your changed adapter)?
One thing our support fellas pointed out is that once you hit the non 100% residency mark we need to check for every add and replace (I checked the original thumbtack code) it needs to check if the doc is already on disk or not which will slow it down.
With value ejection only this does not need to happen.
One thing, I think if you use upserts instead of insert/replace, there is an optimization on the server since we can just write. Try to change the ops to upsert instead of insert/replace and see what behavior you get.
Nov 23, 2014 11:31:26 AM com.couchbase.client.deps.io.netty.util.ResourceLeakDetector reportLeak
SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dcom.couchbase.client.deps.io.netty.leakDetectionLevel=advanced' or call ResourceLeakDetector.setLevel()