We have run few simple tests to check if couchbase ops / second meets our requirements using a while loop on a single thread as following cases:
a. get same document (~1.25k ops/sec)
b. set same document (~1k ops/sec)
c. set same document with persist / replicate (at max 300 ops/sec)
bucket.upsert(doc, PersistTo.ONE, ReplicateTo.ONE, 15, TimeUnit.SECONDS)
Java SDK: 2.1.3
Server (4 nodes on VMs): 4 cpu, ~10GB ram, ~20GB storage each
Item Count: 200
Bucket RAM/Quota Usage: 69.1MB/37.3GB
Bucket Data/Disk Usage: 107MB/114MB
We are targeting to manage at least 10K ops/second but it seemed quite far from our expectation.
Q1) Is there some configuration in SDK / couchbase server specified the max throughput and therefor our test a) and test b) are limited under 1.25k?
Q2) Is there another way to improve the performance (i.e.: introduce lots threads to increase throughput)?
Q3) Test c) (save with persistence) actually will have a long pause (5~15 seconds) if same document is saved more than twice. Is this normal? How to predict how long should we put for upsert timeout to ensure the operation can be completed (otherwise we will get timeout exception). And how to improve this ops/second?
Same problem here, I got maximum of 3000 sets/second using a single process, obviously using more threads can increase the performance, but that doesn’t suit my need.
I have much (MUCH) better results using SDK 1.4, do you want to give it a try?
Kevin: that’s certainly not what we’d expect to see from a four node Couchbase Server cluster. The disk quota you’ve allowed for the bucket looks low compared to the storage available to the node but I can’t think why that would cause slow through-put.
Assuming a fairly typical cloud VM set-up, without unusual network load, you should be seeing at many tens of thousands, if not hundreds of thousands, of ops per second.
Can you describe a bit more about your environment. For example, is the app server in the same network as the Couchbase servers?
@daschl or @simonbasle can you answer more fully, please?
From the disk overview it said Total Cluster Storage = 76.8GB but no sure why disk quota is 37.3GB. Maybe you can let me know where we can set it probably.
Currently the testing is running under one of the couchbase server and all nodes are in the same host using same network card so it might not be network issues.
By the way, I am use Coubase bucket type + Full Eviction Cache with High I/O priority. I don’t know if this will have side effects. Please let me know if more information I can provide to troubleshoot this issue. Many thanks.
Are you using SDK1.4 with Couchbase v3.0.2? I might try to change my SDK to see if performance gets better.
@kevin_cheng, my initial plan was to use latest client provided, however my test shows SDK 1.4 is much faster, I am still checking why it happens that way, please share your result after you try with SDK 1.4. Thanks.
BTW my server is 3.0.3, but I doubt that makes any big difference
We are at same stage to upgrade the SDK to the latest version.
By the way, as suggested, I updated the SDK to 1.4.9 and found out that the ops/second increased to 21k ops/second. But instead of using
upsert I used
CouchbaseClient.set(docID, 0, doc); // persistent storage
The jump from 1.5k to 21k is quite big and now I am not sure should we perform the upgrade.
I also tried the persistTo and replicateTo flags and performance did degraded to 5 ops/second… (0.2 ops/second for few operations then 5~8 seconds delay).
CouchbaseClient.set(docID, 0, doc, PersistTo.ONE, ReplicateTo.ONE);
Anyways, about add PersistTo and ReplicateTo which I will raise another topic about it.