Timeout when system is under load

Couchbase Server
1

Hi,

I am using couchbase server 4.5.0-2601 Enterprise Edition (build-2601) with Reactive couchbase plugin. Node config is different on different env(2 to 10). Future get’s timeout frequently when the load is high.

Below is the result when I try to do some load test locally (fyi, I flushed the DB before running the load test).

I used the same couchbase server(4.5) with 1 node locally. I add 20000 items in the bucket. Later, I try to retrieve the same by calling [MemcachedClient#getBulk] (https://github.com/dustin/java-memcached-client/blob/04285ea97c1b4f61d3285512c53f92d16b4512d2/src/main/java/net/spy/memcached/MemcachedClient.java#L1616). Below is the result when I use different parallel config.

  • When load is 1(1 thread active at a time), average time to serve request is ~600ms
  • When load is 8, average time to serve request is ~7000ms
  • When load is 50, average time to serve request is ~27000ms

Above results explains the reason behind time out on the server.

Just wondering if there is something I am doing wrong or this is expected.

Thanks in advance.

2

The first thing you need to find out is where the slowdown comes from. Can you please post the client and server specs as well as how many ops/s and what kind of workload are you running through it?