Performance decline severely when upgrading from Server 2.2 to 4.6.1

Couchbase Server
1

Hi,

We’ve been using Couchbase Server 2.2 with Java Client version 1.4 for couple of years,
and very happy with couchbase performance, stability and in general.

Lately we wanted to upgrade to the latest server version in order to enjoy the new features and specially
the new backup&restore mechanism cause we’re started having OOM errors while backuping our cluster
(our production environment having 20M~ docs)

I did a stress test to see how the new server is behaving, and encountered a major decline in the performance.
Our test environment is as follows:

  1. 1 Couchbase Server on Aws linux m4.large machine.
  2. 1 App Server on different AWS Linux m4.large machine.

In the test I’m creating 1000 documents with 500 Chars long size.
Then in an infinite loop for couple of minutes i’m:

  • Reading a document (through java client ‘find’ method)
  • Modifying it, and then saving it again (through java client ‘set’ method)
  • then moving to the next document in cycle.

Doing that for couple of minutes, while logging the amount of Ops and the amount of time.
The results are that without any other change but upgrading the server from 2.2 to 4.6.1 (and staying with the same java client 1.4.4 library)
the Ops per Second declined in about 30%
from 5000~ Ops Per Second
to 3400~ Ops Per second

The Couchbase server & app servers configuration stayed the same.

Any thoughts on why this is happening?
Is any one encountered the same decline?
is there a change i can do in the 4.6.1 cluster configuration to improve this?

Thanks,
Shay

2

Overall those numbers sound low - irrespective of the Server version.

It sounds like you’re doing synchronous operations - which will never be the most efficient (you’re essentially benchmarking your network round-trip between the App server and CB Server).

Having said that, I wouldn’t expect a 30% drop (I’d expect it to be faster).

  • What eviction policy (new in 3.x) are you using on the 4.6 cluster?

I’d recommend testing with the newer 2.x Java SDK also - particulary if you’re using async operations you should see a massive (order of magnitude) speedup.

3

Hi @drigby,
Thanks for your reply

I’m using value ejection

and a question regarding sync/async:
I’m using find & save methods - so essentially async approch can help me only in the save methods, cause find methods
are synced anyway.
am i correct?

Anything else you know of i can check in my server configuration to understand why the performance declined?