High cpu usage in memcached process (couchbase 2.0)
I am testing couchbase 2.0 as a "persistent" memcached alternative in a semi-production environment.
I set up a cluster made of 4 m1.large servers, with a single bucket and a dedicated 4GB of RAM per node (16GB total).
The bucket is configured as "couchbase" and not simply "memcached" because we would like to have strict expiration times.
On all the client I setup the moxi-server v1.8, so that they can connect to localhost using the memcached protocol, and it deals with connecting to the right server for fetching the requested data.
The rate of ops is currently oscillating between 600 ("idle time") and 1000 per second(peak time).
We are testing this since mid december, but suddenly on december 30th the CPU usage of the memcached process started oscillating between 100 and 111%.
There are no evident issues with the service performance (looking at the response time), but the cpu usage is alarming, since in this way we feel not able to detect if the cluster is really operating at maximum capacity or not.