On the client side (php-ext-couchbase) we’re seeing a lot of these “Failed to get a value from server: Operation timed out”. I researched this a bit and it seems this is a very generic message that could indicate a number of different issues most of which are a result of a server issue.
On the server side I see a lot of errors like this:
info.20:[ns_server:info,2013-10-03T3:01:49.214,ns_1@10.133.145.111:ns_port_memcached<0.13767.0>:ns_port_server:log:171]memcached<0.13767.0>: Thu Oct 3 03:01:49.013772 UTC 3: 877 Closing connection due to read error: Connection reset by peer
I wasn’t sure if these are related b/c I wasn’t sure whether the server errors are referring to a connection with another member of the cluster or a client or perhaps both. If these are related, I guess the next step would be to look at some packet captures? If they aren’t related, where I should I be looking? I’ve ruled out rebalances as the cause (this came up a number of times during research).
We’re running 2.0.1 on the server and we’re using 1.0.6 of the php-ext-couchbase extension.