Couchbase extension issues
I've run into a few issues in my production environment using the Couchbase extension that have me scratching my head pretty good. I'm running a cluster of 20 Centos 5.5 servers with Membase 126.96.36.199 installed. Connecting to them are 45 app servers with the Couchbase PHP extension, version 1.0.1. We went live with the application that's running on all this today and say a large amount of the following error:
Error: Couchbase::__construct(): Failed to connect libcouchbase to server: Connection failure
We saw the most errors when we had around 2500 ops/sec to the cluster. My initial thought is that I'm running out of connections. In the space of about 5 hours, one Membase server had a total of 10 million connections to it. The general flow of the program is we do
$membase = new Couchbase('my.membaseServer.com:8091', 'username', 'password', 'default');
Then from there we normally do a few gets and sets and then the script finishes execution. However, this happens quite a bit. The number of errors increased as our traffic increased and then the number of errors decreased as the traffic decreased. At this point, though, I'm not sure where to start looking. A couple things we've done is we took out the bucket specification in the Couchbase constructor call and we took out a call to getStats() that we were using to verify the connection. I looked into updating the Couchbase extension to the latest version, 1.0.4, since I saw in the release notes and various other forum posts that a timeout issue had been fixed as well as a few other things. But when I updated the extension on my test server I received a 'Couchbase::get(): unknown payload type' error, which had me puzzled too since the last time I saw that was when I switched from using the Memcached extension and moxi client to the Couchbase extension.
So at this point I'm pretty confused and not really sure where to start looking. Anyone have any insight into things I can look for to trouble shoot this error to get rid of those pesky failed to connect errors?