Java Client Connection keep closing and reopening, Auth failure

Java Client: 1.1.9
Couchbase Server: 2.1.1 Enterprise
OS: CentOS in VirtualBox

int numOfCouchbaseConnection = 1
List<URI> hosts = Arrays.asList(new URI(""));
    CouchbaseConnectionFactoryBuilder builder = new CouchbaseConnectionFactoryBuilder();
    couchbaseClients = new ArrayList<>();
    for (int i = 0; i < numOfCouchbaseConnection; i++) {
      couchbaseClients.add(new CouchbaseClient(builder.buildCouchbaseConnection(hosts, "products", "Administrator", "")));

If I switch the Couchbase Server to 2.0.1 Community, there will be another error

2013-09-05 12:38:56.932 WARN net.spy.memcached.auth.AuthThread:  Authentication failed to
2013-09-05 12:38:56.938 ERROR net.spy.memcached.protocol.binary.SASLStepOperationImpl:  Error:  Auth failure
2013-09-05 12:38:56.939 INFO com.couchbase.client.CouchbaseConnection:  Reconnection due to exception handling a memcached operation on {QA sa=, #Rops=1, #Wops=0, #iq=0, topRop=SASL steps operation, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
OperationException: SERVER: Auth failure
 at net.spy.memcached.protocol.BaseOperationImpl.handleError(
 at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(
 at net.spy.memcached.protocol.binary.SASLBaseOperationImpl.finishedPayload(
 at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(
 at net.spy.memcached.protocol.binary.SASLBaseOperationImpl.readFromBuffer(
 at net.spy.memcached.MemcachedConnection.handleReads(
 at net.spy.memcached.MemcachedConnection.handleIO(
 at net.spy.memcached.MemcachedConnection.handleIO(
2013-09-05 12:38:56.940 WARN com.couchbase.client.CouchbaseConnection:  Closing, and reopening {QA sa=, #Rops=1, #Wops=0, #iq=0, topRop=SASL steps operation, topWop=null, toWrite=0, interested=1}, attempt 0.
2013-09-05 12:38:56.940 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl:  Discarding partially completed op: SASL steps operation
2013-09-05 12:38:57.039 WARN net.spy.memcached.auth.AuthThread:  Authentication failed to
2013-09-05 12:38:58.941 INFO com.couchbase.client.CouchbaseConnection:  Reconnecting {QA sa=, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL steps operation, toWrite=0, interested=0}
2013-09-05 12:38:58.942 INFO com.couchbase.client.CouchbaseConnection:  Connection state changed for
2013-09-05 12:38:58.943 WARN net.spy.memcached.auth.AuthThreadMonitor:  Incomplete authentication interrupted for node {QA sa=, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL steps operation, toWrite=0, interested=8}
2013-09-05 12:38:58.944 WARN net.spy.memcached.auth.AuthThread:  Authentication failed to
2013-09-05 12:38:58.948 INFO net.spy.memcached.auth.AuthThread:  Authenticated to

I have the same issue. I spend stress testing and at the same time kill one of nodes. There are no operations I have "WARN net.spy.memcached.auth.AuthThread: Authentication failed to ..." What mechanism will allow to continue the work of the application after the restoration of failing node.

1 Answer

« Back to question.


When you connect to Couchbase cluster you should not use the Administrator user. The Administrator should only be used to do some administration operation such as create/edit bucket using the ClusterManager class

When you connect to the cluster to do "normal operations" you have 2 options
- connect without any security/password
- connect with an SASL password (that you set at the bucket level).

So for example in your case you will set a password to the products bucket, and connect using:

CouchbaseClient cb = new CouchbaseClient(builder.buildCouchbaseConnection(hosts, "products", "mypassword"));

Also I see in the code snippet you put in the question that you are creating a list of CouchbaseClient. This is not a a good idea. You just need one client in your application. The "pool" is manage internally in the client.


Hey tgrall! I did a performance test for my pool

#threads	#connections	#queries
1	        1             21869
2         1             22191
4         1             21212
2         2             40548
4         4             103663
1         2             23193
1         4             20889

My Testing Code

    couchbaseClients = new ArrayList<>();
    for (int i = 0; i < numOfCouchbaseConnection; i++) {
      CouchbaseClient couchbaseClient = new CouchbaseClient(hosts, "products", "");
      couchbaseClients.add(new AbstractMap.SimpleEntry<>(couchbaseClient, couchbaseClient.getView("products", "default")));
		for (int i = 0; i < numOfCurrency; i++)
			Thread thread = new QueryThread(categories.get(random.nextInt(categories.size())));
    for (int i = 0; i < numOfCurrency; i++)
public synchronized static Map.Entry<CouchbaseClient, View> getCouchbaseClient() {
    couchbaseIndex = couchbaseIndex % couchbaseClients.size();
    return couchbaseClients.get(couchbaseIndex++);
public void run()
		long start = System.currentTimeMillis();
		long end = start + Couchbase.seconds;
		while(System.currentTimeMillis() < end)
      Map.Entry<CouchbaseClient, View> a = Couchbase.getCouchbaseClient();
      Query query = new Query();
			query.setRange(ComplexKey.of("waejk;jkgrad", "3789560789", m_category, null), ComplexKey.of("waejk;jkgrad", "3789560789", m_category, ""));
			ViewResponse viewResponse = a.getKey().query(a.getValue(), query);

No matter how many threads share the same connection, the number of queries of different threads are nearly same. if you have multiple connections for different threads, there will be a huge difference in performance

No matter if you use auth or you don't, authentication failure after one of servers restart/crash.
If only one thread is making requests through the client (single instance in my case, not a list of) then occasionally auth failure disappears (after ~5-10 minutes), if many threads try to do operations on the client (many means 50-100), reconnection time increases or even no reconnection occurs.

Any thoughts?