[JCBC-304] Client leads to a deadlock when all the servers are unavailable Created: 21/May/13  Updated: 24/May/13

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Deepti Dawar Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
Stack Trace :-

"pool-17-thread-8" prio=10 tid=0x00007fdea007c800 nid=0xacac waiting on condition [0x00007fdf168e6000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000007f8d383f0> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.jav
a:994)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:
1303)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
        at com.couchbase.client.vbucket.BucketUpdateResponseHandler.getReceivedFuture(BucketUpdateResponseHandler.java:147)
        at com.couchbase.client.vbucket.BucketUpdateResponseHandler.getLastResponse(BucketUpdateResponseHandler.java:127)
        at com.couchbase.client.vbucket.BucketMonitor.startMonitor(BucketMonitor.java:212)
        at com.couchbase.client.vbucket.ConfigurationProviderHTTP.subscribe(ConfigurationProviderHTTP.java:333)
        - locked <0x00000007f81cb9c0> (a com.couchbase.client.vbucket.ConfigurationProviderHTTP)
        at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:247)

    

 Comments   
Comment by Deepti Dawar [ 23/May/13 ]
The usage of CountDownLatch here in BucketMonitor may not be quite justified in the case where the user application is calling this from a multi-threaded env in a non-synchronized manner.
Here either our code needs to be intelligent enough to avoid deadlocks or the calling application needs to be aware of the same.

Looking at the customer issue which lead to this, their usage of the CouchbaseClient instance seems incorrect.

Matt, Can you please suggest ?




[JCBC-311] Expose the Server Error Code in the OperationStatus Created: 23/May/13  Updated: 23/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.6
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Minor
Reporter: Tug Grall Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The Java SDK does not use the Error Code from the server side, as documented here:
http://blog.couchbase.com/handling-runtime-errors-ruby-python-and-c-clients

It would be very useful for the developer to be able to use the same code in Java. We could use the OperationStatus to expose this enumeration.




[JCBC-310] Create a getBulk() operation that returns CASValue object Created: 23/May/13  Updated: 23/May/13

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1.6
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Minor
Reporter: Tug Grall Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The current client.getBulk(keys) returns the values only,

It would be useful to create an operation that returns the list of CASValue to allow developer to do CAS operations.

This should also be supported by the views when doing a "includeDocs" operation




[JCBC-309] Catch ConcurrentModificationException in IO thread as safety net. Created: 23/May/13  Updated: 23/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.6
Fix Version/s: 1.1.7
Security Level: Public

Type: Task Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-305] Client does not recover with Memcached connection to the server and primary node goes down. Created: 21/May/13  Updated: 21/May/13

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Deepti Dawar Assignee: Deepti Dawar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File mem_all_nodes_input.log     Text File mem_primary_node_failure.log    

 Description   
2013-05-21 12:59:04.158 ERROR net.spy.memcached.protocol.ascii.StoreOperationImpl: Error: SERVER_ERROR temporary failure
 Deepti Exception 1 java.util.concurrent.ExecutionException: OperationException: SERVER: SERVER_ERROR temporary failure
2013-05-21 12:59:04.158 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=/10.3.121.207:11211, #Rops=2, #Wops=0, #iq=0, topRop=Cmd: add Key: Emp0000000365cbc Flags: 0 Exp: 0 Data Length: 79, topWop=null, toWrite=0, interested=8}. This may be due to an authentication failure.
OperationException: SERVER: SERVER_ERROR temporary failure
at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:164)
at net.spy.memcached.protocol.ascii.OperationImpl.readFromBuffer(OperationImpl.java:151)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:537)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:430)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:247)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:915)
2013-05-21 12:59:04.159 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000366cbc.
2013-05-21 12:59:04.159 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=/10.3.121.207:11211, #Rops=2, #Wops=0, #iq=0, topRop=Cmd: add Key: Emp0000000365cbc Flags: 0 Exp: 0 Data Length: 79, topWop=null, toWrite=0, interested=8}, attempt 22.
2013-05-21 12:59:04.159 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000366cbc.
2013-05-21 12:59:04.159 WARN net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl: Discarding partially completed op: Cmd: add Key: Emp0000000365cbc Flags: 0 Exp: 0 Data Length: 79
2013-05-21 12:59:04.160 WARN net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl: Discarding partially completed op: Cmd: version
2013-05-21 12:59:04.160 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000366cbc.
2013-05-21 12:59:04.160 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000366cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:06.660 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000367cbc.
2013-05-21 12:59:06.661 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000367cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:09.163 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000368cbc.
2013-05-21 12:59:09.164 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000368cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:11.666 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000369cbc.
2013-05-21 12:59:11.667 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000369cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:14.168 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000370cbc.
2013-05-21 12:59:14.169 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000370cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:16.671 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000371cbc.
2013-05-21 12:59:16.672 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000371cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:19.174 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000372cbc.
2013-05-21 12:59:19.175 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000372cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:21.677 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000373cbc.
2013-05-21 12:59:21.678 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000373cbc.
 Deepti Exception 1 java.lang.RuntimeException: Timed out waiting for operation
2013-05-21 12:59:24.179 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000374cbc.
2013-05-21 12:59:24.180 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for Emp0000000374cbc.

 Comments   
Comment by Deepti Dawar [ 21/May/13 ]
In this case, server is stopped one by one on all the server nodes starting with the primary node.
Even when the first node is brought up, the client does not recover. It recovers much after all the nodes are started and are active in the cluster.
Comment by Deepti Dawar [ 21/May/13 ]
One more observation, the client is only trying to insert data to the server nodes. Even if the primary node fails, it should not wait for the recovery of this node, instead should point to the other active node of the cluster and add the data there.
Not sure of the behavior here. Matt, can you please help me in understanding what should be the correct behavior in this case.
Comment by Deepti Dawar [ 21/May/13 ]
When all nodes are given as input, then also the behavior remains the same or even worse. Now getting the ConcurrentExecutionException while cancelling the connection to the primary node when it goes down. Attatched is the log.




[JCBC-308] Trouble in resubscribing to primary node via cbc, when all servers go down one by one. Created: 21/May/13  Updated: 21/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.6
Fix Version/s: .next
Security Level: Public

Type: Bug Priority: Critical
Reporter: Deepti Dawar Assignee: Deepti Dawar
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File cbc_server_all_nodes_failure.log    

 Description   
PFA the logs.





[JCBC-307] Hybrid workload with rebalance scenario - error in accessing view Created: 21/May/13  Updated: 21/May/13  Resolved: 21/May/13

Status: Resolved
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.6
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File HYBRID_rb-2-out_1.1.6_views_access_error.log    
Issue Links:
Duplicate
duplicates JCBC-231 Handle View Errors (Vbucket & Merge) ... Open

 Description   
Attached is the log and following is the stack trace -


SDKD: May 10, 2013 1:32:51 AM com.couchbase.sdkd.cbclient.CommandResult warnAbout
SDKD: WARNING: Unknown exception encountered (for operation) future warnings will be suppressed
SDKD: java.lang.RuntimeException: Failed to access the view
SDKD: at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:871)
SDKD: at com.couchbase.sdkd.cbclient.ViewQueryCommandContext.execIter(ViewQueryCommandContext.java:252)
SDKD: at com.couchbase.sdkd.cbclient.CommandContext.execute(CommandContext.java:374)
SDKD: at com.couchbase.sdkd.server.SdkServer.executeCommand(SdkServer.java:168)
SDKD: at com.couchbase.sdkd.server.SdkServer.handleRequest(SdkServer.java:189)
SDKD: at com.couchbase.sdkd.server.SdkServer.run(SdkServer.java:245)
SDKD: Caused by: java.util.concurrent.ExecutionException: OperationException: SERVER: error Reason: A view spec can not consist of merges exclusively.
SDKD: at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:90)
SDKD: at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
SDKD: at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
SDKD: at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:864)
SDKD: ... 5 more
SDKD: Caused by: OperationException: SERVER: error Reason: A view spec can not consist of merges exclusively.
SDKD: at com.couchbase.client.protocol.views.NoDocsOperationImpl.parseError(NoDocsOperationImpl.java:106)
SDKD: at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:68)
SDKD: at com.couchbase.client.ViewNode$MyHttpRequestExecutionHandler.handleResponse(ViewNode.java:204)
SDKD: at org.apache.http.nio.protocol.AsyncNHttpClientHandler.processResponse(AsyncNHttpClientHandler.java:417)
SDKD: at org.apache.http.nio.protocol.AsyncNHttpClientHandler.inputReady(AsyncNHttpClientHandler.java:242)
SDKD: at com.couchbase.client.http.AsyncConnectionManager$ManagedClientHandler.inputReady(AsyncConnectionManager.java:249)
SDKD: at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:172)
SDKD: at org.apache.http.impl.nio.DefaultClientIOEventDispatch.inputReady(DefaultClientIOEventDispatch.java:155)
SDKD: at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:161)
SDKD: at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:335)
SDKD: at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
SDKD: at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:275)
SDKD: at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
SDKD: at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:542)
SDKD: at java.lang.Thread.run(Thread.java:619)

 Comments   
Comment by Michael Nitschinger [ 21/May/13 ]
Hey,

this is a duplicate (tracked somewhere else, but I dont remeber the exact ticket)

More importantly, this is a bug on the server side, we can't do much about it at the moment. Anyway, closing this since we don't want to track it twice!




[JCBC-231] Handle View Errors (Vbucket & Merge) appropriately Created: 02/Feb/13  Updated: 21/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-307 Hybrid workload with rebalance scenar... Resolved

 Description   
During rebalance out it can happen that the view node is still able to service http connections, but fails with the following errors:

- error Reason: A view spec can not consist of merges exclusively.
or
- no_active_vbuckets Reason: Cannot execute view query since the node has no active vbuckets

If this is happening, the client needs to deal with it. Appropriate solution needs still to be dicussed, but one approach could be to retry on a different node. Note that this needs to work for both async and sync calls.

 Comments   
Comment by Michael Nitschinger [ 02/Feb/13 ]
The corresponding server ticket can be found here: http://www.couchbase.com/issues/browse/MB-7661




[JCBC-306] Add a method that returns the list of design document for a bucket Created: 21/May/13  Updated: 21/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.6
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Tug Grall Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
It would be great to add a method to return the list of design document for a bucket.

We should do something like:

List< DesignDocument > client.getDesignDocuments()




[JCBC-300] Add as Example: Java/NoSQL Tricks Created: 28/Mar/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Karen Zeller Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://www.infoworld.com/d/application-development/how-teach-java-ee-app-new-nosql-tricks-215277

 Comments   
Comment by Karen Zeller [ 09/May/13 ]
I'm not sure i would take this request as high priority, but back in March, someone (I can't remember) sent me an email saying we should add this type of content to our doc set.

I would just keep this in your queue for now. It might just be more informational/inspirational......My suspicions is this is a priority 2.




[JCBC-204] document increment/decrement behavior with non-numeric value Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-incr.html


 Description   
Increment and decrement should document what happens if the key exists on the server, but the value is invalid. It should describe what a valid value is (and link to the Developer's Guide page that describes the server-side behavior in greater detail).

Valid value is a string. String is decimal representation of an unsigned 64-bit integer.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-11] Need more unit tests for couchbase-client Created: 03/Feb/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.0.1
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Raghavan Srinivas Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
        CouchbaseConnectionFactoryBuilder cfb = new CouchbaseConnectionFactoryBuilder();
        cfb.setOpTimeout(10000); // wait up to 10 seconds for an operation to succeed
        cfb.setOpQueueMaxBlockTime(5000); // wait up to 5 seconds when trying to enqueue an operation
        

For example, will fill up the funnel (the blocking queue) to the rim, but not overflow (immediately timeout). Once it's at the rim, it'll have to wait to wait until at least one operation flows out to add another operation. This *will* slow down the callers (their async calls will actually block on this internal queue), but that's okay in a bulk loader.

However, there are no unit tests in couchbase-client to test this.





[JCBC-15] add showtype-options to documentation Created: 23/Feb/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Many of the Java docs should show the type in the options summary.

 Comments   
Comment by Michael Nitschinger [ 15/Nov/12 ]
Can you give me a quick example on what you mean? Reassign it back to me then and I'll fix it!
Comment by Matt Ingenthron [ 15/Nov/12 ]
If you look at the docs, there are many places where we have types that are returned, but we don't sufficiently describe those types. For example:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-set-add.html#table-couchbase-sdk_java_add

It mentions the OperationFuture, but nowhere really tell how to use it (to my knowledge).

You should be able to work with MC on the right way to fix these.




[JCBC-18] NPE if hostnames in server bootstrap list are mixed case Created: 12/Mar/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A user described a scenario where using mixed case in their URIs lead to an NPE. This is from the map lookup, since what the couchbase cluster sends us is different than what the user entered, I think.

See: http://www.couchbase.com/forums/thread/java-client-101-exception-using-couchbaseclient-servlet-filter




[JCBC-61] Expose returned CAS value in CASResponse when available from binary protocol Created: 06/Jun/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1dp
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Customer request to add the capability to retrieve new cas value after a cas() operation to avoid a subsequent gets()

 Comments   
Comment by Michael Nitschinger [ 29/Nov/12 ]
This may already be implemented, need to check.




[JCBC-108] observe loop implementation behind mutations should be adaptive to server persistence latencies Created: 11/Sep/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The current observe loop implementation does not acknowledge the average persistence latency which is in the observe response.




[JCBC-89] re-enable optimization after error handling in binary optimized sets is fixed in dependent spymemcached Created: 31/Jul/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.0.3
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Matt Ingenthron [ 17/Dec/12 ]
I do think this one needs to be addressed before 1.2.




[JCBC-83] replace use of java assert in tests with junit assertions, including messages Created: 12/Jul/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Mike Wiederhold Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2 because its a smaller change.




[JCBC-76] Replica read command Created: 12/Jul/12  Updated: 20/May/13

Status: In Progress
Project: Couchbase Java Client
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Implement a simple replica read command

 Comments   
Comment by Matt Ingenthron [ 29/Aug/12 ]
See http://www.couchbase.com/wiki/display/couchbase/Replica+Read for details on implementation.
Comment by Raghavan Srinivas [ 14/Sep/12 ]
This will be experimental functionality for 1.1
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to .next until it is decided if this will make it into 1.2 or not.
Comment by Matt Ingenthron [ 14/Feb/13 ]
Michael: this has come up again recently so if possible we'd like to get it in a build for testing soon and see about adding it as experimental, if not supported, in 1.1.3.




[JCBC-117] mention that OperationFuture.get(tmo) changes state when timeout has been reached Created: 24/Sep/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.0.3
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks JCBC-114 Command Futures never receive results... Reopened

 Description   
get(tmo) should not change the underlying state of the command to being timed out. It should simply respond with a TimeoutException but allow the command to continue.

Specifically, when the arg-tmo (timeout passed as an argument) expires, the underlying command is marked as timed out. For example, if one waits for 50ms on the command and a response has not been received within that time, the command is now dead ('TIMEDOUT', or similar) and waiting again will not help.

It is understandable that some code might rely on the old behavior, so at the very least, this should be documented in 'BIG RED LETTERS' in the get(tmo) method.

 Comments   
Comment by Matt Ingenthron [ 05/Oct/12 ]
Please explain further.
Comment by Michael Nitschinger [ 16/Oct/12 ]
Hey Mark,

Can you explain in more detail what you want to see changed? When the argument is timed-out what should happen then with it?

Thanks,
Michael
Comment by Matt Ingenthron [ 24/Oct/12 ]
As currently designed, the client uses get() to determine timeout. This is not going to change at the moment. There's no other appropriate place internal to the client to check for this timeout of the operation at the moment.
Comment by Mark Nunberg [ 24/Oct/12 ]
Moving this as a documentation bug
Comment by Matt Ingenthron [ 06/Nov/12 ]
Michael, I'd like you to give this one a shot as your first docs bug, I'll help you with this as needed.




[JCBC-116] Implement the ObserveSet for better observing of replication/persistence Created: 18/Sep/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-113] code injection during development/debug causing IllegalStateException Created: 13/Sep/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1dp, 1.1dp2, 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Minor
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Caused by: java.lang.IllegalStateException: Shutting down
at net.spy.memcached.MemcachedConnection.checkState(MemcachedConnection.java:824) ~[spymemcached-2.8.4.jar:2.8.4]
at net.spy.memcached.MemcachedConnection.enqueueOperation(MemcachedConnection.java:640) ~[spymemcached-2.8.4.jar:2.8.4]
at net.spy.memcached.MemcachedClient.asyncGet(MemcachedClient.java:841) ~[spymemcached-2.8.4.jar:2.8.4]
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1003) ~[spymemcached-2.8.4.jar:2.8.4]
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1024) ~[spymemcached-2.8.4.jar:2.8.4]

It happens everytime I change the code in debug mode (JVM code injection)

From thread: http://www.couchbase.com/forums/thread/java-client-issue-after-code-injection

 Comments   
Comment by Michael Nitschinger [ 16/Nov/12 ]
Asked "cb" on the forums for more information to be able to reproduce the issue.
Comment by cb [ 23/Dec/12 ]
I tested on v1.1 and there are still issues:

Caused by: java.lang.IllegalStateException: Shutting down
at net.spy.memcached.MemcachedConnection.checkState(MemcachedConnection.java:825) ~[spymemcached-2.8.9.jar:2.8.9]
at net.spy.memcached.MemcachedConnection.enqueueOperation(MemcachedConnection.java:641) ~[spymemcached-2.8.9.jar:2.8.9]
at net.spy.memcached.MemcachedClient.asyncGetAndTouch(MemcachedClient.java:1284) ~[spymemcached-2.8.9.jar:2.8.9]
at net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:955) ~[spymemcached-2.8.9.jar:2.8.9]
at net.spy.memcached.MemcachedClient.getAndTouch(MemcachedClient.java:978) ~[spymemcached-2.8.9.jar:2.8.9]
at models.AdminUserSession.findByKey(AdminUserSession.java:35) ~[classes/:na]
Comment by Michael Nitschinger [ 24/Dec/12 ]
Hi cb,

thanks for responding. Would it be possible to help me out reproducing it? What kind of setup are you using? (OS, IDE, App Server) and so on? I never encountered this in my environments here... thanks very much!
Comment by cb [ 24/Dec/12 ]
Play Framewok 1.0.4
Couchbase 2 GA
Java Client 1.1

You can use the example app that you created in your Blog Post. (BTW - Still looking forward for the rest of the parts of this super good post!!)
Change something in the code - while running in sbt like this:

~run (note the "~") which tells sbt to inject any change into the running process.

Let me know if you can reproduce the bug.

Comment by Michael Nitschinger [ 24/Dec/12 ]
You mean 2.0.4?

Are you running on 1.6 or 1.7 java? Or which scala version? On which operating system are you running it?

Never came across this (neither while working on the blog posts nor working on it from other apps).

Comment by cb [ 24/Dec/12 ]
sorry 2.0.4

Jave Version:

Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-11M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)

OSX 10.8.2




[JCBC-115] thoroughly test on_error arguments Created: 18/Sep/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Support for on_error was recently added, but it does does not currently have a test for on_error=stop.

 Comments   
Comment by Michael Nitschinger [ 09/Nov/12 ]
Mark,

Can you add this to your sdkd testsuite? Its hard to deal with this in unit tests (would require lots of mocking) and I assume you can shutdown nodes and verify the behavior is correct?

Thanks,
Michael
Comment by Michael Nitschinger [ 29/Nov/12 ]
We can do this in functional tests as well, there are easy ways to produce errors on the server.
Comment by Mark Nunberg [ 29/Nov/12 ]
Please share :)




[JCBC-114] Command Futures never receive results after rebalance-out (or other sorts of topology/network changes) Created: 17/Sep/12  Updated: 20/May/13

Status: Reopened
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.0.3
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on JCBC-117 mention that OperationFuture.get(tmo)... Reopened

 Comments   
Comment by Mark Nunberg [ 03/Oct/12 ]
This is a real blocker, and seems to be related to a few vbuckets. This issue is preventing me from properly measuring command durations
Comment by Farshid Ghods [ 03/Oct/12 ]
Matt/Rags,

This issue is a blocker for executing more integration tests on java sdk. are there workarounds to avoid this use case or a fix on the way ?
Please assign this back to Mark if more information or logs needed for this issue
Comment by Matt Ingenthron [ 04/Oct/12 ]
Please have a look at this.
Comment by Mark Nunberg [ 05/Oct/12 ]
Michael,

I would not try this test manually.. the use case in more detail is as follows:

- Single CouchbaseClient object
- 20 user threads. 10 setting and 10 getting the same sorts of kv
- Operations are done asynchronously. They are submitted into a queue which is then checked periodically for isDone/isCancelled.
- 4 node cluster. Nodes are removed, connections are broken

The issue is those polling methods never returning true, unless they are retrieved synchronously (i.e. ft.get()).. which is actually an accidental detail
Comment by Matt Ingenthron [ 24/Oct/12 ]
We looked at this pretty closely today. The issue here is that the client as designed relies on the get() from the caller to trigger the timeout. An operation will, somewhat correctly, never transition to isDone() or isCancelled() unless someone cares to use it.

The scenario that was likely in play over the WAN here is that the request was in flight to the server while the config was in flight down to the client. It arrives at the server, but is never responded to. Since the get() is never called, it'll never time out and transition to the canceled state.

We recommend you change the test code to use the queue more like a queue and just get() each one. Iterating through the queue is a bit funny in the first place, but if using the get() on the Future objects, you'll still have asynchronous behavior and much of the time the get() will be returning since the data is already there.
Comment by Matt Ingenthron [ 24/Oct/12 ]
This behavior should be better documented, both in the javadoc and in the API reference.




[JCBC-151] Client does timeout on connect in specific java environments (was believed to be java7 related). Created: 21/Nov/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Flagged:
Release Note

 Description   
People report that the Client does not work with 1.7. Here is a sample stack trace:

2012-11-20 00:29:11.228 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-20 00:29:11.240 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@3f757322
2012-11-20 00:29:11.502 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-20 00:29:11.505 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-20 00:29:11.647 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-20 00:29:12.051 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-20 00:29:12.060 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
2012-11-20 00:29:17.111 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [localhost/127.0.0.1:8092(closed)]
java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:68)
at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:428)
at Example1.main(Example1.java:43)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:80)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:65)
... 2 more
2012-11-20 00:30:12.168 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-20 00:30:12.177 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Disconnected from the target VM, address: '127.0.0.1:65280', transport: 'socket'

Process finished with exit code 0


Also See http://www.couchbase.com/forums/thread/java-asyncconnectionmanager-timed-out-waiting-operation-please-help-console-log-included
And http://stackoverflow.com/questions/13466010/using-java-api-in-scala-to-query-views-in-couchbase-throws-timeout-exception?utm_source=twitterfeed&utm_medium=twitter

 Comments   
Comment by Quinn Slack [ 27/Nov/12 ]
This doesn't appear to be strictly related to Java7. I was able to get similar code to work on Java7. I even simulated loading a Play2 environment with other JARs that could potentially cause conflicts. It's possible that my simulation of the Play2 environment was insufficient and that Play2 does other stuff...

Code at https://github.com/sqs/couchbase-scala-example. Run with:

CBURL=http://localhost:8091/pools CBPASSWORD=mypassword sbt -Dconfig.file=conf/application.conf '~run'

Here is the output on my system that shows it's on Java7 (openjdk) and that shows the expected output from the views:

on java version 1.7.0_09
[info] play - Application started (Prod)
2012-11-27 01:34:27.115 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-27 01:34:27.120 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@6267e5a2
2012-11-27 01:34:27.147 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-27 01:34:27.149 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-27 01:34:27.176 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-27 01:34:27.282 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-27 01:34:27.288 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
Res = List(com.couchbase.client.protocol.views.ViewRowNoDocs@4667820f, com.couchbase.client.protocol.views.ViewRowNoDocs@358bcae5, com.couchbase.client.protocol.views.ViewRowNoDocs@6cb59bd9, com.couchbase.client.protocol.views.ViewRowNoDocs@70afb51, com.couchbase.client.protocol.views.ViewRowNoDocs@61f98673)
2012-11-27 01:34:27.396 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-27 01:34:27.403 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Comment by Michael Nitschinger [ 27/Nov/12 ]
Are you running on jdk7 on mac? Also, when you use play 2.1 it should work. Play2-related issues more were because of netty version incompatibilities.
Comment by Quinn Slack [ 27/Nov/12 ]
This is jdk7 on Arch Linux and Play 2.1-RC1.

I tried using a different netty, but no luck. What worked for me was moving calls to "new CouchbaseClient" out of static "object" initializers, since they appeared to be getting called in the Netty I/O thread (I got this exception in new CouchbaseClient: java.lang.IllegalStateException: await*() in I/O thread causes a dead lock or sudden performance drop. Use addListener() instead or call await*() from a different thread).
Comment by Michael Nitschinger [ 28/Nov/12 ]
Hi Quinn, this issue is a different one! The issue described here is that people seem to find connection timeouts with java 7.
Comment by Quinn Slack [ 28/Nov/12 ]
They may be related--or may just have similar symptoms. I was not getting timeout exceptions, but I was seeing view requests hang indefinitely until I made the fix described above.
Comment by dragos [ 13/Mar/13 ]
Hi Michael, can you help with this error ? I am trying to find a workaround or something to work. This is the only thing which is keeping us from using Couchbase. I also posted here : http://www.couchbase.com/forums/thread/couchbase-connectivity-problem-aws-vpc . As you can see in the times the timeout is received really fast.

2013-03-13 11:40:47.702 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.X.XXX:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 11:40:52.712 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@5cbfe9d
2013-03-13 11:40:52.840 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.X.XXX/10.0.X.XXX:11210
2013-03-13 11:40:57.860 INFO com.couchbase.client.ViewConnection: Added 10.0.X.XXX to connect queue
2013-03-13 11:40:57.863 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 11:40:58.070 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 11:40:58.077 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.X.XXX:8092 - Session request successful
2013-03-13 11:41:03.113 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.X.XXX/10.0.X.XXX:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:63)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi dragos,

can you please post the full code where you connect to couchbase? (including factories and such)

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Mike, this is the POC client for Couchbase:
package couchbase.training.view.poc;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URI;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.TimeUnit;

import org.apache.commons.io.IOUtils;
import org.json.JSONException;
import org.json.XML;

import com.couchbase.client.CouchbaseClient;
import com.couchbase.client.CouchbaseConnectionFactory;
import com.couchbase.client.CouchbaseConnectionFactoryBuilder;
import com.couchbase.client.protocol.views.ComplexKey;
import com.couchbase.client.protocol.views.Query;
import com.couchbase.client.protocol.views.Stale;
import com.couchbase.client.protocol.views.View;
import com.couchbase.client.protocol.views.ViewResponse;
import com.couchbase.client.protocol.views.ViewRow;

public class CouchbasePOC
{
public static void main(String[] args) throws FileNotFoundException, JSONException, IOException
{
System.out.println("Hello from CouchBase");

// Set the URIs and get a client
final List<URI> uris = new LinkedList<URI>();

// Connect to localhost or to the appropriate URI(s)
uris.add(URI.create("http://10.0.7.164:8091/pools"));

final String mappedName="problems";
long start = System.currentTimeMillis();

CouchbaseConnectionFactoryBuilder m_couchbaseConnectionFactoryBuilder = new CouchbaseConnectionFactoryBuilder();
m_couchbaseConnectionFactoryBuilder.setMaxReconnectDelay(10000);
m_couchbaseConnectionFactoryBuilder.setOpQueueMaxBlockTime(100);
m_couchbaseConnectionFactoryBuilder.setOpTimeout(20000);
m_couchbaseConnectionFactoryBuilder.setShouldOptimize(true);
m_couchbaseConnectionFactoryBuilder.setViewTimeout(300000);

CouchbaseConnectionFactory connFactory = m_couchbaseConnectionFactoryBuilder.buildCouchbaseConnection(uris, mappedName, "");

CouchbaseClient client = null;
try
{
client = new CouchbaseClient(connFactory);
}
catch (IOException e)
{
System.err.println("IOException connecting to Couchbase: " + e.getMessage());
System.exit(1);
}

final View view = client.getView(mappedName, mappedName);
final Query query = new Query();

final ComplexKey key = ComplexKey.of("problem",null,null,null,null);
query.setKey(key);
query.setStale( Stale.UPDATE_AFTER );
int counter=0;
try
{
final ViewResponse result = client.query(view, query);
for(ViewRow row : result)
{
counter++;
System.out.println(row.getId());
}
}
catch(java.lang.VerifyError e)
{
System.out.println("Error: " + e.getMessage());
}

System.out.println("time: " + (System.currentTimeMillis()-start) );
System.out.println("documents: " + counter );

client.shutdown(3, TimeUnit.SECONDS);
System.exit(0);
}
}
Comment by dragos [ 13/Mar/13 ]
I hope you would not mind that i called you Mike and not Michael.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hehe no worries.

Can you do me a favor and remove the factory for a second and just use CouchbaseClient() without anything else? Let me know if the problem still persists or not! Thanks
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
First i want to thank you for the quick feed-back. Your owesome !!!

Before using the factory i used the CouchbaseClient(uris, bucketName, "") which is also timing out. I added the factory to be able increase the timeout times for the connection but still the same problem.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hmm okay, so its not the factory (we had some issues with default values in the past, thats why I'm asking).

Can you pin me down your operating system and exact java version you're using?Thanks for your help!
Comment by Michael Nitschinger [ 13/Mar/13 ]
Also, since you're so responsive (I hope we can pin this down finally!).. Can you run the same code with DEBUG log enabled (see https://code.google.com/p/spymemcached/wiki/Logging).. just use Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST); instead of Logger.getLogger("net.spy.memcached").setLevel(Level.FINEST);

Thanks!
Comment by dragos [ 13/Mar/13 ]
This is my test environment for the client which will be similar for production.
OS Ubuntu Server 11.10 x64
java -version result:
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

My network infrastructure is like this:
In an AWS virtual private center (VPC) i have a subnet and in this subnet there are 2 machines. The client machine and the Couchbase server. The machines have identical OS and java version as mentioned above. I have open the ports and also checked the network ACL. The traffic is free to go,no restriction. If i am using a rest query with curl for the view i receive the result, the problem is only in the java SDK.
If needed more information please ask. I will gladly offer it.

Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

yes, if you can please rerun the script with DEBUG logging enabled (see last comment).. Thanks very much in advance!
Comment by dragos [ 13/Mar/13 ]
I've setted the logger to finest and this is the error log obtained. Also i tested again the connection with curl and no problemes.

Mar 13, 2013 1:30:14 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
2013-03-13 13:30:14.679 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 13:30:19.689 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@4e99353f
2013-03-13 13:30:19.810 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.7.164/10.0.7.164:11210
2013-03-13 13:30:24.828 INFO com.couchbase.client.ViewConnection: Added 10.0.7.164 to connect queue
2013-03-13 13:30:24.830 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 13:30:25.025 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 13:30:25.033 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.7.164:8092 - Session request successful
2013-03-13 13:30:30.083 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:68)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
        ... 2 more

curl result

curl http://10.0.7.164:8092/problems/_design/dev_problems/_view/problems?stale=false\&connection_timeout=60000\&limit=10\&skip=0
{"total_rows":0,"rows":[
]
}
[4]+ Done
Comment by dragos [ 13/Mar/13 ]
I am available for next steps in debuging. With what can i help next ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, hmm no there should be more output (DEBUG), not just info... you need to copy this whole snippet before your code actually executes:

        // Tell spy to use the SunLogger
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);


I guess you're running it from your IDE?
Comment by dragos [ 13/Mar/13 ]
I am building the client on local IDE then uploading it to EC2 test machine. The snipped of code word and now i have a full log.

Mar 13, 2013 1:53:10 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default?uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default/buckets?v=120523822&uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM net.spy.memcached.MemcachedConnection createConnections
INFO: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Updating nodesMap in VBucketNodeLocator.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Adding node with address 10.0.7.164:11210.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Node added is {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8}.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=false, c=true, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
INFO: Connection state changed for sun.nio.ch.SelectionKeyImpl@2c76e369
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added Cmd: 10 Opaque: 1 to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=1, topRop=null, topWop=null, toWrite=0, interested=8}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 24 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: Cmd: 10 Opaque: 1 and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added SASL auth operation to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=true, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=true, w=false, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=1, #Wops=0, #iq=0, topRop=SASL auth operation, topWop=null, toWrite=0, interested=1})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 37 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.auth.AuthThread$1 receivedStatus
INFO: Authenticated to 10.0.7.164/10.0.7.164:11210
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: SASL auth operation and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewConnection createConnections
INFO: Added 10.0.7.164 to connect queue
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseClient <init>
INFO: viewmode property isn't defined. Setting viewmode to production mode
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP subscribe
FINE: Subscribing an object for reconfiguration updates com.couchbase.client.CouchbaseClient
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686] OPEN


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is true and Channel State is OPEN.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] BOUND: /10.0.7.213:55729


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.213:55729 and Channel State is BOUND.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] CONNECTED: /10.0.7.164:8091


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.164:8091 and Channel State is CONNECTED.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: STATUS: 200 OK
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: VERSION: HTTP/1.1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Cache-Control = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Content-Type = application/json; charset=utf-8
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Date = Wed, 13 Mar 2013 13:55:42 GMT
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Pragma = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Server = Couchbase Server 2.0.0-1976-rel-enterprise
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Transfer-Encoding = chunked
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER:

Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: CHUNKED CONTENT {
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: {"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3066
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 2048
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: ],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3072
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 361
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketMonitor logFiner
FINER: Getting server list returns this last chunked response:
{"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@4cf3fdc4
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@fadb83cf
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager processConnectionRequests
INFO: Opening new Couchbase HTTP connection
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback completed
INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, thanks for the log..

interestingly, there is nothing unusual in this.. I'll investigate further and come back to you if I need more info..

thanks so far!
Comment by dragos [ 13/Mar/13 ]
Thank you for spending time on this. Looking forward for a fix :) .
Comment by Michael Nitschinger [ 13/Mar/13 ]
interestingly, the connection is open and then times out...

INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]

this sounds highly suspicious to me, maybe this is a bug in apache httpcore-nio...
Comment by dragos [ 13/Mar/13 ]
The time out is displayed after 5 seconds. As you suggested i removed the factory and the detailed logged above i obtained with using CouchbaseClient(uris, mappedName, ""). Do you have some way of testing the httpCore-nio ? Should i try to look for latest version of httpCore-nio.. ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi,

yes you could try that.. I ran the test suite with 4.2.3 which is the latest one and there were no errors, so our code should work with it. You need to exchange the httpcore and httpcore-nio with the latest 4.2.3 jars

I don't think this is 100% the issue, but it helps us get rid of one more factor!

Thanks!
Michael
Comment by Michael Nitschinger [ 13/Mar/13 ]
Okay, the 5 second period can come from this:

 HttpParams params = new SyncBasicHttpParams();
      params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 5000)
          .setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 5000)
....


SO_TIMEOUT is: Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).

and CONNECTION_TIMEOUT is: Determines the timeout in milliseconds until a connection is established.


Dragos, can it be the case that it takes more than 5 seconds to return a result when using curl or so? That would explain the "fail fast" timeout here.. not saying that these times are okay this way, just to find the root cause..

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
I've made a test with httpcore-nio-4.2.jar and with httpcore-4.2.1.jar and your intuition is good. The httpcore is not the problem because the timeout is still there. With curl the results is back fast, in less then half a second. So the problem must be somewhere in the java SDK in the way the view is retrieved.
Comment by dragos [ 13/Mar/13 ]
I've moved the Couchbase server to the same machine as the client to have all in same environment for testing. Now all is working fine but it is not possible for our product to stay on the same machine as Couchbase server for obvious reasons. In production we will have more then one app machine which we will try to access the Couchbase cluster.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

okay so its definitely something with those timeouts.. The thing is we can increase them for sure, but the question is if it would work for your application given the long network latency - and if your application stays performant that way.

When you're running curl on a request, how long does it take?
Comment by dragos [ 14/Mar/13 ]
The curl request is very fast (a few milliseconds, and less then 0.5 seconds i am sure) and the network latency it is not a problem. Some how the couchbase client is taking to much time connecting. When it need to connect to a network machine in the same lan it takes more then 10 seconds. I experimented this in EC2 environment and on my local machine with a VM. If couchbase is local with the test app, the couchbase client is connecting fast.
Do you have any points on how a connection should be setup for couchbase client ? I am trying to use the factory builder and the connection factory from java sdk but still with no success.
Comment by Tim Smith [ 14/Mar/13 ]
Dragos,

I can think of two separate things to try. One is to increase the SO_TIMEOUT to 8000, CONNECTION_TIMEOUT to 12000. Identify what's being hit.

Probably more effective would be to use Wireshark to capture all traffic on the client machine, and see what is really happening. Start the capture (with tshark command-line tool, for example), then run the test client, let it time out, then run the curl command, let it succeed, then stop the capture. Gzip the capture data and attach it to this issue.

Tim
Comment by dragos [ 14/Mar/13 ]
Hi Tim,
I've started some load tests to be able to evaluate Couchbase, so this means i moved Couchbase server on the same machine as the load test client. I tried to use a pool of couchbase clients always connected to the server, and after a while the test failed because couldn't create more connections. From my initial research i seen that couchbase client is managing his own connection can you tell me how this is done or what are the policies of shutting down connections ? Can you point me to some best practices for couchbase client connectivity ?
Comment by Tim Smith [ 14/Mar/13 ]
I will handle that question about connection pooling, etc., separately, since it is unrelated to this bug about timeouts.

On this specific bug report, the current state as I understand it is that: curl returns very quickly with the correct results. The Java client times out after 5 seconds without returning the results.

My suggestion is to set up a very simple Java client that performs a single view query, and to get a packet capture of both curl and the Java client, to identify what the two are doing differently and why the curl succeeds but the Java client fails.
Comment by Jahangir Zinedine [ 27/Mar/13 ]
Hi Michael,

Any update on this issue?
I faced the same issue and here is my environment:
JRuby 1.7.3
Oracle JDK 1.7(with 1.6 the same thing happened)
Couchbase 2.0.1
Java SDK 1.1.3(with 1.1.4 the same thing happened)
And latest netty and http-core and other jar files.

Appreciate your help.

Thanks,
Jani
Comment by dragos [ 01/Apr/13 ]
Hi All,
Finally after a long time i found the workaround needed for the java client to not timeout. As has been suggested by the Couchbase team to use a host name for the Couchabase server i applied the same fix but to the client machine. Meaning on the client machine were the java Couchbase client was timing out ( in Amazon VPC ) we added a hostname for the Couchbase server. Our test URI from http://10.0.7.164:8091/pools has become http://couchbase:8091/pools. This workaround is also fixing a 10 seconds connectivity for the client.

On Ubuntu the hostname file is at: /etc/hosts
On Windows 7 the hostname file is at: C:\Windows\system32\drivers\etc\hosts

Example of host name entry in client machine:
10.0.7.146 couchbase
Comment by Michael Nitschinger [ 02/Apr/13 ]
Thanks very much dragos for tracking this down!

Does this mean that in your environment after the change, you're not seeing any performance impact anymore? Everything works as expected and performant?

Thanks!
Comment by dragos [ 02/Apr/13 ]
Hi Michael,
In this moment the client does not time out and the time taken by java couchbase client to connect takes around 1.5 seconds from 10 seconds which we can consider a normal behavior. As for performance with this workaround we can continue the POC and evaluate the performance. As a first impression i can say that we see a good speed but i will keep my reservation until final POC is tested and we can achieve a full load test.

What i want to emphasize that this is a welcomed workaround but it is not a solution on a longer term for our cloud production site. In a scenario where we need to create new machines and maybe new couchbase clusters it involves manual or specific automatic changes to the hosts file and of course extra managing of this. Bottom line is that we are waiting the fix on the couchbase java sdk 1.1.5 which will make this workaround obsolete.

Also i feel the need to mention that this workaround was found in collaboration with the Couchbase team which provided the basic information for the workaround to be found, so thank you guys.
Comment by Michael Nitschinger [ 02/Apr/13 ]
Hi Dragos,

I'm not sure if there is a thing that we can fix inside the client when this is a DNS/hostname resolution issue with the OS and amazon?

We would definitely need to investigate further, but all that the client does is try to use the hostname/IP provided during bootstrap. If nothing comes back or the host can't be found, we can't do much about it. Anyway, I think we need to do some more investigation here to really see whats the problem.




[JCBC-138] Java Client does not recover when only bootstrap node provided and failovered Created: 05/Nov/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1-dp3, 1.1-dp4
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Within the current implementation, when the bootstrap node is failovered/removed (and only that one is provided in the list) it gets correctly removed from the SDK, but either a new try to get a different streaming connection fails or something else is wrong.

Even when the node is added back into the cluster no new view connection is created (most arguably because no map updates are received anymore).

This issue may also be the cause for other bugs related to failover scenarios reported.




[JCBC-169] add overloaded methods for various types to the Query class Created: 06/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, all types other than strings must go through the ComplexKey class. This may not be the best approach since it leads to a bit of oddness with needing to tell the ComplexKey class to handle a single element differently.

One alternative is to improve the Query class to handle other types, then deprecate the special handling in the ComplexKey class and change the behavior for a single element. This would be a small API change, but maybe small enough that with sufficient warning, we can do it in a minor release.




[JCBC-164] Support CAS for delete operation Created: 04/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1-beta
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: schp Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-193 Allow CAS with delete Closed

 Description   
It would be nice if the DELETE operation would support a CAS parameter, a feature which is available in e. g. the Ruby client library. Without the possibility for checking for a given CAS value it may happen that a DELETE removes a key-value pair which was just updated by an other thread.

 Comments   
Comment by Michael Nitschinger [ 04/Dec/12 ]
Thanks for your input!

I'll discuss this and then get back to you in this ticket!
Comment by Michael Nitschinger [ 19/Dec/12 ]
What do you think about this?
Comment by Matt Ingenthron [ 19/Dec/12 ]
I do think it makes sense, and it's in there actually for operations with durability requirements.




[JCBC-161] Remove Duplication in Observe checks Created: 03/Dec/12  Updated: 20/May/13

Status: In Progress
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The fix for JCBC-148 works, but it checks against all nodes in the cluster. If you have a larger cluster, you have much more nodes than there could be assigned to a specific key. As an enhancement, the code in checkConfigAgainstPersistence should check against the nodes for the given key, not for everything.

 Comments   
Comment by Michael Nitschinger [ 13/May/13 ]
http://review.couchbase.org/#/c/26261/
Comment by Michael Nitschinger [ 13/May/13 ]
Note that the new mechanism was implemented in a different changeset, and by now we had both in place behind one after another. since the other one was more accurate, the general one will be removed in favor of the other one.




[JCBC-166] sync up the getting started guide on the web page to the docs Created: 05/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
http://www.couchbase.com/develop/java/current should be a perfect subset of the getting started in the docs repo




[JCBC-184] API reference should show return types Created: 14/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: 8h
Time Spent: Not Specified
Original Estimate: 8h
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-summary.html


 Description   
The return type is critical for understanding how to use a function. In particular with the Java client library, it's critical to know if a function returns a Future (i.e., it's an async function) or not. Otherwise it's impossible to use it correctly.

Currently one must click through, scroll around, and pick out the return value somewhere in a list of parameters on the details page. It would be worth the extra screen real estate on the api reference page to show the return value as well.






[JCBC-183] Many methods in api reference don't link to corresponding documentation page Created: 14/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: 0.5h
Time Spent: Not Specified
Original Estimate: 0.5h
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-summary.html


 Description   
Many of the links in the api-reference-summary.html don't go anywhere. They should all link to the detail page for the particular method.

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-summary.html

It *may* be just the ones with durability requirements (persistto, replicateto) in the signature.




[JCBC-196] append/incr/decr/add/replace should not require CAS value to be passed Created: 21/Jun/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs, library
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-143 Prepend should not require CAS Resolved

 Description   
In the current implementation, there is no method for a non-CAS version of the append/prepend/incr/decr/add/replace operations. As per the memcached protocol, these operations do not require (but may be used with) a CAS id and requiring one greatly reduces the performance available by requiring a gets prior to the operation.

Perry

 Comments   
Comment by Raghavan Srinivas [ 22/Aug/12 ]
The incr, decr, add and replace do have calls that do not require CAS Id. For the append and prepend methods

you could do something like this

     c.prepend(0, "prepend-key", "value");

which would not require a CAS to be supplied/fetched. Will this suffice for now?

My suggestion would be to close this issue and open a separate issue for just append and prepend if it's still deemed a bug.

 
Comment by Michael Nitschinger [ 21/Dec/12 ]
http://review.couchbase.org/#/c/23486/
Comment by Perry Krug [ 21/Dec/12 ]
This will need a docs update as well...
Comment by Perry Krug [ 06/Feb/13 ]
Michael, just to add here, I think it would be sufficient to document this rather than changing the API itself. Documenting that you can (and should!) pass 0 for the CAS value when using these operations.




[JCBC-173] flush will not work owing to MB-7381 Created: 07/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Task Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on MB-7381 [2.0.1RN]REST API for flush requires ... Resolved
Duplicate
is duplicated by JCBC-242 The flush() operations always fails w... Resolved
Flagged:
Release Note

 Description   
During development of 1.1, RESTful flush was written, but it turns out that there is an issue with the server implementation of flush, and it requires Administrator level credentials. This makes it impractical for clients that don't have those credentials, like a default CouchbaseClient.

This means the flush implementation is done, but won't work for now.

 Comments   
Comment by Matt Ingenthron [ 07/Dec/12 ]
Using the ClusterManager interface, one can work around this issue for now if using the Administrator creds is acceptable.




[JCBC-189] Views having odd timeout issues on some clusters Created: 18/Dec/12  Updated: 20/May/13

Status: In Progress
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File failover-debug.args     Text File fo-dbg.log.txt     File vm-4nodes-2.ini    
Issue Links:
Dependency
depends on MB-7661 View query on Rebalance.Out fails wit... Open
blocks JCBC-176 Using BucketTool helper - error in co... Resolved

 Description   
We're seeing really strange timeout issues which seem to affect only this specific cluster and only in terms of views. We've re-installed this cluster time and time again, and we've had similar configurations run successfully as well.

More details in comment..

 Comments   
Comment by Mark Nunberg [ 18/Dec/12 ]
Just try to do this with a simple java script (i.e. connect and query the view)
Comment by Deepti Dawar [ 20/Dec/12 ]
This issue persists when the view retrieval is ran from standalone java program as well.
I tried this on both the VMs - 10.3.3.203, 10.3.3.209.
Comment by Deepti Dawar [ 20/Dec/12 ]
Its working fine for server deployed on localhost.
Comment by Matt Ingenthron [ 02/Jan/13 ]
Michael: found out today that this is a large issue for SDKQE. Can you have a quick look at this in the next day? You may find the underlying issue.
Comment by Michael Nitschinger [ 03/Jan/13 ]
Please pass me the script as commented and then (or if you can't) please assign it back to me! Thanks
Comment by Michael Nitschinger [ 18/Jan/13 ]
According to the posted logs, it looks like debugging was not turned on. Can you please run this again with debugging turned on? To get the full logs to STDOUT, use this before initializing the CouchbaseClient in The App:

      // Tell spy to use the SunLogger
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);


Would be great if we can get all the output so we can investigate where the timeouts come from. I'm sure with a debug log it will be much easier. Thanks!
Comment by Mark Nunberg [ 23/Jan/13 ]
So this bug isn't such an "unknown" anymore, and has exposed itself in 1.1.0 as well as 1.1.1 and isn't limited to particular clusters - (it just seems that some clusters are more likely than others to trigger this bug).

However this still needs a lot of care and analysis
Comment by Michael Nitschinger [ 01/Feb/13 ]
This is "kinda" blocker.




[JCBC-171] document how to set config options in xml config files, and all config options available Created: 06/Dec/12  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.0.3
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: 80h
Time Spent: Not Specified
Original Estimate: 80h


 Description   
I can't find any info on our web site, docs, tutorials, etc. for how how to set config options for Java client usage in an xml config file. Need examples of that, and exhaustive reference of all the options that can go in a config, along the lines of what we have for the CouchbaseConnectionFactoryBuilder class at:

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-ccfb.html

Also, it's very confusing to have 1.1 docs with useful info that applies to 1.0 docs, too, but have the 1.0 docs not updated. For example, there's no mention in the 1.0 docs of any of the stuff that is mentioned on the 1.1 sdk-ccfb page, even though lots of that would be relevant to 1.0 too.






[JCBC-206] Need clear info and examples on proper error handling Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/getting-started.html

Issue Links:
Duplicate
is duplicated by JCBC-236 Error handling documentation Resolved

 Description   
It's very difficult for people to understand the error handling behavior of the client library, and how to properly handle errors. The async methods make it even more difficult. Information is spread across multiple sections, hidden in individual method descriptions, or else not mentioned at all. The docs should provide a clear description of how to use the API correctly, including example code that does correct error handling that can be cut-and-pasted.

There is a brief mention of error handling here, but it's hidden and very cursory, with no example code:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/create-update-docs.html

This doc mentions that .get() returns null on failure, but gives no hints on how to determine why, or what the right way to recover is:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/read-docs.html

This gives a very cursory mention of failure, but no way to get further details:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/delete-docs.html

No mention of JSON encoding errors or what to expect:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/json-handling-docs.html

Mentions that OperationFuture may return false even if operation succeeds due to durability requirement failure, but no clear example of how to tell the difference, what status codes to expect in each case, etc.:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/advanced-persistence.html

Mentions that synchronous call may return an exception, but no info on what the exception would be, or in what circumstances it may return null with no exception:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-summary-synchronous.html

No mention of .isSuccess() method, or any info on how errors are reported via *Future objects. Also no mention of when the async method may fail immediately vs. when the failure will be reported only after checking .isSuccess() or another method on the future. No mention of what errors are reported as exceptions vs. what errors are reported only via the future object's methods.:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-summary-asynchronous.html

No mention of when the constructor might fail or what exceptions might be thrown:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-connection.html

No mention of how to tell if the FALSE result is due to timeout, impossible persistence requirements, or other problem:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-set-observe.html

Not clear enough that .add(), .set(), .replace() are an async methods, too easy for new user to misunderstand how to use them:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-set-add.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-set-set.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-replace.html

No mention that .get() may throw an exception, or how errors other than NotFound may be reported:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-get.html

Yay, here is an example that has a bit of error handling! Unfortunately, it is a "catch(Exception)" (i.e., very generic, and not intended to catch Couchbase-specific errors or timeouts), and no indication of how to handle actual Couchbase errors. Also gives false impression that get() will throw an exception on all failure, when in fact you need to check for failure with .isSuccess() or something.:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-get-async.html

Also on that page (async get), mentions that a TimeoutException may be thrown. But that won't happen on asyncGet() call, instead it will be when checking .get() on the Future result. I think. It is welcome to list exceptions that can be thrown, but I think this one is incorrect.
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-get-async.html

.getBulk() and .asyncGetBulk() are complicated enough that they deserve their own example of proper error handling, and what happens when one server node is down and other nodes are up, so partial results are available, etc.:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-bulk.html

Nice that these pages mention what exceptions can be thrown, so I can infer how to tell a timeout from a not found, etc. But why only here? Shouldn't these exceptions be more generally described elsewhere as applying to all operations (RuntimeException, InterruptedException, OperationTimeoutException)?:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-get-and-lock.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-unlock.html

Append and Prepend should normally use 0 for the casunique, and then I would not expect an EXISTS error. The example code is clunky, less efficient (suggests doing a .gets() call just to get the cas id), and likely to fail much more often. The example code does not do proper retry handling on an EXISTS error. It does not show how to distinguish an EXISTS, NOTFOUND, or server down error.:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-append.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-prepend.html

View queries might time out, on all or some servers. Should document how the client should handle errors from view queries. Also, if client requests a view that doesn't exist, how is that handled:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-view.html



There is no mention of error handling in the "Using the APIs" nor "Java Troubleshooting" sections, either:

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-started.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-troubleshooting.html

There is a brief mention of RuntimeException being "returned" (should say "thrown") in the section on handling timeouts, but this is too buried and too specific to be of real use. And there is no example code:
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-troubleshooting.html

Also, the tutorial doesn't do error handling at all, for example:

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/managing-beers.html

Look for "client.set" and "client.query" on that page and you'll see it ignores error handling entirely. I think this is a mistake in tutorial code, it should be complete and correct, or at least it should provide a warning saying that there is no error handling, with a link to the docs that describe how to do proper error handling.



 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-205] docs for touch() should mention getAndTouch() Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-touch.html


 Description   
It would be nice for the touch() docs to mention getAndTouch() as a common optimization:

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-gat.html




[JCBC-203] append/prepend are ASYNC, and can specify 0 for casunique Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-update-append.html


 Description   
This is a two-for-one bug report for the docs. First, the return value of append/prepend is Future<Boolean>, not "Object ( Binary object )". These are async ops, and need to be documented as such.

Next, the append and prepend methods say that the casunique argument must be specified, but don't mention that it is OK to pass 0 here if you just want to append to whatever is there.

In fact, using 0 for casunique is going to be the normal use case for these functions, since memcached (and Couchbase) defines them to be atomic. It's too bad that spymemcached made this interface clunky, but we can at least document this normal use case.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-201] All async ops should link to the Asynchronous Operations page Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-set-add.html
http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-retrieve-get-async.html

Etc., there are a bunch of these.


 Description   
There are a lot of async operations. They return *Future<*>. They should all have a link back to the overview of how Asynchronous Operations work, at http://www.couchbase.com/docs/couchbase-sdk-java-1.1/couchbase-sdk-java-summary-asynchronous.html

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-199] constructor documented as client method returning "(none)" Created: 02/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Tim Smith Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-connection.html


 Description   
The constructor is documented to return "(none)", which isn't really correct.

http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-connection.html

Also, the docs make it look like it's a method call on an existing client object ("client.new"), which isn't accurate:

"client.new CouchbaseClient([ url ] [, urls ] [, username ] [, password ])"



 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-235] Write the Java on Linux using Eclipse section of the Essentials Guide Created: 04/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: MC Brown Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-243 Docs: Reference installation Resolved

 Description   
Write the Java on Linux using Eclipse section of the Essentials Guide

Needs to cover:

Basic Installation/Setup of Eclipse
Adding the Couchbase Java components to Eclipse
Writing your first (small) app using Couchbase within Eclipse

Submissions should be to MC, either through the couchbase/docs repo, or direct to MC in whatever format suits. Must include both the text and images.




[JCBC-234] Write the Java on Mac OS X using Eclipse section of the Essentials Guide Created: 04/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: MC Brown Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Write the Java on Mac OS X using Eclipse section of the Essentials Guide

Needs to cover:

Basic Installation/Setup of Eclipse
Adding the Couchbase Java components to Eclipse
Writing your first (small) app using Couchbase within Eclipse

Submissions should be to MC, either through the couchbase/docs repo, or direct to MC in whatever format suits. Must include both the text and images.





[JCBC-233] Write the Java on Windows with Eclipse installation instructions for essentials guide Created: 04/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: MC Brown Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Write the Java on Windows using Eclipse section of the Essentials Guide

Needs to cover:

Basic Installation/Setup of Eclipse
Adding the Couchbase Java components to Eclipse
Writing your first (small) app using Couchbase within Eclipse

Submissions should be to MC, either through the couchbase/docs repo, or direct to MC in whatever format suits. Must include both the text and images.





[JCBC-228] a no-args constructor and an init method are needed Created: 31/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, the CouchbaseClient does a number of things that are not so pretty, like spinning up a thread from it's ctor. It would be better to add an additional, optional no-args constructor which expects an init method to be called with the other params needed, start thread, etc.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Assigning towards 1.1.2 but we can defer it to 1.1.3 as well.




[JCBC-217] Create Source-Code Styleguide Created: 23/Jan/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.0
Fix Version/s: 1.1.7
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A style guide for all java applications should be developed and paired with checkstyle rules. If possible, also provide a howto on IDE integration and git hooks.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Moving to 1.1.2, depending on the size of the change maybe will be 1.1.3 or in between.




[JCBC-248] ensure there is a floor to tuneables such that they can't be set so a client misbehaves Created: 15/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.0.3, 1.1.0, 1.1.1, 1.1.2
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There have been a few situations where people have either inavertently or not knowing what they're doing set various internal timings to incorrect values. We should create a floor or a ceiling for these and change them with a warning if the user attempts to misuse it.

From a recent code example:
    private static String serverList = "";
    private static long opTimeout = -1;
    private static long opQueueMaxBlockTime = -1;
    private static long obsPollInterval = -1;
    private static int obsPollMax = -1;
    private static long msReconnectThresholdTime = -1;
    private static long maxReconnectDelay = -1;
    private static boolean shouldOptimize = false;
    private static int timeoutExceptionThreshold = -1;
    private static boolean useNagleAlgorithm = false;
    private static FailureMode failureMode = FailureMode.Cancel;
    private static int ttl = 86400;
    private static int threads = 1;
    private static long msBeforeGet = 5000l;
    private static long repeat = -1;






[JCBC-270] client does not handle failure of EPT node with memcached bucket Created: 13/Mar/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.2
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3 node cluster of 2.0, one memcached bucket authenticated, Couchbase Java Client 1.1.2 with required spymemcached 2.8.11

Attachments: Text File CBSE-472repro-2.txt    

 Description   
When investigating a case, it seems that the dropped configuration when using a memcached bucket is never reestablished. I'm not sure if this is because we rely on timeouts (which we won't get in this case) or something else.

Steps to reproduce:
1. Set up three node cluster
2. Run a constant workload, starting off of one of the nodes (192.168.1.200 in my config)
3. Remove that node from the cluster

Observed behavior:
The client sees the config dropped, but doesn't reconfigure.

Expected behavior:
Client bootstraps off of one of the other nodes.

Attached file shows the config log in this case.

 Comments   
Comment by Michael Nitschinger [ 14/Mar/13 ]
Hey Matt,

one quick update.. I don't know yet if its related, but you've been using the wrong netty version.. your logs show /Users/ingenthr/lib/netty-3.2.5.Final.jar' but the correct one is 3.5.5!

Since netty handles the streaming connection, this may be related - I dont know yet.
Comment by Michael Nitschinger [ 15/Mar/13 ]
Matt, with the change proposed in JCBC-271 and using Netty 3.5.5, I dont see this behaviour (anymore). I tried with 3.2.5 but my connections always go from unbound to connected after some time, even when I wildly failover/rebalance - as it should be. It even waits as we implemented it when I remove both EPT nodes for some time and then add them back it comes back nicely.

Can you try to repro with those two changes and if it still fails lets do a quick screen sharing.
Comment by Matt Ingenthron [ 18/Mar/13 ]
Indeed, I could not reproduce this one after moving to the right dependencies.




[JCBC-250] ClassCast Exception when using durability against memcached bucket Created: 20/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.2
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File 1.1.5-sdk.log     Text File 1.1.6-sdk.log    

 Description   
Apparently trying to use these operations:
CouchbaseClient.add(String key, int exp, String value, PersistTo req, ReplicateTo rep)

Against a memcached bucket causes an exception even if "req" and "rep" are set to 0. From the user's perspective, they expect this to work. Is it a major undertaking to make the classes compatible from the client side, independent of the bucket types and then let the operation succeed/fail when it can or can't?

java.lang.ClassCastException: com.couchbase.client.CouchbaseMemcachedConnection cannot be cast to com.couchbase.client.CouchbaseConnection
  at com.couchbase.client.CouchbaseClient.observePoll(CouchbaseClient.java:1708)
  at com.couchbase.client.CouchbaseClient.add(CouchbaseClient.java:1293)

 Comments   
Comment by Deepti Dawar [ 14/May/13 ]
This problem has been fixed in the version 1.1.6. It was there till 1.1.5 but now has been fixed.

Please find the same log of run in 1.1.5 and 1.1.6 to strengthen the findings.

Its good to close.

Thanks !




[JCBC-269] subsequent resubscribers should not run if a resubscriber is successful Created: 13/Mar/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.2
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When investigating a case recently, I noticed many resubscribers running once a second. Looking at the code, it appears the kind of threadpool being used would just queue more and more up, but only run one at a time.

The subsequent resubscribers should exit immediately if resubscribe is no longer needed. Or the threadpool could be changed so we can't queue another while one is running.




[JCBC-258] documentation needs to be much clearer about Query.setKey() requiring JSON encoded strings or use of ComplexKey Created: 06/Mar/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.3
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Neither the javadoc nor the existing documentation really cover the potential issues with setKey using a number or something that will autobox into a string value. This should be much clearer.




[JCBC-237] Docs: Document how to use observe properly outside of the "wrapped" methods Created: 05/Feb/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.1
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There will be some cases where a user wants to use the observe functionality outside of the set()/add()/replace() methods what we have allowed for them to specify the "durabililty requirements".

Specifcally this may come up when using incr/decr/prepend/append and wanting to "observe" on that...or just using observe manually to control the behavior, retry, timeouts, etc.






[JCBC-291] Authentication failed on 5-th client Created: 17/Apr/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Improvement Priority: Major
Reporter: Andrew Kulikov Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Debian GNU/Linux 6.0 (amd64), Couchbase community edition.


 Description   
Let's assume I have:
- a cluster made of 4 couchbase nodes (couchbase1-couchbase4);
- 10 instances of client application (client1-client10).

In my client Java application I connect to the cluster using login, password and the following server list string:

{code}
"http://couchbase1:8091/pools,http://couchbase2:8091/pools,http://couchbase3:8091/pools,http://couchbase4:8091/pools"
{code}

When I start four instances of client application on client1-client4 -- everything is going OK. But when I am trying to start 5-th instance of client application on client5 -- I get the following error:

{code}
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.564 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=couchbase1/192.168.0.129:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL steps operation, toWrite=0, interested=0}
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.565 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@28a3001c
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.565 WARN net.spy.memcached.auth.AuthThreadMonitor: Incomplete authentication interrupted for node {QA sa=couchbase1/192.168.0.129:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL steps operation, toWrite=0, interested=8}
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.565 WARN net.spy.memcached.auth.AuthThread: Authentication failed to couchbase1/192.168.0.129:11210
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.566 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to exception on {QA sa=couchbase1/192.168.0.129:11210, #Rops=1, #Wops=0, #iq=0, topRop=SASL auth operation, topWop=null, toWrite=0, interested=1}
Mar 29 12:56:28 client1: at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:453)
Mar 29 12:56:28 client1: at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:381)
Mar 29 12:56:28 client1: at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:243)
Mar 29 12:56:28 client1: at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:229)
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.566 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=couchbase1/192.168.0.129:11210, #Rops=1, #Wops=0, #iq=0, topRop=SASL auth operation, topWop=null, toWrite=0, interested=1}, attempt 0.
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.566 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: SASL auth operation
Mar 29 12:56:28 client1: 2013-03-29 12:56:28.666 WARN net.spy.memcached.auth.AuthThread: Authentication failed to couchbase1/192.168.0.129:11210
{code}

So, if I shutdown one of client1-client4 -- this moment client5 connects to the cluster without problems.

PS: No matter if I make cluster from 2 nodes. The behaveour is the same -- 5-th node get the same error.

 Comments   
Comment by Andrew Kulikov [ 20/May/13 ]
We have solved our problem! The reason was in limit for number of opened files. Have you ever tested couchbase with pam_limits enabled? If so, default configuration (nofile=1024) is suitable only for 3 clients simultaneously connected. 4-th client fails. We have greatly increased nofile parameter for couchbase user in limits.conf -- and the problem was solved.

I think you should add this issue in your documentation. Also, you should write an error in logfile -- if this limit (for open files or any other) is reached.
Comment by Michael Nitschinger [ 20/May/13 ]
Thanks for reporting your findings! I'll change it to a docs enhancement!




[JCBC-276] Client does not detect silently dying Streaming Node Created: 20/Mar/13  Updated: 20/May/13

Status: In Progress
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When connected to the EPT/streaming node and the node is "frozen" or dies silently otherwise (doesn't force the closing of the chunked socket), the connection stays established.

This can easily be reproduced outside of the client by connecting the browser to the streaming URL and then freezing a VM. The browser will still "spin" and wait for new chunks to come up.

The proposed solution is to have a netty handler in place that raises a exception when there is not traffic for N number of seconds (like 30) over the streaming connection. After this is detected, we have two possibilities:

- reconnect completely, but this involves lots of overhead every 30 seconds.
- send a HTTP HEAD packet and only if this doesnt work out reconnect. This means in the normal case we only have a HTTP HEAD request sent every 30 seconds, not much overhead.
If this fails, we then trigger the reconfigure.

Netty has a ReadTimeoutHandler to help with this. My POC already kinda works, I just need to find a way to properly distinguish the HEAD response on the ResponseHandler from regular chunks that arrive from the same channel.

 Comments   
Comment by Matt Ingenthron [ 20/Mar/13 ]
I had fixed this with couchbase buckets and we test for this in SDKQE. Is this possibly isolated to memcached buckets.
Comment by Michael Nitschinger [ 21/Mar/13 ]
Actually, this has been discovered while using Couchbase buckets. Let's chat about this, but I think thats a different issue and not related to it.
Comment by Matt Ingenthron [ 21/Mar/13 ]
Sure, the solution previously was to have that threshold if we were getting unexpected failures. Once we pass that threshold, we'd try to re-subscribe.

I'm not opposed to a heartbeat, but I don't think the HTTP HEAD is good, since the mochiweb erlang implementation is effectively the same as a GET. I'll reach you and chat through it.
Comment by Michael Nitschinger [ 21/Mar/13 ]
Okay after talking this through with Matt, I reran the script to see if the proposed solution (increasing ops/s) fixed the problem.

Interestingly, it turns out when doing set/get's it never hits our anticipated codepath but instead the get operations just time out, the threshold never gets increased and nothing happens. This is something we need to reinvestigate.




[JCBC-296] Document changing compression threshold Created: 06/May/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: docs
Affects Version/s: 1.1.5
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Major
Reporter: Mike Wiederhold Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Please see the feature request below.

http://www.couchbase.com/forums/thread/java-client-compression-threshold




[JCBC-287] Failover + Readd of Streaming Node against 1.8.1 fails Created: 18/Apr/13  Updated: 20/May/13

Status: Open
Project: Couchbase Java Client
Component/s: library
Affects Version/s: 1.1.5
Fix Version/s: 1.1.7
Security Level: Public

Type: Task Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Any Couchbase SDK against a 1.8 cluster


 Comments   
Comment by Michael Nitschinger [ 19/Apr/13 ]
Let me describe the observed behaviour in greater detail to clarify what is the issue.
I just verified against 2.0 that it's the same behaviour, but I don't know why wie didn't observe it there this way.

When someone clicks failover in the UI on the streaming node, our streaming connection gets closed. Now the client code is implemented that way that it gets back to the node list passed in and iterates over it. In the case observed it is the first node in the list which we failed over, but since it has not been removed yet the client is able to connect (and even retrieve the bucket information!).

Now comes the part that I don't understand fully. With the changeset not applied, it connects to the streaming connection from the failovered node but somehow it doesnt get the information fast enough when the same node (!) gets added back. Maybe because when the node gets rebalanced out the streaming connection gets closed again and we lag little bit behind to connect to the second node in the list.

That said, WITH the changeset applied we will not connect to the failovered node immediately because we recognize its in the cluster, but not part of the node list (which kinda sucks from a logical perspective in my opinion, because why do we close the streaming connection on failover AND allow to reestablish it afterwards?). So what will happen is that the SDK immediately connects to the second one and will observe the upcoming operations (rebalance out, rebalance in) correctly.

Does this make more sense now?
Comment by Matt Ingenthron [ 19/Apr/13 ]
Not 100%, no. Do we have a log that shows responses from nodes by chance?
Comment by Michael Nitschinger [ 19/Apr/13 ]
You mean the JSON responses from the nodes? Or the netty logs on the SDK side.
Comment by Michael Nitschinger [ 19/Apr/13 ]
So if you set "failover" on one of the nodes, all HTTP Rest resources are still 100% accessible, even if the streaming connection was closed from the server before. We can even establish it again.

The main diff is that the node shows up in pools, but not in the bucket node list. It also shows a "failovered" state, so we could also try to identify it at that point.

{
systemStats: {
cpu_utilization_rate: 1,
swap_total: 1069543424,
swap_used: 0
},
interestingStats: { },
uptime: "226",
memoryTotal: 1572306944,
memoryFree: 761069568,
mcdMemoryReserved: 1199,
mcdMemoryAllocated: 1199,
couchApiBase: "http://192.168.56.101:8092/",
clusterMembership: "inactiveFailed",
status: "healthy",
thisNode: true,
hostname: "192.168.56.101:8091",
clusterCompatibility: 131072,
version: "2.0.1-170-rel-enterprise",
os: "x86_64-unknown-linux-gnu",
ports: {
proxy: 11211,
direct: 11210
}
},

(this is pools) .. its not in the node list from the bucket info (see the clusterMembership part).

I can get you SDK debug logs as well, but they are already in the CBSE from mark.




[JCBC-282] Add new junit tests as per additions in the Test Plan Created: 09/Apr/13  Updated: 20/May/13  Resolved: 20/May/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Deepti Dawar Assignee: Deepti Dawar
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
Add new junit tests as per additions in the Test Plan


 Comments   
Comment by Deepti Dawar [ 09/Apr/13 ]
Two tests added for connection refused and network un-reachable.
Comment by Deepti Dawar [ 20/May/13 ]
http://review.couchbase.org/#/c/25563/




Generated at Sat May 25 17:07:22 CDT 2013 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.