[JCBC-479] ComparisonFailure in running unit tests for 2.0 client Created: 24/Jun/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Infrastructure
Affects Version/s: 2.0.0
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
1 warning
:processTestResources UP-TO-DATE
:testClasses
:test

com.couchbase.client.java.document.json.JsonObjectTest > shouldExportMix FAILED
    org.junit.ComparisonFailure at JsonObjectTest.java:27

com.couchbase.client.java.query.ViewQueryTest > shouldHandleEndKey FAILED
    org.junit.ComparisonFailure at ViewQueryTest.java:178

77 tests completed, 2 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///root/couchbase/sdkd-2.0/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED


 Comments   
Comment by Wei-Li Liu [ 15/Jul/14 ]
I am getting different failure running unit test

===================================================================================
1 warning
:processTestResources UP-TO-DATE
:testClasses
:test

com.couchbase.client.java.convert.JacksonJsonConverterTest > shouldEncodeSimpleJsonObject FAILED
    org.junit.ComparisonFailure at JacksonJsonConverterTest.java:70

com.couchbase.client.java.convert.JacksonJsonConverterTest > shouldEncodeMixedNestedJsonValues FAILED
    org.junit.ComparisonFailure at JacksonJsonConverterTest.java:177

77 tests completed, 2 failed
:test FAILED

FAILURE: Build failed with an exception.
Comment by Wei-Li Liu [ 15/Jul/14 ]
Failure running integration tests


============================================================================
wei-lis-mbp:couchbase-java-client wei-li$ ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava UP-TO-DATE
:processIntegrationResources UP-TO-DATE
:integrationClasses UP-TO-DATE
:integrationTest

com.couchbase.client.java.BinaryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.BinaryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.QueryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.QueryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.ViewTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.ViewTest > classMethod FAILED
    java.util.NoSuchElementException

6 tests completed, 6 failed
:integrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':integrationTest'.
> There were failing tests. See the report at: file:///Users/wei-li/repo/couchbase/sdkd-2.0/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED
Comment by Wei-Li Liu [ 15/Jul/14 ]
Disregard the above integration test failure. It requires to create "default" bucket and set no password to run.
Comment by Wei-Li Liu [ 18/Aug/14 ]
Running 2.0-dp3 feature test against 1118 server, and all of the unit test and integration tests passed. I am resolving this ticket.
Comment by Deepti Dawar [ 19/Aug/14 ]
Hi Michael, Please close this when you push the fix for it.

Thanks !
Comment by Michael Nitschinger [ 19/Aug/14 ]
Should be fixed in master, please reopen if not.
Comment by Wei-Li Liu [ 20/Aug/14 ]
I run int test on master today and there is 1 failure on shouldUpsertLegacyObjectDocument

com.couchbase.client.java.BinaryTest STANDARD_ERROR
    log4j:WARN No appenders could be found for logger (com.couchbase.client.core.logging.CouchbaseLoggerFactory).
    log4j:WARN Please initialize the log4j system properly.
    log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument FAILED
    java.lang.NullPointerException
        at com.couchbase.client.java.BinaryTest.shouldUpsertLegacyObjectDocument(BinaryTest.java:295)
Gradle Worker 1 finished executing tests.

28 tests completed, 1 failed
Comment by Michael Nitschinger [ 21/Aug/14 ]
Hm there might be an issue with the new legacy document, a different test is failing for me.

Is there any other info in the logs what's going on?
Comment by Deepti Dawar [ 21/Aug/14 ]
The unit tests now run fine.
Tried with the latest of DP3.
Comment by Wei-Li Liu [ 21/Aug/14 ]
stacktrace for legacy document test

11:40:54.879 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument STARTED
11:40:54.918 [DEBUG] [TestEventLogger]
11:40:54.918 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument FAILED
11:40:54.918 [DEBUG] [TestEventLogger] java.lang.NullPointerException
11:40:54.918 [DEBUG] [TestEventLogger] at com.couchbase.client.java.BinaryTest.shouldUpsertLegacyObjectDocument(BinaryTest.java:295)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.919 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
11:40:54.921 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.922 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
11:40:54.922 [DEBUG] [TestEventLogger] at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:105)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.923 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:355)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
11:40:54.923 [DEBUG] [TestEventLogger] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
11:40:54.923 [DEBUG] [TestEventLogger] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
11:40:54.923 [DEBUG] [TestEventLogger] at java.lang.Thread.run(Thread.java:745)
11:40:54.925 [DEBUG] [TestEventLogger]
11:40:54.925 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldInsertAndGet STARTED
11:40:54.925 [DEBUG] [TestEventLogger]
11:40:54.925 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldInsertAndGet PASSED
11:40:57.110 [QUIET] [system.out] 11:40:57.110 [
11:40:57.110 [DEBUG] [TestEventLogger]
11:40:57.110 [QUIET] [system.out] DEBUG] [org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor] Executing test class com.couchbase.client.java.DesignDocumentTest
Comment by Wei-Li Liu [ 21/Aug/14 ]
This exception is caused by integration test, which is somewhat a duplicate of JCBC-481.
unit test works fine for me too.




[JCBC-481] Integration Tests hang up after 85% progress Created: 25/Jun/14  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
[root@pomelo-11017 couchbase-java-client]# ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava UP-TO-DATE
:processIntegrationResources UP-TO-DATE
:integrationClasses UP-TO-DATE
> Building 85% > :integrationTest

 Comments   
Comment by Wei-Li Liu [ 18/Aug/14 ]
The process hangs at 85% due to view query test.
Just runs 2.0.0-dp3 integration test against the new beta2 build, and all the tests completes.
Comment by Deepti Dawar [ 21/Aug/14 ]
Following error appears with latest DP3 -

[root@pomelo-11017 couchbase-java-client]# ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
1 warning
:processIntegrationResources
:integrationClasses
:integrationTest

com.couchbase.client.java.BinaryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.BinaryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.DesignDocumentTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.DesignDocumentTest > classMethod FAILED
    java.util.NoSuchElementException

4 tests completed, 4 failed
:integrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':integrationTest'.
> There were failing tests. See the report at: file:///root/couchbase/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 22.09 secs
Comment by Michael Nitschinger [ 21/Aug/14 ]
Does it also happen on master?
Comment by Wei-Li Liu [ 21/Aug/14 ]
I test it on master and only see JCBC-479




[JCBC-450] 1.4.0 is not published on maven central Created: 17/Apr/14  Updated: 21/Apr/14  Resolved: 18/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Infrastructure
Affects Version/s: 1.4.0
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Just reported by support, 1.4.0 doesn't seem to be on maven central.

 Comments   
Comment by Michael Nitschinger [ 18/Apr/14 ]
It shows up:

http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22com.couchbase.client%22%20AND%20a%3A%22couchbase-client%22

and the artifacts can be downloaded.
Comment by Michael Nitschinger [ 18/Apr/14 ]
Dont think this is a bug, maybe the person needs to update his maven index in the IDE from time to time so it gets picked up properly by autocomplete.
Comment by Matt Ingenthron [ 21/Apr/14 ]
We determined that the issue is the lexical sort of the artifacts. In the future, we'll need to be more careful about how we publish DPs. I think the -SNAPSHOT approach understood by maven needs to be implemented. This should be fixed with a 1.4.1 shortly.




[JCBC-399] ViewFuture Listeners get completed twice Created: 11/Jan/14  Updated: 14/Jan/14  Resolved: 14/Jan/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.3.0
Fix Version/s: 1.3.1
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This is a regression from 1.3.0.

See https://github.com/ReactiveCouchbase/ReactiveCouchbase-core/issues/3 for more details

 Comments   
Comment by Michael Nitschinger [ 13/Jan/14 ]
http://review.couchbase.com/#/c/32305/




[JCBC-334] release notes for 1.2.0 Created: 24/Jul/13  Updated: 17/Sep/13  Resolved: 17/Sep/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 1.2
Security Level: Public

Type: Task Priority: Blocker
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Need published release notes for 1.2.0.

This would be expected around first week of September.

 Comments   
Comment by kzeller [ 24/Jul/13 ]
Important: as of 7/24 we are waiting on Ray to get ubu-1701 running. Java 1.2 files exist but are not yet publishable on the web.
Comment by Matt Ingenthron [ 29/Jul/13 ]
Update from Karen last week, the VM has been recovered. I believe we're good to go on docs.




[JCBC-360] Prepare for move to maven central Created: 10/Sep/13  Updated: 17/Sep/13  Resolved: 17/Sep/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.2
Security Level: Public

Type: Task Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-349] Calling flush() on CouchbaseClient throws a ClassCastException Created: 26/Aug/13  Updated: 04/Sep/13  Resolved: 04/Sep/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.9
Fix Version/s: 1.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Alan Kleiman Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When calling flush() on CouchbaseClient a ClassCastException is thrown:

java.lang.ClassCastException: com.couchbase.client.CouchbaseMemcachedConnection cannot be cast to com.couchbase.client.CouchbaseConnection
at com.couchbase.client.CouchbaseClient.flush(CouchbaseClient.java:2421) ~[couchbase-client-1.1.9.jar:1.1.9]
at com.couchbase.client.CouchbaseClient.flush(CouchbaseClient.java:2410) ~[couchbase-client-1.1.9.jar:1.1.9]

The problem seems to be in CouchbaseClient:2421, maybe related to JCBC-323?

 Comments   
Comment by Michael Nitschinger [ 26/Aug/13 ]
Jup looks like a bug! Expect a fix for the next release.
Comment by Michael Nitschinger [ 27/Aug/13 ]
http://review.couchbase.org/#/c/28599/




[JCBC-339] Couchbase java client ignores provided username and always use bucket as username during HTTP authorization step Created: 31/Jul/13  Updated: 31/Jul/13  Resolved: 31/Jul/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.8
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Sergey Bushik Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Server: CentOS 2.6.32 x86_64, Client: MacOS Darwin 12.4.0, Java 1.6


 Description   
new CouchbaseClient(servers, "default", "username", "password") doesn't honor provided username and always use bucket as username during HTTP authorization step.

Outgoing request is:
FINE: www.MessageHeader@658f73867'>sun.net.www.MessageHeader@658f73867 pairs: {GET /pools HTTP/1.1: null}{Accept: application/json}{user-agent: Couchbase Java Client}{X-memcachekv-Store-Client-Specification-Version: 1.0}{Authorization: Basic ZGVmYXVsdDpwYXNzd29yZA==}{Host: 192.168.1.79:8091}{Connection: keep-alive}
Text value of Authorization header corresponds to "Basic default:password", while it should be "Basic username:password"

Therefore 401 unauthorized response received:
www.MessageHeader@92f1bf07'>sun.net.www.MessageHeader@92f1bf07 pairs: {null: HTTP/1.1 401 Unauthorized}{WWW-Authenticate: Basic realm="Couchbase Server Admin / REST"}{Server: Couchbase Server 2.1.0-718-rel-enterprise}{Pragma: no-cache}{Date: Wed, 31 Jul 2013 13:10:21 GMT}{Content-Length: 0}{Cache-Control: no-cache}

The actual bug hides at line #131 of method com.couchbase.client.CouchbaseConnectionFactoryBuilder.buildCouchbaseConnection(final List<URI> baseList, final String bucketName, final String usr, final String pwd), wher usr parameter is not used anyhow and just ignored

Or see it at:
https://github.com/couchbase/couchbase-java-client/blob/master/src/main/java/com/couchbase/client/CouchbaseConnectionFactoryBuilder.java#L131

 Comments   
Comment by Michael Nitschinger [ 31/Jul/13 ]
Hi Sergey,

that's intended and not an issue. This is because the "user" is currently not being used, you always need to use the bucket name as the user and the password.

The only occasion where you dont get around with bucket level credentials is by adding/removing buckets, with the ClusterManager, and there you need to provide the admin creds.

Makes sense?
Comment by Sergey Bushik [ 31/Jul/13 ]
Technically, I see extended signature which accepts bucket, username & password parameters. Even though the credentials are valid, it throws out error, which says nothing about username parameter usage, but...

java.io.IOException: Server returned HTTP response code: 401 for URL: http://192.168.1.79:8091/pools
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readToString(ConfigurationProviderHTTP.java:424)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.readPools(ConfigurationProviderHTTP.java:210)
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:147)
at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:229)
at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:241)
Exception in thread "main" com.couchbase.client.vbucket.ConfigurationException: Configuration for bucket "default" was not found in server list ([http://192.168.1.79:8091/pools]).

That's confusing. By the way, I debugged and found that it's possible to authenticate using bucket/username/password over HTTP (modified ConfigurationProviderHTTP).




[JCBC-278] ConnectionFactoryBuilder has wrong default settings Created: 02/Apr/13  Updated: 03/Apr/13  Resolved: 03/Apr/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.4
Fix Version/s: 1.1.5
Security Level: Public

Type: Task Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The Builder uses wrong failure mode and default hash, they differ from instantiating the factory directly. This causes inconsistencies and also EPT node failures not to be detected properly.

 Comments   
Comment by Frank Weigel [ 03/Apr/13 ]
WIll changing hash affect customers upgrading? (i.e. not finding old data)




[JCBC-261] Warmup Backoff breaks Memcache Buckets Created: 11/Mar/13  Updated: 12/Mar/13  Resolved: 12/Mar/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.3
Fix Version/s: 1.1.4
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A recent change regarding warmup backoff breaks compat with memcache type buckets.




[JCBC-251] Observe ignores replica on index 0 Created: 24/Feb/13  Updated: 27/Feb/13  Resolved: 27/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1, 1.1.2
Fix Version/s: 1.1.3
Security Level: Public

Type: Task Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
A replica on index 0 is not added to the broadcast observe list and therefore leads to timeouts when persistence constraints are provided.




[JCBC-245] Docs: Javadoc states that a java.lang.String is required for the value in a write operation, whereas online docs state object Created: 12/Feb/13  Updated: 27/Feb/13  Resolved: 27/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.2
Fix Version/s: 1.1.3
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Java docs here: http://www.couchbase.com/autodocs/couchbase-java-client-1.1.2/

All set/replace/add/cas operations state something like:
cas(java.lang.String key, long cas, java.lang.String value, PersistTo req)

Whereas the docs here: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-summary.html

State that it's an "Object value"

Looking deeper, it appears that the memcached methods inherited from spy use java.lang.object which leads to user confusion when trying to compare the two.

 Comments   
Comment by Perry Krug [ 12/Feb/13 ]
Adding a library component since the Java docs show what the required type is, and if we are limiting to strings, we are preventing people from using these methods with non-string objects without writing their own transcoder which seems a little arduous in the simpler cases.
Comment by Michael Nitschinger [ 19/Feb/13 ]
That's purely a lib bug, since those should also require Object instead of String.




[JCBC-242] The flush() operations always fails with "401 Unathorized" message Created: 08/Feb/13  Updated: 08/Feb/13  Resolved: 08/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Balint Ureczky Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: 1m
Time Spent: Not Specified
Original Estimate: 1m
Environment: uname -a
Linux Mint 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

java -version
java version "1.7.0_11"
Java(TM) SE Runtime Environment (build 1.7.0_11-b21)
Java HotSpot(TM) 64-Bit Server VM (build 23.6-b04, mixed mode)

Java couchbase client library
1.1.2 taken from: http://files.couchbase.com/maven2/couchbase/couchbase-client/1.1.2/couchbase-client-1.1.2.jar

Attachments: Java Source File CouchbaseMagic.java    
Issue Links:
Duplicate
duplicates JCBC-173 flush will not work owing to MB-7381 Resolved

 Description   
The flush operations seems to require some kind of authorization, however the client library does not provide a way to do it.

The documentation is does not mention anything about this.

Is there a magical way I can call flush?
I attached the source code which reproduces the problem.

 Comments   
Comment by Michael Nitschinger [ 08/Feb/13 ]
Note that this is a known limitation because of the CouchbaseServer

See the linked issue for more information!
Comment by Michael Nitschinger [ 08/Feb/13 ]
Note that this is marked to be fixed in 2.0.1 which is the next bugfix release of couchbase server. Once upgraded, it should "magically" work!




[JCBC-241] Paginator object doesn't move onto next "page" correctly Created: 07/Feb/13  Updated: 08/May/13  Resolved: 08/May/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.2
Fix Version/s: 1.1.6
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File paginator-java.diff    
Issue Links:
Dependency

 Description   
Customer found his own issue:
I had a look at the source code for the java driver and found the error there. It is in:

com.couchbase.client.protocol.views.Paginator.getNextPage()

Right after calculating the number of "remaining" keys, it sets the limit of the query to the remaining number of keys and re-runs the query. What is missing here is:

    q.setSkip(totalDocs);

To actually start from the entry after the last entry on the current page. I have tried this and it works. But there is more code in this method and it is not very easy to follow so important to make sure that this fits.


I am enclosing a diff.

 Comments   
Comment by Perry Krug [ 07/Feb/13 ]
Update, but the bug still stands as described.

It seems that my fix is not the correct one. The bug seem to be that although the Paginator specifies setStartkeyDocID, the query doesn't start from this key.

I have debugged the Paginator class – and when resolving the next page it instructs the query to start from a specific doc id, e.g:

Query: ?limit=50&startkey=144&skip=0&stale=false&startkey_docid=144

But as a result of this query I get:

Got key: 1
Got key: 10
Got key: 100
Got key: 101
Got key: 102
Got key: 103

Got key: 141
Got key: 142
Got key: 143

Every time. So the bug is in the Query context somewhere.

Also – a note of inefficiency in the Paginator class: why perform two queries every time? One for the content of the "page" and one to find the row for the next page? Would be better to query for page + 1 and keep the last separate from the ViewResponse.

So, what I can conclude is that when the Paginator bug is resolved it should work fine for us. The additional query for the last row could be an issue though, since it uses setSkip(). So could this be treated as a bug as well?




[JCBC-219] Fix reconnect logic when server closes the socket. Created: 24/Jan/13  Updated: 29/Jan/13  Resolved: 29/Jan/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 1.1.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When the server disconnects on us during a valid remove/failover rebalance, it is the case that the CouchbaseClient stays connected and doesnt try to disconnect and retry. This leaves the client in a dangerous state where no updates are fetched and therefore the map is incorrect.

 Comments   
Comment by Michael Nitschinger [ 25/Jan/13 ]
http://review.couchbase.org/#/c/24182/




[JCBC-168] CouchClient.getView always throws an exception Created: 05/Dec/12  Updated: 11/Dec/12  Resolved: 11/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1-beta
Fix Version/s: 1.1.0
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Chris Tashjian Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
It would seem that client.getView(String designdocName, String viewName) has stopped working in 1.1-beta. It now throws:

java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:66)
at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:492)

This was working fine in 1.1-dp4.

 Comments   
Comment by Michael Nitschinger [ 06/Dec/12 ]
Hi Chris,

thanks for filing this. There has been a change that makes the ViewTimeout tunable, which I think may be the problem here. All my tests go through without problems, so I think you're using the FactoryBuilder right?

Can you please give me your bootstrap code and all the timeouts you are using? Also, please check your boot logs if it says something about a low view timeout. Thanks!
Comment by Chris Tashjian [ 06/Dec/12 ]
We're using the default ViewTimeout...

CouchbaseConnectionFactoryBuilder builder = new CouchbaseConnectionFactoryBuilder();
builder.setAuthDescriptor(new AuthDescriptor(new String[]{"PLAIN"}, new PlainCallbackHandler(bucketName, password)));
builder.setOpTimeout(opTimeout);
if (failureMode != null) {
    builder.setFailureMode(FailureMode.valueOf(failureMode));
}
CouchbaseClient client = createClient(builder, uris);
Comment by Chris Tashjian [ 06/Dec/12 ]
Ok, I think I was able to get this to work by adding "builder.setViewTimeout(5000);".

At one point I had it set to 3000 and got a timeout exception... however, if you don't explicitly call setViewTimeout, it seems that you get the less informative RuntimeException that I originally filed this for. It might be helpful if the constructor for CouchbaseConnectionFactoryBuilder set some kind of default timeout.
Comment by Matt Ingenthron [ 11/Dec/12 ]
http://review.couchbase.org/#/c/23189/
Comment by Michael Nitschinger [ 11/Dec/12 ]
fixed and pushed to master!




[JCBC-162] re-enable delete observe against 2.0 server Created: 03/Dec/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1-dp3, 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
At one point, the observe delete had been disabled on the master branch owing to changes in approach and needing to get a change merged for delete. Once this is complete, we need to revert the change.

See the review for the remove here: http://review.couchbase.org/20918

 Comments   
Comment by Michael Nitschinger [ 03/Dec/12 ]
implemented and pushed to master, will be available in beta.




[JCBC-160] merge in changes from the release10 branch for 1.1beta Created: 03/Dec/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In particular, the fix for JCBC-70 is needed, but there are a couple of changes which need to be merged in

 Comments   
Comment by Michael Nitschinger [ 03/Dec/12 ]
merged, will be available in the beta release.




[JCBC-158] add a debug=true option to view query options Created: 29/Nov/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
To be able to diagnose issues that may come up with views, please add a debug=true query parameter. Optionally, we could add a cbclient.properties parameter to be able to turn on view debugging (for all view requests or just some that match a regex?) without code changes (though, with restarts).

 Comments   
Comment by Michael Nitschinger [ 30/Nov/12 ]
http://review.couchbase.org/#/c/22923
Comment by Michael Nitschinger [ 03/Dec/12 ]
pushed to master, will be available in the beta relase.




[JCBC-154] merge in changes from the release11c branch Created: 26/Nov/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When reviewing some changes on the "c" branch, I happened to notice there may be one change that hasn't been submodule merged

Upon further inspection, it looks like a few other changes haven't been merged either. We'll need to merge this branch back in.

 Comments   
Comment by Michael Nitschinger [ 29/Nov/12 ]
You already did the work, so assigning it to you.
Comment by Matt Ingenthron [ 03/Dec/12 ]
http://review.couchbase.org/#/c/22973/
Comment by Michael Nitschinger [ 03/Dec/12 ]
merged into master, available in beta.




[JCBC-153] Raise view timeout from 60s to 75s Created: 21/Nov/12  Updated: 03/Dec/12  Resolved: 27/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Since the server side view timeout is 60s, we should raise the client side a bit higher. 75s seems like the right number. It should be tuneable though.

 Comments   
Comment by Michael Nitschinger [ 22/Nov/12 ]
http://review.couchbase.org/#/c/22755/
Comment by Michael Nitschinger [ 27/Nov/12 ]
Pushed to master, will be available in dp5/beta.




[JCBC-150] "Failed to access the view" error when querying a view with reduce Created: 19/Nov/12  Updated: 27/Nov/12  Resolved: 27/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Chris Tashjian Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Attempting to make a call to CouchbaseClient.query(view, query) with a view that has a reduce (even just "_count") results in the following exception when results are returned:

java.lang.RuntimeException: Failed to access the view
at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:634)

Worth noting:
- If the query returns no results (ViewResponse.size() returns 0), you do not get the exception...
- If the reduce method is removed from the view, everything works fine. However, using query.setReduce(false) does not work.



 Comments   
Comment by Michael Nitschinger [ 20/Nov/12 ]
Hi Chris,

thanks for the report. I'll investigate and let you know!

Thanks,
Michael
Comment by Michael Nitschinger [ 21/Nov/12 ]
Please make sure to use setReduce(true) and .setReduce(false) explicitely on a view with a reduce function right now.

The following code works with setReduce(true) and false, but will throw the exception you mentioned when setReduce is not used at all:

    query = new Query();
    //query.setReduce(false);
    view = client.getView(DESIGN_DOC_W_REDUCE, VIEW_NAME_W_REDUCE);
    reduce = client.query(view, query);
    itr = reduce.iterator();
    while(itr.hasNext()) {
      ViewRow row = itr.next();
      System.out.println(row.getKey());
    }

Comment by Michael Nitschinger [ 21/Nov/12 ]
After more investigation, this is really a shortcoming and should be handled in this changeset:

http://review.couchbase.org/#/c/22710/

It makes sure that reduce = true when nothing is set and the view contains a reduce function. The exception was misleading because it covered a JSON parsing bug underneath.
Comment by Michael Nitschinger [ 27/Nov/12 ]
fixed and pushed to master; will be available in dp5/beta.




[JCBC-147] Rename getViews() to getDesignDocument() Created: 14/Nov/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks JCBC-63 add APIs for creating and deleting de... Resolved

 Description   
The getViews() method should be renamed to getDesignDocument() before the API is stable. This is also in preparation for the createDesignDocument() and deleteDesignDocument() additions.

 Comments   
Comment by Michael Nitschinger [ 21/Nov/12 ]
http://review.couchbase.com/#/c/22713/
Comment by Matt Ingenthron [ 03/Dec/12 ]
This breaks API over DP, and should be release noted.
Comment by Michael Nitschinger [ 03/Dec/12 ]
fixed and merged into master, will be available in beta!




[JCBC-144] flush command needs to use RESTful flush for Couchbase Buckets Created: 11/Nov/12  Updated: 11/Dec/12  Resolved: 11/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.1.0
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Matt Ingenthron Assignee: Matt Ingenthron
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The current flush command needs to be connected to the RESTful flush.

 Comments   
Comment by Michael Nitschinger [ 12/Nov/12 ]
To my findings there are two ways to implement this feature:

- Reuse the BucketManager for this, since it already provides basic capabilities for flushing (but needs some extension).
- Reimplement the whole thing.

I would go with extending the BucketManager for this, since it would also keep the CouchbaseClient itself lean.

There is one thing that we need to decide upon: we can't just override the flush() method, because the returned value is different (we can't return an operation future with the current implementation). If we want to, we could "fake" it into one (but I doubt it makes sense). Therefore I propose a new method (flushBucket) which should be used with the couchbase client and we need to document that the old flush methods only work against memcached servers.
Comment by Michael Nitschinger [ 12/Nov/12 ]
Tracked here: http://review.couchbase.com/#/c/22445/
Comment by Matt Ingenthron [ 03/Dec/12 ]
Pushing out to release 1.1.0, as it's secondary functionality.
Comment by Matt Ingenthron [ 07/Dec/12 ]
http://review.couchbase.org/#/c/22445/




[JCBC-110] observe method does not correctly check persistence or replication status on replica node/vbuckets Created: 12/Sep/12  Updated: 16/Sep/12  Resolved: 16/Sep/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1dp2
Fix Version/s: 1.1-dp3
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When performing some additional testing, Abhinav noticed some unusual behavior in interacting with clients. After checking with Mike a bit, they noticed that observe protocol were not going to replica node/vbuckets during the poll interval loop.

 Comments   
Comment by Matt Ingenthron [ 12/Sep/12 ]
Assigning to Mike to update/replace the description.
Comment by Mike Wiederhold [ 16/Sep/12 ]
http://review.couchbase.org/#/c/20847/
http://review.couchbase.org/#/c/20848/
http://review.couchbase.org/#/c/20849/
http://review.couchbase.org/#/c/20850/
http://review.couchbase.org/#/c/20851/
http://review.couchbase.org/#/c/20852/




[JCBC-47] Query.copy does not copy includeDocs property Created: 12/May/12  Updated: 01/Jun/12  Resolved: 01/Jun/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1dp
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Steven Cooke Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: all java


 Description   
Query.java line 200 in function copy():
setIncludeDocs(willIncludeDocs());

Should be
query.setIncludeDocs(willIncludeDocs());

Paginated queries depend on copy() so paginatedQuery requiring the document fail with:
 java.lang.UnsupportedOperationException: This view result doesn't contain documents

 Comments   
Comment by Steven Cooke [ 12/May/12 ]
Perhaps a better solution:

  /**
   * All values in args map must be immutable
   */
  private Query(Query src) {
    args = new HashMap<String, Object>(src.args);
    includedocs = src.willIncludeDocs();
  }

  public Query copy() {
    return new Query(this);
  }




[JCBC-40] paginatedQuery throws NoSuchElementException and NPE Created: 29/Apr/12  Updated: 03/Dec/12  Resolved: 27/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1dp
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Steven Cooke Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: java


 Description   
view iteration is incomplete and throws exceptions for
groovy code:
paginator = client.paginatedQuery(view, query, n)
while (paginator.hasNext()) {
  row = paginator.next() // Exception on this line
  ...
}


java.util.NoSuchElementException
        at java.util.LinkedList$ListItr.next(LinkedList.java:698)
        at com.couchbase.client.protocol.views.Paginator.next(Paginator.java:76)
        at com.couchbase.client.protocol.views.Paginator.next(Paginator.java:35)
        at java_util_Iterator$next.call(Unknown Source)

AND

java.lang.NullPointerException
        at com.couchbase.client.protocol.views.Paginator.getNextPage(Paginator.java:93)
        at com.couchbase.client.protocol.views.Paginator.hasNext(Paginator.java:67)
        at java_util_Iterator$hasNext.call(Unknown Source)

Paginator also has a couple odd returns
1) next() can return null, but it should never return null unless lastRow is null. I don't think lastRow should ever be null.
2) getNextPage() has a return type, HttpFuture<ViewResponse>, but always returns null


 Comments   
Comment by Steven Cooke [ 15/May/12 ]
some workarounds for NPE in getNextPage():

1) Check for a null ViewResponse in getNextPage and return the ViewResponse instead of a future or null. The hasNext method can then check the value of getNextPage returning false if there is no ViewResponse.

 2) to bypass isJsonObject for numeric keys in Query.getArgs:
boolean numericKey = (key.equals(STARTKEY) && value.toString().matches("\\d+")); // still not json number but catches most common case of positive
integer keys if (key.equals(STARTKEYDOCID) || numericKey ) { return key + "=" + value;

 3) Skip parameter needs to be zero for pages after the first. In Paginator:
 public boolean hasNext() {
   if (!pageItr.hasNext() && page.size() < docsPerPage) {
     return false;
   } else if (!(rowsIterated < docsPerPage)) {
     lastRow = pageItr.next();
     query.setSkip(0);
     query.setStartkeyDocID(lastRow.getId());
     query.setRangeStart(lastRow.getKey());
     return (getNextPage(query) != null);
   }
   return true;
}
Comment by Steven Cooke [ 15/May/12 ]
Note: #2 fails for numeric keys that are expected to be strings, an issue raised in SPY-62
Comment by Raghavan Srinivas (Inactive) [ 16/Sep/12 ]
I have revamped the paginator and the code to iterate over page and within page looks something like this.

       while (result.hasNext()) {
           ViewResponse response = result.next();
           for (ViewRow row: response) {
             System.out.println("Inner loop " + row.getId());
           }
           System.out.println("<===Outer loop====>");
       }
  
Comment by Matt Ingenthron [ 18/Sep/12 ]
http://review.couchbase.org/#/c/20898/
Comment by Matt Ingenthron [ 18/Sep/12 ]
After discussion with Rags, there are some scenarios not as well tested as we'd like and there was some clarification on how this is expected to work.
Comment by Raghavan Srinivas (Inactive) [ 26/Sep/12 ]
This revolves around getKey() returning a string which is not the case always. It could return other types (such as int) and this will affect the way PaginatedQuery and other queries behave.
Comment by Matt Ingenthron [ 06/Oct/12 ]
In an email, Steven Cooke commented:

A couple other things I noticed:

The fix breaks all the original test cases as return type of next is no longer the same. If the API is intended to be non-breaking, the
 test cases and next should be reverted.

The private method
getNextPage has a return value that is never used. This looks fishy
Comment by Matt Ingenthron [ 06/Oct/12 ]
This isn't intended to be API compatible actually. Rags and I talked through it and decided that the odd returns of nulls were odd. I don't know that the API you see here is final either.

We would appreciate any additional feedback on this API or how you think it should work.

Once we hit beta, we'll keep the API stable.
Comment by Michael Nitschinger [ 14/Nov/12 ]
http://review.couchbase.org/#/c/22513/
Comment by Michael Nitschinger [ 27/Nov/12 ]
Fixed and pushed to master. Will be available in dp5/beta.




[JCBC-35] Exception during reconfiguring of client Created: 23/Apr/12  Updated: 26/Mar/13  Resolved: 26/Mar/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.0.2
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Marcus Nylander Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: couchbase-client 1.0.2

java version "1.6.0_30"
Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
Java HotSpot(TM) 64-Bit Server VM (build 20.5-b03, mixed mode)

Linux 2.6.18-238.19.1.el5 #1 SMP Sun Jul 10 08:43:41 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

Attachments: Text File log.txt    

 Description   
Multiple different stackstraces below also see the following right before it starts to fail:

"Node exepcted to receive data is inactive. This could be due to afailure within the cluster. Will check for updated configuration. Key without a configured node is: akey12324234"

Then it's fails a number of times and

Failed to reconfigure client, staying with previous configuration.
java.lang.IllegalArgumentException: TODO: refactor this
        at com.couchbase.client.vbucket.config.CacheConfig.getVbucketsCount(CacheConfig.java:55)
        at com.couchbase.client.vbucket.config.DefaultConfig.compareTo(DefaultConfig.java:145)
        at com.couchbase.client.vbucket.VBucketNodeLocator.updateLocator(VBucketNodeLocator.java:133)
        at com.couchbase.client.CouchbaseConnection.reconfigure(CouchbaseConnection.java:120)
        at com.couchbase.client.CouchbaseClient.reconfigure(CouchbaseClient.java:176)
        at com.couchbase.client.vbucket.ReconfigurableObserver.update(ReconfigurableObserver.java:54)
        at java.util.Observable.notifyObservers(Observable.java:142)
        at com.couchbase.client.vbucket.BucketMonitor.setBucket(BucketMonitor.java:257)
        at com.couchbase.client.vbucket.BucketMonitor.startMonitor(BucketMonitor.java:187)
        at com.couchbase.client.vbucket.ConfigurationProviderHTTP.subscribe(ConfigurationProviderHTTP.java:243)
        at com.couchbase.client.vbucket.ConfigurationProviderHTTP.finishResubscribe(ConfigurationProviderHTTP.java:215)
        at com.couchbase.client.CouchbaseConnectionFactory.checkConfigUpdate(CouchbaseConnectionFactory.java:182)
        at com.couchbase.client.CouchbaseConnection.addOperation(CouchbaseConnection.java:165)
        at net.spy.memcached.MemcachedConnection.enqueueOperation(MemcachedConnection.java:639)
        at net.spy.memcached.MemcachedClient.asyncGet(MemcachedClient.java:835)
        at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:997)




 Failed to reconfigure client, staying with previous configuration.
java.lang.IllegalArgumentException: TODO: refactor this
        at com.couchbase.client.vbucket.config.CacheConfig.getVbucketsCount(CacheConfig.java:55)
        at com.couchbase.client.vbucket.config.DefaultConfig.compareTo(DefaultConfig.java:145)
        at com.couchbase.client.vbucket.VBucketNodeLocator.updateLocator(VBucketNodeLocator.java:133)
        at com.couchbase.client.CouchbaseConnection.reconfigure(CouchbaseConnection.java:120)
        at com.couchbase.client.CouchbaseClient.reconfigure(CouchbaseClient.java:176)
        at com.couchbase.client.vbucket.ReconfigurableObserver.update(ReconfigurableObserver.java:54)
        at java.util.Observable.notifyObservers(Observable.java:142)
        at com.couchbase.client.vbucket.BucketMonitor.setBucket(BucketMonitor.java:257)
        at com.couchbase.client.vbucket.BucketMonitor.replaceConfig(BucketMonitor.java:307)
        at com.couchbase.client.vbucket.BucketUpdateResponseHandler.messageReceived(BucketUpdateResponseHandler.java:81)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at com.couchbase.client.vbucket.BucketUpdateResponseHandler.handleUpstream(BucketUpdateResponseHandler.java:194)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndfireMessageReceived(ReplayingDecoder.java:513)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:497)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:434)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:540)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:340)
        at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:272)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:192)
        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)


 Comments   
Comment by Matt Ingenthron [ 11/Oct/12 ]
This may have been fixed in configuraton changes in 1.0.3, but that should be verified.
Comment by Michael Nitschinger [ 28/Nov/12 ]
http://review.couchbase.org/#/c/22878/
Comment by Michael Nitschinger [ 29/Nov/12 ]
fixed and pushed to master, will be available in in dp5/beta.
Comment by Michael Nitschinger [ 26/Mar/13 ]
Already fixes back in time, don't know why it is open.




[JCBC-27] race condition during startup Created: 23/Mar/12  Updated: 04/Mar/13  Resolved: 04/Mar/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.0, 1.0.1, 1.1dp, 1.1.0
Fix Version/s: 1.1.3
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
During startup, if there is no authentication (and thus no authentication latch) we can reply with errors before we get the configuration back and settled in with the node locator. This should be more reliable.

 Comments   
Comment by Matt Ingenthron [ 23/Mar/12 ]
See http://www.couchbase.com/forums/thread/fast-computer-race-condition-java-client
Comment by Matt Ingenthron [ 29/Aug/12 ]
I think part of the solution on this is to poll the configuration until it transitions from warmup to healthy.
Comment by Matt Ingenthron [ 08/Oct/12 ]
The idea is that there is a section of the code that walks the URIs, finds the bucket, then after finding it sets up the stream for the configuration. When it first finds the bucket, if it's in a "warmup" state, (easy to simulate by restarting a server) it will show that it is and will not have a vbucket map. At that point, we should loop without setting up the stream *or* we should set up the stream and let anything handling reconfigure handle the transition from warmup to warmed up.
Comment by Michael Nitschinger [ 30/Nov/12 ]
http://review.couchbase.com/#/c/22933/1
Comment by Matt Ingenthron [ 03/Dec/12 ]
Still working out the flow here. Based on our current understanding, this can be deferred to 1.1.0 or even post since it's an enhancement for reliable operation in a secondary or tertiary circumstance. Should be release noted though.
Comment by Matt Ingenthron [ 27/Feb/13 ]
Determined that the proposed approach is a good change, but better change is needed. That's tracked under JCBC-255.




[JCBC-21] Move the view code into CouchbaseClient Created: 23/Jan/12  Updated: 21/Mar/12  Resolved: 21/Mar/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1.0
Security Level: Public

Type: Task Priority: Blocker
Reporter: Mike Wiederhold Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-20] CouchbaseClient instances consume all CPU resources Created: 10/Nov/11  Updated: 29/Jun/12  Resolved: 29/Jun/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1dp2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Kurtis Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Win7, centOS 5.4, tomcat 6.x, java 1.6.x

Attachments: Zip Archive CouchbaseTest.zip     PNG File resource_usage.png     Java Source File ViewConnection.java    

 Description   
When creating one or more CouchbaseClient instances, each one appears to consume 100% of the available CPU resources per core, at least on machines I've tested.
Test case (eclipse project and ant build) included.
Also included a snapshot of my system graph on my dev machine while running the included test.

 Comments   
Comment by Kurtis [ 10/Nov/11 ]
Also relevant are, this issue at google code:
http://code.google.com/p/spymemcached/issues/detail?id=218

And this thread in the Couchbase forums:
http://www.couchbase.org/forums/thread/using-couchbaseclient-connect
Comment by Matt Ingenthron [ 10/Nov/11 ]
I've reproduced this issue.

After investigating it a bit, I've found that simplifying the test to use either MemcachedClient or MembaseClient doesn't demonstrate the same issue. Under both of those, CPU usage is modest.

Further, more profiling seemed to show time on MacOS off in kqueue poll from the CouchbaseNode's run() method.

I suspect there's a thread safety issue here causing badness.

Asking Mike to review for any possible thread safety issues when there are multiple CouchbaseClients running.
Comment by Mike Wiederhold [ 30/Nov/11 ]
Were currently moving a few things around in the client. No user facing API changes, but still moving this around. I'll make sure to get this fixed as soon as possible though.
Comment by Alan Wood [ 04/May/12 ]
I've put a possible fix as an attachment to this issue. It works for us, and doesn't block shutdown, other nodes, etc.

The file was also submitted here (as an alternative to the ViewNode patch placed there earlier):
http://www.couchbase.com/issues/browse/JCBC-26
Comment by Steven Cooke [ 15/May/12 ]
Is this still open? ViewConnection and CouchbaseConnection are both threads with tight run loops. The quick fix is to sleep(50) or so. Not sure what the real solution should be, but it seems any threaded connection to the server to get server state can be handled by a singleton.

e.g.

public void run() {
  while(true) {}
}

vs

public void run() {
  while (true) {
    sleep(50); // utility function to hide try/catch
  }
}

First will hose the CPU, second will not
     




[JCBC-70] Client fails to reconnect to server of non-default memcached bucket after failover and add back Created: 28/Jun/12  Updated: 30/Jan/13  Resolved: 30/Jan/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.0.3
Fix Version/s: 1.1.2
Security Level: Public

Type: Bug Priority: Blocker
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: GZip Archive sc_1_plabq11.dev.sabre.com.out.gz    
Issue Links:
Dependency
depends on SPY-102 Ensure all nodes are in the list of n... Resolved
Duplicate
is duplicated by SPY-111 Assertion/NPE Exception when trying t... Resolved

 Description   
In earlier tests with reconnecting to a node on failover we used default memcached bucket. But when we tested the same scenario with a non-default bucket, we noticed the client did not reconnect (due to a null pointer exception internally). I have attached the SDK logs for this scenario where we used "IndexByLniataData" memcached bucket. The problem presents when adding the node back after a failover.
  
11:34:43,411 DEBUG [Memcached IO over {MemcachedConnection to /10.14.5.119:11210}] [CouchbaseMemcachedConnection] Selecting with delay of 3038ms
Exception in thread "Thread-3" java.lang.NullPointerException
        at net.spy.memcached.auth.AuthThread.buildOperation(AuthThread.java:117)
        at net.spy.memcached.auth.AuthThread.run(AuthThread.java:86)

Logs/stack trace attached.

 Comments   
Comment by Matt Ingenthron [ 22/Aug/12 ]
I've spent a bit of time analyzing this issue, and it's not clear what the cause is. It is correct though that this would cause the auth thread to die, and as such authentication to the node would never complete.

There is a safeguard already in that the continuous timeout threshold will kick in and then the connection will be rebuilt. I don't know if this issue comes up all of the time, but assuming it's a rare event we'd see 1000 operations timeout (by default) followed by the connection being rebuilt.

We'd have to add some diagnostic information to the client and reliably reproduce this to identify the issue. I think the scenario is:
1) set up a cluster of say 3 nodes
2) configure a client, have it work with an authenticated memcached bucket on the cluster
3) faillover a node by clicking on "failover" in the console
4) add the node back by clicking on "add back"

Is this correct?
Comment by Perry Krug [ 23/Aug/12 ]
That appears correct. The customer has been able to reliably reproduce this, but since so much time has passed I would be hesitant in going back to them if not necessary...
Comment by Matt Ingenthron [ 09/Jan/13 ]
There is an open changeset for this. Please determine if it is correct, needs to go in.
Comment by Michael Nitschinger [ 30/Jan/13 ]
Duplicate of Spy-111




[JCBC-63] add APIs for creating and deleting design documents for CBS 2.0 Created: 08/Jun/12  Updated: 29/Nov/12  Resolved: 29/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1dp
Fix Version/s: 1.1-beta
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Dipti Borkar Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on JCBC-147 Rename getViews() to getDesignDocument() Resolved

 Description   
There are have been several requests from users to have APIs to create and delete views from the Java (and other) SDKs.
This is a MUST have for some customers and needs to be a part of the Client SDK we release prior to Beta.

 Comments   
Comment by Matt Ingenthron [ 08/Jun/12 ]
I'm in agreement. One question is do we also want this API to be involved in the view materialization process. I think the answer is probably yes, but this can be complicated since it can potentially take a very long time with little feedback.
Comment by Dipti Borkar [ 08/Jun/12 ]
When you say View materialization process? what exactly do you mean? given that index building happens at query time, I don't think so.

However, there are other APIs that may be more interesting to have. explicitly triggering compaction, configuring min time duration / min # of mutations to trigger index building.
Comment by Matt Ingenthron [ 24/Jul/12 ]
What I mean by the view materialization process is that we, in the web console, have this flow where we advise people to take their "dev_aview" and run a full cluster dataset query on it before publishing it to production. This way, when it's published as "aview", the view will have been mostly materialized. It's a nice thing for updating a view.

The reason this is complicated is that it's an HTTP request that may go for a very long, long time. I can think of a way around it (poll it with a timeout until it's fast), but it's suboptimal unless we start querying view materialization progress.
Comment by Michael Nitschinger [ 26/Aug/12 ]
Matt, what if we wait for it in a separate thread and return with a future when it's done? We just need to make the user aware that it can take a long time, and if they add a view for development that they'd have to publish it separately.

Another idea would be to make the "let's do a full dataset query" optional when the view gets published since someone may want to do that later or from the web-ui.
Comment by Matt Ingenthron [ 26/Aug/12 ]
Michael: that's effectively what we do right now since it's the query against the view that takes a long time. The design document management operations do not. The view requests are not quite done asynchronously, that's true.

The main point I was raising was that the Web Console provides for functionality to execute a view before pushing it in production and workflow to take it from "dev_thing" to "thing" knowing that the view has been materialized. In the SDK, I don't think we'll try to replicate that workflow, but the same functionality is definitely available.
Comment by Michael Nitschinger [ 05/Oct/12 ]
Just to keep everyone updated, I started developing it here: http://review.couchbase.org/#/c/21380/
Comment by Michael Nitschinger [ 29/Nov/12 ]
this has finally been merged to master.




[JCBC-470] Implement Carrier-based configuration management Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-469] Implement HTTP-based configuration management Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-468] Implement EBNF-based N1QL DSL Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-551] Implement blocking API around async one Created: 11/Sep/14  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Task Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-525] Add support for full JSON compat docs & common flags Created: 22/Aug/14  Updated: 16/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-457] Force carrier config fetching on reconnect of node. Created: 07/May/14  Updated: 12/May/14  Resolved: 08/May/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.0
Fix Version/s: 1.4.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-454] Query View bug Created: 27/Apr/14  Updated: 05/May/14  Resolved: 05/May/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.0
Fix Version/s: 1.4.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Nas Kavian Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File stack trace.png    
Issue Links:
Duplicate
duplicates SPY-163 Bulk latch never counts down on empty... Resolved

 Description   
1.3.2 works, but 1.4.0 is broken.

Perform a simple query where you set the key to something non-existent and the query will never return. I think an empty result set is properly returned from the server but the Java client doesn't handle it properly afterwards.

      final Query query = new Query();
      query.setIncludeDocs(true);
      query.setKey(ComplexKey.of("abc")); << Non-existent key
      query.setStale(Stale.FALSE);
      final ViewResponse response = client.query(view, query); << Doesn't return.

I think the attached stack trace gives some idea of where it's hung up.

 Comments   
Comment by Michael Nitschinger [ 05/May/14 ]
Hi,

thanks for reporting this - it is a known issue and is already fixed in master, will be in the next bugfix release 1.4.1 this week hopefully.




[JCBC-453] Also check for new configs with different failure modes. Created: 25/Apr/14  Updated: 08/May/14  Resolved: 08/May/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-449] Add an overview to the javadoc and the CouchbaseClient class Created: 15/Apr/14  Updated: 17/Jun/14  Resolved: 17/Jun/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The HTML generated autodocs should have a better intro and get the user to the CouchbaseClient class more directly. For instance, if you go here:
http://www.couchbase.com/autodocs/couchbase-java-client-1.4.0/index.html

One would have to click around for a bit to find what they're searching for.

 Comments   
Comment by Michael Nitschinger [ 01/Jun/14 ]
Are you sure this is possible with javadoc? I googled around a bit and didn't find anything.. well we could make it work with changing the files after generation or so.
Comment by Michael Nitschinger [ 17/Jun/14 ]
I fixed that in my build script, check out: http://www.couchbase.com/autodocs/couchbase-java-client-1.4.2/index.html




[JCBC-418] Deadlock in ViewConnection with Rebalance and retry requests Created: 21/Feb/14  Updated: 22/Feb/14  Resolved: 22/Feb/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.0-dp1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-413] Race condition in cluster manager class Created: 11/Feb/14  Updated: 02/Jun/14  Resolved: 02/Jun/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.4.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Juan Manuel Musacchio Assignee: Michael Nitschinger
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I've been experiencing some connection issues ("Unable to connecto to cluster") while listing the couchbase buckets. After some debugging I realized that could be a race condition in the ClusterManager sendRequest method. The following are the involved lines (took it from master branch):

latch.countDown(); //line 474
success.set(true); //line 475
response.set(result); //line 476

The thing is that the mutex is decreased before setting sucess and response values, so later the code will check the success variable and could fail if success is not set before the thread starts (mutex is 0). I think the code should be:

success.set(true);
response.set(result);
latch.countDown();

 Comments   
Comment by Michael Nitschinger [ 02/Jun/14 ]
Thanks, good catch. I'll get the fix in 1.4.2
Comment by Michael Nitschinger [ 02/Jun/14 ]
http://review.couchbase.org/#/c/37719/1
Comment by Michael Nitschinger [ 02/Jun/14 ]
merged into master




[JCBC-403] Fix undetected streaming disconnect on pure-view workloads Created: 22/Jan/14  Updated: 06/Feb/14  Resolved: 06/Feb/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.3.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-461] Use alternative constructor to allow configuration of max New I/O client Worker thread count Created: 21/May/14  Updated: 02/Jun/14  Resolved: 02/Jun/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.3.1
Fix Version/s: 1.4.2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Vanessa Ablaya Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Linux, Java 1.6


 Description   
We are running multiple instances of an application on a blade with 24 cores. Netty creates (2 * #PROCESSORS) New I/O Client Worker threads per couchbase client. This is causing high thread count, as each of our application instances does not have complete access to the resources 24 processors. I want to be able to configure the maximum number of threads Netty creates.

Line 100 of BucketMonitor.java uses the following constructor from the netty library:

 public NioClientSocketChannelFactory(
            Executor bossExecutor, Executor workerExecutor)


I would like it to use the alternative constructor that netty provides, so that I can configure the workerCount value:

public NioClientSocketChannelFactory(
            Executor bossExecutor, Executor workerExecutor, int workerCount)



 Comments   
Comment by Michael Nitschinger [ 22/May/14 ]
That makes total sense. I'd rather not add an additional config setting for 1.* since the whole thing is revamaped for the 2.0 where the complete IO layer is based on netty and fully tunable.

What I could imagine though is a system property that you can set, which we could document. So people which run on those very powerful boxes can set it, and the others just don't have to care.
What do you think?
Comment by Michael Nitschinger [ 02/Jun/14 ]
Actually, I think we can trim that down since we are using it for streaming connections only. No need to provide that much. I'll investigate further and see what I can get into 1.4.2.
Comment by Michael Nitschinger [ 02/Jun/14 ]
Btw, two more things:

1) If you go to 1.4.1 and higher, it uses a different config fetching mechanism, so you should not see any of those threads created if you go with a server 2.5.0 and higher.
2) Even on your version, netty should create those threads only when needed, so do you mean you create lots of CouchbaseClient instances? You should reuse them as a singleton. Or do you see lots of those created even for one CouchbaseClient object? If so, this is a bug I think.

Can you provide more details on that? Maybe with a thread dump?
Comment by Michael Nitschinger [ 02/Jun/14 ]
http://review.couchbase.org/#/c/37722/
Comment by Michael Nitschinger [ 02/Jun/14 ]
We know only use one, so no need to configure. Please create a new ticket if you still then see issues with massive amount of threads generated, this is something else then for sure.




[JCBC-460] New configuration gets falsely discarded if only replica vbuckets change. Created: 12/May/14  Updated: 16/May/14  Resolved: 16/May/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.4.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-459] Ignore Configurations with a smaller or equal revid (if set) Created: 12/May/14  Updated: 16/May/14  Resolved: 16/May/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.0, 1.4.1
Fix Version/s: 1.4.2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-446] After all nodes have been tried on NMVB, clean out the list to start over again. Created: 04/Apr/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-439] Fix overriding of the auth descriptor in the builder Created: 26/Mar/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-436] Expose custom auth timeout setting Created: 24/Mar/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-431] Add tainted config poller for carrier configs Created: 18/Mar/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.0-dp1
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-427] Unintended rollback to old configuration on http disconnect. Created: 11/Mar/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.0-dp1
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-426] add envvar or property to force use of HTTP or Carrier Publication style bootstrapping Created: 10/Mar/14  Updated: 15/Apr/14  Resolved: 15/Apr/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.0-dp1
Fix Version/s: 1.4.0-dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
For testing as well as in-the-field workarounds for particular cases, please add a feature to be able to force one type of configuration or another.




[JCBC-389] Make PersistTo/ReplicateTo really async and not block Created: 16/Dec/13  Updated: 16/Dec/13  Resolved: 16/Dec/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.3.0
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates JCBC-361 Add capability of async+durability wi... Resolved
is duplicated by JCBC-361 Add capability of async+durability wi... Resolved

 Comments   
Comment by Michael Nitschinger [ 16/Dec/13 ]
See JCBC-361




[JCBC-380] alternate bootstraps have a hardcoded port 8091 Created: 15/Nov/13  Updated: 28/Nov/13  Resolved: 28/Nov/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.9, 1.2, 1.2.1, 1.2.2
Fix Version/s: 1.2.3
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Since it is possible, though not easy, to make Couchbase run on a port other than 8091, the hostnames extracted from the configuration can't necessarily be on port 8091. The initial bootstrap port number should be applied to all of these derived URIs.

 Comments   
Comment by Michael Nitschinger [ 16/Nov/13 ]
Two things here

- IMHO this is an enhancement, since its a new requirement to run against != 8091
- Not sure if we can grab it from the initial list because things can change, picking it up from the new config may be better - hope the server supplies it correctly.
Comment by Michael Nitschinger [ 16/Nov/13 ]
Is there a way to change the ports to test it?
Comment by Michael Nitschinger [ 18/Nov/13 ]
http://review.couchbase.org/#/c/30372/




[JCBC-373] ReplicaGetFuture is not thread safe. Created: 28/Oct/13  Updated: 31/Oct/13  Resolved: 31/Oct/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.1.8, 1.1.9, 1.2, 1.2.1
Fix Version/s: 1.2.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-372] ReplicaRead can cause Huge Heap/StackOverflowException. Created: 28/Oct/13  Updated: 19/Dec/13  Resolved: 19/Dec/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Under specific scenarios, ReplicaReads can cause "inifinte" redispatch and therefore either overflow on the stack or cause huge heaps (created by large callbacks as a result).

 Comments   
Comment by Michael Nitschinger [ 19/Dec/13 ]
This has been fixed in the 12.3 and 1.2.2 series.




[JCBC-369] observePoll has wrong logic on delete Created: 15/Oct/13  Updated: 31/Oct/13  Resolved: 31/Oct/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.2
Fix Version/s: 1.2.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-368] Deadlock in BucketMonitor.startMonitor() on CountDownLatch when channel creation fails. Created: 14/Oct/13  Updated: 24/Oct/13  Resolved: 24/Oct/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.9, 1.2
Fix Version/s: 1.2.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Deniss Afonin Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-326 Couchbase client stuck indefinitely i... Resolved

 Description   
Deadlock in BucketMonitor.startMonitor() on CountDownLatch when channel creation fails.

Before deadlocking it also logs following exception (which by itself is another bug):

java.lang.IllegalStateException: An Executor cannot be shut down from the thread acquired from itself. Please make sure you are not calling releaseExternalResources() from an I/O worker thread.
at org.jboss.netty.util.internal.ExecutorUtil.terminate(ExecutorUtil.java:73)
at org.jboss.netty.util.internal.ExecutorUtil.terminate(ExecutorUtil.java:49)
at org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.releaseExternalResources(NioClientSocketChannelFactory.java:180)
at org.jboss.netty.bootstrap.Bootstrap.releaseExternalResources(Bootstrap.java:319)
at com.couchbase.client.vbucket.BucketMonitor$1.operationComplete(BucketMonitor.java:193)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:428)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:419)
at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:381)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processConnectTimeout(NioClientSocketPipelineSink.java:394)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:289)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)



 Comments   
Comment by Michael Nitschinger [ 14/Oct/13 ]
Hi, Thanks for reporting.

can you give me more details about your env? Particulary, which sdk version do you run and on which platform/os?
Comment by Deniss Afonin [ 14/Oct/13 ]
Yeah, sorry for missing that. Here it is:

couchbase-client-1.2.0

running with

java version "1.7.0_40"
Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)

on OS X 10.8.5
Comment by Michael Nitschinger [ 14/Oct/13 ]
I see what's going on there, but I'd like to add a test to reproduce. How are you running into this?
Comment by Deniss Afonin [ 14/Oct/13 ]
I can't always reproduce that, this happens from time to time.
Comment by Michael Nitschinger [ 14/Oct/13 ]
I see, thanks for reporting it. Look forward to a proper fix in the next bugfix release.
Comment by Michael Nitschinger [ 14/Oct/13 ]
I think I fixed it, but in any case something went wrong during connection to the streaming connection.

Does the error is expected to happen in your environment or was it considered stable? I wonder in which circumstances you saw that. Even with my fix, its just shutting down properly but still the streaming conn attachment won't succeed.
Comment by Deniss Afonin [ 14/Oct/13 ]
Well, this happens for us only when couchbase server is not in local network i.e. we are connecting via internet with ~100ms latency and that internet connection might not be that stable.

Issue JCBC-326 might also be related to this.
Comment by Michael Nitschinger [ 14/Oct/13 ]
Yes that's what I assumed. So the thing is, in general we don't recommend running your app servers and db servers across the internet (they should be in the same datacenter, especially if you care about latency).

But this shouldn't be a limitation of the client itself. So I have a bugfix for this upcoming, but I'll also see what I can do to add some more retry logic to try different nodes from the list so if one fails we eventually connect to another one.
Comment by Michael Nitschinger [ 24/Oct/13 ]
fix has been merged into master and will be available in 1.2.2.




[JCBC-366] Properly allow override of metrics and listeners Created: 07/Oct/13  Updated: 11/Oct/13  Resolved: 11/Oct/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.2
Fix Version/s: 1.2.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-350] Race Condition in observe/replica Created: 27/Aug/13  Updated: 11/Sep/13  Resolved: 11/Sep/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.9
Fix Version/s: 1.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Michael Nitschinger [ 27/Aug/13 ]
Pushed into master, awating reporter feedback for close.




[JCBC-351] Enhance ReplicaRead with active node Created: 27/Aug/13  Updated: 13/Sep/13  Resolved: 13/Sep/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.9
Fix Version/s: 1.2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-337] add new method for retrieving configuration over memcached binary protocol Created: 30/Jul/13  Updated: 07/Apr/14  Resolved: 22/Feb/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.4.0-dp1
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: cccp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on MB-8728 add a new operation which allows for ... Closed

 Description   
To support better faster, more reliable response to configuration changes, the Java client should change from HTTP streaming configuration to the new memcached binary protocol method delivered under project CCCP.

In Java, this will necessitate a new constructor to start from hostname with an optional port number, as all current constructors are intended to be RESTful and use URIs.

This is part of project CCCP, as covered at http://www.couchbase.com/wiki/display/couchbase/Cluster+Configuration+Carrier+Publication

 Comments   
Comment by Michael Nitschinger [ 22/Feb/14 ]
I'm closing this since it has been merged technically. Let's open bug reports for all specific issues we identify.




[JCBC-336] IndexOutOfBounds when config changes during observePoll Created: 30/Jul/13  Updated: 01/Aug/13  Resolved: 01/Aug/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.7
Fix Version/s: 1.1.9
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
at java.util.ArrayList.rangeCheck:604
at java.util.ArrayList.get:382
at com.couchbase.client.vbucket.config.DefaultConfig.getServer:81
at com.couchbase.client.vbucket.VBucketNodeLocator.getServerByIndex:113
at com.couchbase.client.CouchbaseClient.observePoll:2185
at com.couchbase.client.CouchbaseClient.set:1286




[JCBC-324] Avoid NPE when Host is Up, but no CB is running Created: 02/Jul/13  Updated: 16/Dec/13  Resolved: 16/Dec/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by JCBC-321 Null Pointer in client when deleting ... Closed
is duplicated by JCBC-379 If bucket details are incorrect, erro... Closed

 Description   
When the target host is reachable, but no server is running, the client throws a unhelpful NPE:

Caused by: java.lang.NullPointerException
at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:148)
at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:231)
at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:237)
at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:175)

The logs also show more infos, but this NPE is not helpful in the stack.

 Comments   
Comment by Brad Wood [ 02/Jul/13 ]
Additional info here:
http://www.couchbase.com/communities/q-and-a/sdk-throws-unhelpful-npe-when-server-unreachable
Comment by Michael Nitschinger [ 18/Nov/13 ]
Brad, are you sure this is not fixed with recent releases?

I tried
 - not running node
 - running node with wrong bucket name
 - running node with wrong password

and I always get com.couchbase.client.vbucket.ConfigurationExceptions and not NPEs anymore... this is with vanilla 1.2.2 but I remeber fixing it earlier.

If you still see it, can you get me more information on the environment?




[JCBC-319] Force Resubscribing in CouchbaseMemcachedConnection Created: 21/Jun/13  Updated: 19/Jul/13  Resolved: 02/Jul/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1.8
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks JCBC-305 Client does not recover with Memcache... Closed




[JCBC-312] Reconfigure not triggered on Master -1 Created: 31/May/13  Updated: 07/Jun/13  Resolved: 07/Jun/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
During some failover/rebalance scenarios, it could be the case that no master is responsible for the document. While this should not be the case, it is observed in scenarios where the client may still have an outdated config from somewhere.

This leads to RuntimExceptions raised, but reconfigure is never actively triggered. In QE tests, this manifests itself in errors during change and rebound.

While it should be elsewhere investigated how those -1 get in place, checking for this and triggering reconfigure is a safety net for running operations.

 Comments   
Comment by Michael Nitschinger [ 31/May/13 ]
No fix applied:

Will show phase timings..
--------------------------
Phase statistics for RAMP
  OK/sec: 3619

  OK: 108596
  ERR: 0
Phase statistics for CHANGE
  OK/sec: 3204

  OK: 999917
  ERR: 141386
Phase statistics for REBOUND
  OK/sec: 3966

  OK: 357026
  ERR: 26898
---------------------

Fix applied:

--------------------------
Phase statistics for RAMP
  OK/sec: 3536

  OK: 106102
  ERR: 0
Phase statistics for CHANGE
  OK/sec: 1870

  OK: 471474
  ERR: 825
Phase statistics for REBOUND
  OK/sec: 4063

  OK: 365714
  ERR: 0
---------------------

stester run: /stester -C 127.0.0.1:8050 -i 20devcluster.ini -c failover.Once --vdsw_dvname ddoc/vquery --hdsw_http_threads 5 --grace_after 30 --ept 1 --ramp 30 --num_nodes 2 --hdsw_mc_threads 10 --workload dsw.Hybrid --action_delay 10 --hdsw_cb_threads 10 --action FO_REBALANCE --dsw_timeres 1 -d -o viewlog_3_f.out
Comment by Michael Nitschinger [ 31/May/13 ]
http://review.couchbase.org/#/c/26636/
Comment by Michael Nitschinger [ 31/May/13 ]
Note that before this change, the RuntimeException bubbled up to the userlevel, blocked everything there - but more importantly, cf.checkConfigUpdate(); never got triggered!
Comment by Michael Nitschinger [ 31/May/13 ]
This also improves this scenario run:

Effective stester command line
    -C 127.0.0.1:8050 \
    -i 20devcluster.ini \
    -c failover.Once \
    --vdsw_dvname ddoc/vquery \
    --hdsw_http_threads 5 \
    --grace_after 30 \
    --ept 1 \
    --ramp 30 \
    --num_nodes 2 \
    --hdsw_mc_threads 10 \
    --workload dsw.Hybrid \
    --action_delay 10 \
    --hdsw_cb_threads 10 \
    --action FO_REBALANCE \
    --dsw_timeres 1 \
    -d \

--------------------------
Phase statistics for RAMP
  OK/sec: 3057

  OK: 91713
  ERR: 0
Phase statistics for CHANGE
  OK/sec: 3172

  OK: 957997
  ERR: 187000
Phase statistics for REBOUND
  OK/sec: 4108

  OK: 369758
  ERR: 63598
---------------------

After

Will show phase timings..
--------------------------
Phase statistics for RAMP
  OK/sec: 3453

  OK: 103594
  ERR: 0
Phase statistics for CHANGE
  OK/sec: 2346

  OK: 731968
  ERR: 549
Phase statistics for REBOUND
  OK/sec: 4064

  OK: 365817
  ERR: 0
---------------------




[JCBC-280] Provision for auth failure in case of calling createBucket in the same txn twice and adding updateBucket functionality Created: 04/Apr/13  Updated: 12/Jun/13  Resolved: 03/Jun/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Deepti Dawar Assignee: Deepti Dawar
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
There are three ways using which we are creating connection in the java client to the server.

1) ClusterManager
2) BucketTool

Both of these classes internally call the ClusterManager.createBucket for creation of the bucket.

Now if I am using all the three instances of the above classes in a single function, the bucket information is overridden without any checks. Ideally using any of the connection classes if I have created a SASL bucket with some information like bucket name = 'SaslBucket', bucket password = 'password', I should not be allowed to change the password using instance of another class. There should be auth failure error being returned the second time client tries to connect to the same bucket. Server supports this because there is an Edit Bucket functionality at the server for the same.

There should be a means of distinction in the request that we want to create the bucket or update.
In case of bucket creation duplicity should be checked where as in case of update this should be allowed as is.

Also, the expectation is that, the user might update the bucket information if he requires to change the password or other details, by explicitly calling updateBucket method which is currently not available.

 Comments   
Comment by Deepti Dawar [ 05/Apr/13 ]
Alright.

Consider the following piece of code :

public void testCreateSaslBktBadPswd() throws Exception {
manager.createNamedBucket(BucketType.COUCHBASE, "bucket1", 100, 0,
"password", false);
List<URI> uris = new LinkedList<URI>();
uris.add(URI.create("http://"
+ TestConfig.IPV4_ADDR + ":8091/pools"));
new CouchbaseConnectionFactory(uris, "bucket1", "password12");
new CouchbaseClient(uris, "bucket1", "password123");
BucketTool bt = new BucketTool();
bt.createSaslBucket("bucket1", BucketType.COUCHBASE, 1, 2, true);
  }

Here, using cluster manager instance, we create a SASL bucket i.e. bucket1 with password as password.
Next you would see the usage of CouchbaseConnectionFactory instance which is trying to manipulate the same bucket, bucket1. It overrides the password to password12. Here it internally calls createBucket again without checking whether such a bucket already exists or not.
Another attempt is made to connect using the CouchbaseClient instance. The bucket password is again changed.
Similarly, the bucket tool instance also does the same.

Instead, there should be auth failure error being returned the second time client tries to connect to the same bucket.
Also, the expectation is that, the user might update the bucket information if he requires to change the password or other details, by explicitly calling updateBucket method which is currently not available.
Comment by Michael Nitschinger [ 24/Apr/13 ]
Is this still an issue?
Comment by Deepti Dawar [ 25/Apr/13 ]
You mean there has been a fix entered for this ?
Comment by Michael Nitschinger [ 25/Apr/13 ]
No, because I think the ticket is somewhat misleading. The Factory and the Client have nothing to do with Bucket creation.

The Manager should be used for clients and the bucket tool is an internal tool used in our testing environment. I still don't get whats the problem here? Just pick one of these depending on the thing you want to do.
Comment by Deepti Dawar [ 25/Apr/13 ]
The problem here is that, lets say there are two users - User 1 and User 2.
User 1 created the instance using CouchbaseConnectionFactory and provided a password to the default bucket as 'password'
User 2 created the instance using CouchbaseClient and provided a password as 'password2'

As both internally go and create the bucket, don't you think the two users are overriding each other's bucket information ?
Shouldn't the second user be returned an auth failure for the same ?
Comment by Michael Nitschinger [ 25/Apr/13 ]
Hi Deepti,

I'm not sure I can follow you. Neither CouchbaseConnectionFactory nor the CouchbaseClient class creates a bucket by any means. The user has to explicitly use the Manager to create a bucket! And the BucketTool uses the Manager underneath as well, it just provides some convenience methods.
Comment by Deepti Dawar [ 25/Apr/13 ]
Ok.

I see that the conflict is between -

manager.createNamedBucket
bucketTool.createSaslBucket

both of which call the same createBucket method.
Comment by Michael Nitschinger [ 25/Apr/13 ]
Ok so now that we pinned that down.

Can you please

1) clarify whats the issue between those methods and what needs to be fixed
2) update the title and description of the ticket to reflect those changes?

Thanks!
Comment by Deepti Dawar [ 28/May/13 ]
http://review.couchbase.org/#/c/26557/
Comment by Matt Ingenthron [ 11/Jun/13 ]
Deepti: any updates on the underlying issue? Thanks.
Comment by Deepti Dawar [ 12/Jun/13 ]
Hi Matt,

I have already checked in the code and it is up for review in gerrit.

Here is the link - http://review.couchbase.org/#/c/26557/

Regards,
Deepti
Comment by Matt Ingenthron [ 12/Jun/13 ]
Yes, but there's already a review on the patch set indicating there's a problem. Please update the patchset, clarify why what is being observed is expected or get with the reviewer to understand the problem if it's not clear.

Thanks!
Comment by Deepti Dawar [ 12/Jun/13 ]
Ah Yes ! Will update in some time.




[JCBC-276] Client does not detect silently dying Streaming Node Created: 20/Mar/13  Updated: 27/May/13  Resolved: 27/May/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When connected to the EPT/streaming node and the node is "frozen" or dies silently otherwise (doesn't force the closing of the chunked socket), the connection stays established.

This can easily be reproduced outside of the client by connecting the browser to the streaming URL and then freezing a VM. The browser will still "spin" and wait for new chunks to come up.

The proposed solution is to have a netty handler in place that raises a exception when there is not traffic for N number of seconds (like 30) over the streaming connection. After this is detected, we have two possibilities:

- reconnect completely, but this involves lots of overhead every 30 seconds.
- send a HTTP HEAD packet and only if this doesnt work out reconnect. This means in the normal case we only have a HTTP HEAD request sent every 30 seconds, not much overhead.
If this fails, we then trigger the reconfigure.

Netty has a ReadTimeoutHandler to help with this. My POC already kinda works, I just need to find a way to properly distinguish the HEAD response on the ResponseHandler from regular chunks that arrive from the same channel.

 Comments   
Comment by Matt Ingenthron [ 20/Mar/13 ]
I had fixed this with couchbase buckets and we test for this in SDKQE. Is this possibly isolated to memcached buckets.
Comment by Michael Nitschinger [ 21/Mar/13 ]
Actually, this has been discovered while using Couchbase buckets. Let's chat about this, but I think thats a different issue and not related to it.
Comment by Matt Ingenthron [ 21/Mar/13 ]
Sure, the solution previously was to have that threshold if we were getting unexpected failures. Once we pass that threshold, we'd try to re-subscribe.

I'm not opposed to a heartbeat, but I don't think the HTTP HEAD is good, since the mochiweb erlang implementation is effectively the same as a GET. I'll reach you and chat through it.
Comment by Michael Nitschinger [ 21/Mar/13 ]
Okay after talking this through with Matt, I reran the script to see if the proposed solution (increasing ops/s) fixed the problem.

Interestingly, it turns out when doing set/get's it never hits our anticipated codepath but instead the get operations just time out, the threshold never gets increased and nothing happens. This is something we need to reinvestigate.
Comment by Michael Nitschinger [ 27/May/13 ]
Closing this for now, because our fallback algorithm works as expected. The threshold bug never triggered has been resolved in a different ticket.




[JCBC-274] Jenkins test results failing. Created: 16/Mar/13  Updated: 19/Jul/13  Resolved: 19/Jul/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Infrastructure
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File git_build_description.png     PNG File issues_latest_run_jenkins.png     PNG File latest_build.png    
Issue Links:
Dependency

 Description   
When the jenkins job for java build is run with the latest source in Git and the tests are run, 6 tests are failing. Those are very obvious errors and can be fixed. Screenshot attached.

 Comments   
Comment by Deepti Dawar [ 18/Mar/13 ]
Michael, kindly check why the NPEs are coming in these results.

Thanks
Deepti.
Comment by Deepti Dawar [ 04/Apr/13 ]
Please review few fixes at :

http://review.couchbase.org/#/c/25482/
Comment by Michael Nitschinger [ 19/Jul/13 ]
This has been solved in general, if there are specific other tests failing lets reopen specific issues for them.




[JCBC-272] removing non-bootstrap node with memcached bucket okay, but adding the same node results in failures Created: 13/Mar/13  Updated: 15/Mar/13  Resolved: 15/Mar/13

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.1.5
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Client 1.1.2 with required spymemcached 2.8.11. Server 2.0.0 with a single authenticated memcached bucket. Assertions enabled.

Issue Links:
Duplicate
duplicates JCBC-271 Adding a node to an existing cluster ... Resolved

 Description   
From the attached log, one will see first the removal of a node (click remove -> click rebalance), and things are generally okay.

Following that, you'll see the node added (via the config wizard, joining to node 192.168.1.200), and one configuration received, but nothing actually used yet.

Following that, you'll see the node rebalanced in with a set of assertion errors and then workload ceases.

Notably:
2013-03-13 19:29:03.924 INFO com.couchbase.client.CouchbaseMemcachedConnection: Reconnecting {QA sa=/192.168.1.201:11210, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: 0 Opaque: 139891 Key: pool-25-thread-1:48454, topWop=null, toWrite=0, interested=1}
Exception in thread "Memcached IO over {MemcachedConnection to /192.168.1.200:11210 /192.168.1.202:11210 /192.168.1.201:11210}" java.lang.AssertionError: Attempting to overwrite channel

 Comments   
Comment by Deepti Dawar [ 14/Mar/13 ]
In this scenario, I am encountering something like this :

[INFO 33.11 cbsdk.scenario failover.py:198] Sleeping for 60 seconds after failover
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] Mar 14, 2013 2:17:37 AM net.spy.memcached.MemcachedConnection handleIO
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] INFO: Reconnecting due to exception on {QA sa=/10.3.2.57:11210, #Rops=1, #Wops=0, #iq=0, topRop=Cmd: 0 Opaque: 5552 Key: SimpleKey__REP_149, topWop=null, toWrite=0, interested=1}
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] java.io.IOException: Disconnected unexpected, will reconnect.
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:526)
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:454)
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:247)
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] at com.couchbase.client.CouchbaseMemcachedConnection.run(CouchbaseMemcachedConnection.java:158)
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] Mar 14, 2013 2:17:37 AM net.spy.memcached.MemcachedConnection queueReconnect
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] WARNING: Closing, and reopening {QA sa=/10.3.2.57:11210, #Rops=1, #Wops=0, #iq=1, topRop=Cmd: 0 Opaque: 5552 Key: SimpleKey__REP_149, topWop=null, toWrite=0, interested=1}, attempt 0.
[SDKD(WARNING) 33.12 cbsdk.sdkd.local executor.py:66] Mar 14, 2013 2:17:37 AM net.spy.memcached.protocol.TCPMemcachedNodeImpl setupResend
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] WARNING: Discarding partially completed op: Cmd: 0 Opaque: 5552 Key: SimpleKey__REP_149
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] Mar 14, 2013 2:17:37 AM com.couchbase.sdkd.cbclient.CommandResult warnAbout
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] WARNING: Unknown exception encountered (for operation) future warnings will be suppressed
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] java.lang.RuntimeException: Cancelled
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:169)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:132)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.PendingCommand.onReady(PendingCommand.java:55)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.OperationCollector.submitSync(OperationCollector.java:114)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.OperationCollector.submit(OperationCollector.java:131)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.GetCommandContext.doOneCommand(GetCommandContext.java:64)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.CommandContext.execIter(CommandContext.java:276)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.cbclient.CommandContext.execute(CommandContext.java:311)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.server.SdkServer.executeCommand(SdkServer.java:135)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.server.SdkServer.handleRequest(SdkServer.java:156)
[SDKD(WARNING) 33.13 cbsdk.sdkd.local executor.py:66] at com.couchbase.sdkd.server.SdkServer.run(SdkServer.java:212)
[SDKD(WARNING) 33.16 cbsdk.sdkd.local executor.py:66] Mar 14, 2013 2:17:37 AM com.couchbase.client.CouchbaseMemcachedConnection reconfigure
[SDKD(WARNING) 33.17 cbsdk.sdkd.local executor.py:66] INFO: Scheduling Node /10.3.2.57:11210for shutdown.




[JCBC-270] client does not handle failure of EPT node with memcached bucket Created: 13/Mar/13  Updated: 27/May/13  Resolved: 27/May/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.2
Fix Version/s: 1.1.7
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3 node cluster of 2.0, one memcached bucket authenticated, Couchbase Java Client 1.1.2 with required spymemcached 2.8.11

Attachments: Text File CBSE-472repro-2.txt    

 Description   
When investigating a case, it seems that the dropped configuration when using a memcached bucket is never reestablished. I'm not sure if this is because we rely on timeouts (which we won't get in this case) or something else.

Steps to reproduce:
1. Set up three node cluster
2. Run a constant workload, starting off of one of the nodes (192.168.1.200 in my config)
3. Remove that node from the cluster

Observed behavior:
The client sees the config dropped, but doesn't reconfigure.

Expected behavior:
Client bootstraps off of one of the other nodes.

Attached file shows the config log in this case.

 Comments   
Comment by Michael Nitschinger [ 14/Mar/13 ]
Hey Matt,

one quick update.. I don't know yet if its related, but you've been using the wrong netty version.. your logs show /Users/ingenthr/lib/netty-3.2.5.Final.jar' but the correct one is 3.5.5!

Since netty handles the streaming connection, this may be related - I dont know yet.
Comment by Michael Nitschinger [ 15/Mar/13 ]
Matt, with the change proposed in JCBC-271 and using Netty 3.5.5, I dont see this behaviour (anymore). I tried with 3.2.5 but my connections always go from unbound to connected after some time, even when I wildly failover/rebalance - as it should be. It even waits as we implemented it when I remove both EPT nodes for some time and then add them back it comes back nicely.

Can you try to repro with those two changes and if it still fails lets do a quick screen sharing.
Comment by Matt Ingenthron [ 18/Mar/13 ]
Indeed, I could not reproduce this one after moving to the right dependencies.




[JCBC-271] Adding a node to an existing cluster causes issues with a running memcached bucket Created: 13/Mar/13  Updated: 16/Mar/13  Resolved: 16/Mar/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.2
Fix Version/s: 1.1.5
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 3 node cluster of 2.0, one memcached bucket authenticated, Couchbase Java Client 1.1.2 with required spymemcached 2.8.11

Attachments: Text File CBSE-472repro.txt    
Issue Links:
Duplicate
duplicates JCBC-267 Memcached bucket still fails with all... Closed
is duplicated by JCBC-272 removing non-bootstrap node with memc... Resolved

 Description   
When adding a node to a cluster, I'd observed a number of AssertionErrors and some very odd errors about connections being null.

This popped up when I wasn't expecting an issue, so I'm a bit surprised. The scenario I think is:

Steps to reproduce:
1. Have two nodes running (.200 and .201 in my case here)
2. Add another node (.202 in my case here)

See the logs for what detail is available.

 Comments   
Comment by Michael Nitschinger [ 15/Mar/13 ]
This is the same root cause, the channel gets overriden on add.
Comment by Michael Nitschinger [ 15/Mar/13 ]
http://review.couchbase.com/#/c/25171/
Comment by Matt Ingenthron [ 16/Mar/13 ]
Yes, that fixed the issue. I had some issues on adding that node back, but there was a problem at the cluster side.




[JCBC-266] NPE in ConfigurationProviderHTTP needs to be handled. Created: 12/Mar/13  Updated: 02/Jul/13  Resolved: 02/Jul/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.1.8
Security Level: Public

Type: Bug Priority: Critical
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
NPE in ConfigurationProviderHTTP needs to be handled.
This is seen at quite a lot of places.

Please refer to the integration test results at :

https://docs.google.com/a/globallogic.com/spreadsheet/ccc?key=0AmLvaJ8oRZ-TdHJBNHJ0eGZfbjBaenBSVGM2VS1oNlE#gid=0


Stack Trace :

SDKD: Mar 12, 2013 1:55:39 AM com.couchbase.client.CouchbaseConnectionFactory$Resubscriber run
SDKD: WARNING: Resubscribe attempt failed:
SDKD: java.lang.NullPointerException
SDKD: at com.couchbase.client.vbucket.ConfigurationProviderHTTP.getBucketConfiguration(ConfigurationProviderHTTP.java:149)
SDKD: at com.couchbase.client.vbucket.ConfigurationProviderHTTP.subscribe(ConfigurationProviderHTTP.java:319)
SDKD: at com.couchbase.client.CouchbaseConnectionFactory$Resubscriber.run(CouchbaseConnectionFactory.java:405)
SDKD: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
SDKD: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
SDKD: at java.lang.Thread.run(Thread.java:619)
SDKD: Mar 12, 2013 1:55:39 AM com.couchbase.client.CouchbaseConnectionFactory$Resubscriber run

 Comments   
Comment by Michael Nitschinger [ 12/Jun/13 ]
http://review.couchbase.org/#/c/26862/




[JCBC-231] Handle View Errors (Vbucket & Merge) appropriately Created: 02/Feb/13  Updated: 19/Jul/13  Resolved: 19/Jul/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 1.1.8
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates JCBC-313 View Requests >= HTTP 300 should be r... Resolved
is duplicated by JCBC-307 Hybrid workload with rebalance scenar... Resolved

 Description   
During rebalance out it can happen that the view node is still able to service http connections, but fails with the following errors:

- error Reason: A view spec can not consist of merges exclusively.
or
- no_active_vbuckets Reason: Cannot execute view query since the node has no active vbuckets

If this is happening, the client needs to deal with it. Appropriate solution needs still to be dicussed, but one approach could be to retry on a different node. Note that this needs to work for both async and sync calls.

 Comments   
Comment by Michael Nitschinger [ 02/Feb/13 ]
The corresponding server ticket can be found here: http://www.couchbase.com/issues/browse/MB-7661
Comment by Michael Nitschinger [ 19/Jul/13 ]
See JCBC-313 which now retries these kind of responses properly.




[JCBC-230] View Connection only released when successful (not on failure) Created: 01/Feb/13  Updated: 06/Feb/13  Resolved: 06/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 1.1.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When a view operation is cancelled (timeout or else), the underlying connection for the ConnectionManager is not released and as a result no new view request can go over it.

 Comments   
Comment by Michael Nitschinger [ 06/Feb/13 ]
https://github.com/couchbase/couchbase-java-client/commit/4b733eb37351aa62bab7c6d58d7f689f829ea704




[JCBC-227] client behaves poorly if entire URI list is not available Created: 31/Jan/13  Updated: 01/Feb/13  Resolved: 01/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.1
Fix Version/s: 1.1.2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, if the client has no valid URIs in it's list, it's reported that it will spin trying to connect nonstop. We should add a backoff if the entire list is traversed and unavailable. An exponential backoff (do-while style) with a ceiling and warning to a log is probably best.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
http://review.couchbase.org/#/c/24322/




[JCBC-198] Using ReplicateTo.ONE after node-failover leads to index out of bounds Created: 28/Dec/12  Updated: 27/Feb/13  Resolved: 27/Feb/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0
Fix Version/s: 1.1.3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
using couchbase client library version 1.1.0, server version 2.0.0 community edition (build-1976-rel).
The cluster configuration remains same as in my initial email - three servers with "Enable Failover" property ON, a dedicated port "couchbase" bucket with 1 replica enabled.

When stopping a server the client fails to communicate with the cluster with several errors.

Client initializing -

List<URI> serverList = parseConnectionProperties(port, hosts);
CouchbaseConnectionFactory cf = new CouchbaseConnectionFactory(serverList, cacheName, "");
client = new CouchbaseClient(cf);

Usage of the client to write data -

public void put(final String key, final String value) {
     client.set(key, expirationSeconds, value, PersistTo.MASTER, ReplicateTo.ONE);
 }

During the tests, when cluster is not available, I did a thread dump to the application , see the print below (ajp--127.0.0.1-8020-1@8796). Is it possible that node that is down is in some way a "master" of the data, and since the client.set() method uses PersistTo.MASTER parameter the things do not work?

"ajp--127.0.0.1-8020-1@8796" daemon prio=5 tid=0x8c nid=NA waiting
  java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Unsafe.java:-1)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1033)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:282)
at com.couchbase.client.CouchbaseClient.observe(CouchbaseClient.java:1650)
at com.couchbase.client.CouchbaseClient.observePoll(CouchbaseClient.java:1750)
at com.couchbase.client.CouchbaseClient.set(CouchbaseClient.java:1199)
at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:40)
at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:17)
at com.liveperson.liveEngage.cache.LEUsersCacheManager.setSessionDataCache(LEUsersCacheManager.java:121)
at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.writeLoginDataToJbossCache(LoginVerification.java:246)
at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.verifySsoLogin(LoginVerification.java:87)
at com.liveperson.liveEngage.filters.AuthFilter.doFilter(AuthFilter.java:71)
at com.liveperson.liveengage.authentication.filters.AuthenticationByProtocolFilter.doFilter(AuthenticationByProtocolFilter.java:71)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at com.liveperson.liveEngage.filters.CommonFilter.doFilter(CommonFilter.java:35)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125)
at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91)
at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88)
at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56)
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:490)
at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:480)
at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:2039)
at java.lang.Thread.run(Thread.java:722)

The errors i get during the test after a server was failed over -

 
17:45:41,911 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) [ERROR] [ajp--127.0.0.1-8020-4] Verify Sso Login failed. sessionId - n2b+0EZEzTaM7a64gKKS7k+L.undefined [com.liveperson.liveEngage.ssoIdpLogin.LoginVerification]
17:45:41,911 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) java.lang.RuntimeException: Timed out waiting for operation
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:134)
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.set(CouchbaseClient.java:1188)
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:40)
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:17)
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.LEUsersCacheManager.setSessionDataCache(LEUsersCacheManager.java:121)
17:45:41,912 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.writeLoginDataToJbossCache(LoginVerification.java:246)
17:45:41,913 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.verifySsoLogin(LoginVerification.java:87)
17:45:41,913 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.AuthFilter.doFilter(AuthFilter.java:71)
17:45:41,913 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveengage.authentication.filters.AuthenticationByProtocolFilter.doFilter(AuthenticationByProtocolFilter.java:71)
17:45:41,913 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
17:45:41,913 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.CommonFilter.doFilter(CommonFilter.java:35)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
17:45:41,914 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125)
17:45:41,915 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91)
17:45:41,915 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88)
17:45:41,915 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56)
17:45:41,915 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
17:45:41,915 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:490)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:480)
17:45:41,916 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:2039)
17:45:41,917 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.lang.Thread.run(Thread.java:722)
17:45:41,917 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: tlv-le-couchbase-int1.tlv.lpnet.com/192.168.24.184:11210
17:45:41,918 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:159)
17:45:41,918 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:132)
17:45:41,918 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) ... 28 more

OR

19:21:47,997 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) [ERROR] [ajp--127.0.0.1-8020-3] Verify Sso Login failed. sessionId - 6tJElgOsTUcr-7lWgJlWh2HV.undefined [com.liveperson.liveEngage.ssoIdpLogin.LoginVerification]
19:21:47,997 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) java.lang.ArrayIndexOutOfBoundsException: -1
19:21:47,998 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.util.ArrayList.elementData(ArrayList.java:371)
19:21:47,998 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.util.ArrayList.get(ArrayList.java:384)
19:21:47,998 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.vbucket.config.DefaultConfig.getServer(DefaultConfig.java:81)
19:21:47,998 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.vbucket.VBucketNodeLocator.getServerByIndex(VBucketNodeLocator.java:112)
19:21:47,999 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.observe(CouchbaseClient.java:1621)
19:21:47,999 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.observePoll(CouchbaseClient.java:1750)
19:21:47,999 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.set(CouchbaseClient.java:1199)
19:21:47,999 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:40)
19:21:48,000 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.put(CouchbaseReadWriteCache.java:17)
19:21:48,000 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.LEUsersCacheManager.setSessionDataCache(LEUsersCacheManager.java:121)
19:21:48,000 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.writeLoginDataToJbossCache(LoginVerification.java:246)
19:21:48,001 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.ssoIdpLogin.LoginVerification.verifySsoLogin(LoginVerification.java:87)
19:21:48,001 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.AuthFilter.doFilter(AuthFilter.java:71)
19:21:48,001 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveengage.authentication.filters.AuthenticationByProtocolFilter.doFilter(AuthenticationByProtocolFilter.java:71)
19:21:48,002 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
19:21:48,002 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
19:21:48,002 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.CommonFilter.doFilter(CommonFilter.java:35)
19:21:48,002 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
19:21:48,003 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
19:21:48,003 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
19:21:48,003 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
19:21:48,003 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125)
19:21:48,004 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91)
19:21:48,004 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88)
19:21:48,004 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56)
19:21:48,004 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
19:21:48,005 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
19:21:48,005 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
19:21:48,005 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
19:21:48,005 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
19:21:48,006 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:490)
19:21:48,006 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:480)
19:21:48,006 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:2039)
19:21:48,006 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.lang.Thread.run(Thread.java:722)

Hopefully it's the same one, but just in case, here's another one:
17:41:34,231 ERROR [stderr] (Couchbase View Thread for node tlv-le-couchbase-int2.tlv.lpnet.com/192.168.24.185:8092) 2012-12-10 17:41:34.231 INFO com.couchbase.client.ViewNode: I/O reactor terminated for tlv-le-couchbase-int2.tlv.lpnet.com
17:42:23,120 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) [ERROR] [ajp--127.0.0.1-8020-2] Exception while trying to remove data from cache for session id: y6mpopojZkaJDcWeRDL4J0xp.undefined [org.apache.jsp.views.logout_jsp]
17:42:23,121 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
17:42:23,121 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.util.ArrayList.rangeCheck(ArrayList.java:604)
17:42:23,122 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.util.ArrayList.get(ArrayList.java:382)
17:42:23,122 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.vbucket.config.DefaultConfig.getServer(DefaultConfig.java:81)
17:42:23,122 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.vbucket.VBucketNodeLocator.getServerByIndex(VBucketNodeLocator.java:112)
17:42:23,123 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.observe(CouchbaseClient.java:1624)
17:42:23,123 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.observePoll(CouchbaseClient.java:1753)
17:42:23,124 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.couchbase.client.CouchbaseClient.delete(CouchbaseClient.java:1113)
17:42:23,124 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.remove(CouchbaseReadWriteCache.java:28)
17:42:23,124 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.CouchbaseReadWriteCache.remove(CouchbaseReadWriteCache.java:17)
17:42:23,125 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.cache.LEUsersCacheManager.removeDataFromCache(LEUsersCacheManager.java:132)
17:42:23,125 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.jsp.views.logout_jsp._jspService(logout_jsp.java:82)
17:42:23,125 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
17:42:23,126 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
17:42:23,126 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:369)
17:42:23,126 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:326)
17:42:23,127 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:253)
17:42:23,127 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
17:42:23,127 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
17:42:23,128 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:42:23,128 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.ExceptionFilter.doFilter(ExceptionFilter.java:26)
17:42:23,128 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
17:42:23,129 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:42:23,129 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveengage.authentication.filters.SSOAuthorizationFilterChain.doFilter(SSOAuthorizationFilterChain.java:30)
17:42:23,130 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.AuthFilter.doFilter(AuthFilter.java:65)
17:42:23,130 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveengage.authentication.filters.AuthenticationByProtocolFilter.doFilter(AuthenticationByProtocolFilter.java:71)
17:42:23,130 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
17:42:23,131 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:42:23,131 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at com.liveperson.liveEngage.filters.CommonFilter.doFilter(CommonFilter.java:35)
17:42:23,132 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
17:42:23,132 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
17:42:23,133 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
17:42:23,133 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
17:42:23,133 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125)
17:42:23,134 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91)
17:42:23,134 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88)
17:42:23,134 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56)
17:42:23,135 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
17:42:23,136 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
17:42:23,136 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
17:42:23,136 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
17:42:23,137 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
17:42:23,137 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProcessor.process(AjpAprProcessor.java:490)
17:42:23,137 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.coyote.ajp.AjpAprProtocol$AjpConnectionHandler.process(AjpAprProtocol.java:480)
17:42:23,138 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:2039)
17:42:23,140 INFO [stdout] (AsyncAppender-Dispatcher-Thread-56) at java.lang.Thread.run(Thread.java:722)




[JCBC-190] Allow ComplexKeys to work with Floats (not only Doubles) Created: 19/Dec/12  Updated: 18/Jan/13  Resolved: 18/Jan/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0
Fix Version/s: 1.1.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
See: http://www.couchbase.com/forums/thread/set-floating-point-key-couchbase-client-1-1-0-jar

 Comments   
Comment by Michael Nitschinger [ 28/Dec/12 ]
http://review.couchbase.com/#/c/23605/




[JCBC-165] ComplexKey does not support partial compound keys with single field Created: 04/Dec/12  Updated: 06/Dec/12  Resolved: 06/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: 1.1-beta
Fix Version/s: 1.1.0
Security Level: Public

Type: Bug Priority: Critical
Reporter: Chris Tashjian Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I can't figure out how to do partial compound keys with a single value --

ex. 1 - Specifying both parts of the key:
http://localhost:8092/bucketname/_design/mydesigndoc/_view/myview?group_level=1&startkey=["0",1351742400000]&endkey=["Z",1353073898844]

ex. 2 - Only specifying the first part of the key:
http://localhost:8092/bucketname/_design/mydesigndoc/_view/myview?group_level=1&startkey=["0"]&endkey=["Z"]

I can get ex. 1 to work via the Java client library, but I've had no luck with ex. 2. I've tried setting ranges, complex keys and regular keys. The closest I get is a query string where the "[" and "]" have been escaped which results in a bad URL. I was only able to get this far by manually concatenating the "[" and "]" onto my key and using the query.setKey(String) method.

I believe that ComplexKey should be able to handle ex 2 by calling ComplexKey.of("0") and ComplexKey.of("Z").

 Comments   
Comment by Michael Nitschinger [ 04/Dec/12 ]
Correct, I came across it today as well.
Comment by Michael Nitschinger [ 05/Dec/12 ]
http://review.couchbase.org/#/c/23086/
Comment by Michael Nitschinger [ 06/Dec/12 ]
you can now use the forceArray method on the ComplexKey to do this! .. will be available in 1.1.0.




[JCBC-155] add javadoc for AbstractView, View, SpatialView Created: 27/Nov/12  Updated: 03/Dec/12  Resolved: 29/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When fixing up a test after a merge commit, I noted that I had to create a View object directly. In the process, I found that many of these classes are public, have public ctors, but have no javadoc. The client API takes AbstractView as an argument, so it may be tempting to just construct one. Is it correct to do so? Maybe. Javadoc doesn't tell me much.

The arguments to the ctors could also be expanded as they're a bit outside code style. Specifically, variable names should be meaningful and at least three or more characters if the variable is to be around for a while.

 Comments   
Comment by Michael Nitschinger [ 28/Nov/12 ]
http://review.couchbase.org/#/c/22872/
Comment by Michael Nitschinger [ 29/Nov/12 ]
Pushed to master, will be available in dp5/beta.




[JCBC-152] Document testing procedures for unit/functional tests Created: 21/Nov/12  Updated: 03/Dec/12  Resolved: 29/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
It would be good to have clear documentation in the repo about how to:

(1) run individual tests
(2) change cluster parameters
(3) write more tests

This might be trivial or obvious to do, but very much needed.

For an example of what should be available, see:

(This does not say how to run individual tests though)
https://github.com/couchbase/libcouchbase#run-the-testsuite-towards-a-running-cluster

A more extensive example for a more complex test suite:
https://github.com/couchbase/php-ext-couchbase/blob/master/TESTING.pod

I'm marking this as a 'library' bug because this would require some detailed knowledge of how the tests are written :)

 Comments   
Comment by Michael Nitschinger [ 28/Nov/12 ]
http://review.couchbase.org/#/c/22874/
Comment by Michael Nitschinger [ 29/Nov/12 ]
This has been fixed and is available under TESTING.md in the lib.




[JCBC-148] Issue with Observe API Persist.TWO and 1 dead node: Time Out when doing set operation Created: 18/Nov/12  Updated: 03/Dec/12  Resolved: 03/Dec/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Critical
Reporter: Tug Grall (Inactive) Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: 2 nodes cluster on couchbase-server-community_x86_2.0.0-1947-rel
  1 node on Ubuntu (VM)
  1 node on OS X

Bucket configure with 1 replica

Attachments: Zip Archive CouchbaseSamples.zip    

 Description   
I have a very simple Java program that connect to the 2 nodes and do a set with the following code:

1. So I try to connect to multiple nodes
{code}
            List<URI> couchbaseServerUris = new ArrayList<URI>();
            couchbaseServerUris.add( new URI("http://192.168.0.108:8091/pools") );
            couchbaseServerUris.add( new URI("http://192.168.0.104:8091/pools") );
            CouchbaseClient client = new CouchbaseClient( couchbaseServerUris , "default" , "" );
{code}

2. Then I call the set operation
{code}

        OperationFuture<Boolean> stored = client.set( "my-dummy-key",0, "{\"name\" : \"foo\", \"title\" : \"bar-test\"}", PersistTo.TWO);
{code}

---
So everything is working as expected when the 2 nodes are up.

When I kill 1 node (for example : disconnecting, or stopping, or pausing the Ubuntu VM) I have the following behavior:


When I execute this program:
1- I have an exception saying that 1 node is down : Expected behavior (even if we could avoid a long stack trace)
{code}
2012-11-18 08:14:55.830 WARN com.couchbase.client.vbucket.ConfigurationProviderHTTP: Connection problems with URI http://192.168.0.108:8091/pools ...skipping
java.net.ConnectException: Host is down
{code}


2- When I do the set the program is stopped/blocked until it reaches a network timeout
2012-11-18 08:20:13.462 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
{code}
Error while storing : Observe Timeout - Polled Unsuccessfully for at least 40 seconds.
2012-11-18 08:20:13.466 INFO done : true
done : {OperationStatus success=false: Observe Timeout - Polled Unsuccessfully for at least 40 seconds.}
com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
2012-11-18 08:20:13.467 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
{code}


Note that it is only happening with PersistTo.TWO
if I use PersistTo.MASTER or PersistTo.ONE : the program is executed with no error and no stop
if I use PersistTo.THREE ( or more) : the program is executed, no stop with the expected observe message : ( Error while storing : Requested persistence to 3 node(s), but only 2 are available.
 )




 Comments   
Comment by Tug Grall (Inactive) [ 18/Nov/12 ]
Sample program
Comment by Matt Ingenthron [ 20/Nov/12 ]
I do believe that's actually expected behavior, but let's talk through it to get your opinion.

We have a couple of options in the state of unexpected failure. one is we try our hardest to get the operation requested of us done and we rely on timeouts to keep from blocking forever. The second is that we keep tabs on our connections, and if the connection is down, we fail operations immediately so as to not have the application code waiting for something that may or may not succeed.

Had you gone in and removed the second node (click 'remove' and 'rebalance'), then the client should have done something similar to when you requested three nodes. The failure you describe above is unexpected. Further, the client library doesn't really know if it's temporary or permanent.

Finally, I do want to note, and I think this is well documented, that many things with Observe protocol under them end in timeouts. This is not the only one. Generally speaking, application code should be ready to do *something* in the case of a timeout.
Comment by Matt Ingenthron [ 20/Nov/12 ]
Tug explained this further. The PersistTo.THREE check must be happening after doing some operations, which is a bit late considering this operation can never succeed. The failure should be the same with a cluster that has a down node as it is with a cluster that just doesn't have a primary and to replica locations.
Comment by Mike Wiederhold [ 21/Nov/12 ]
The way Rags wrote this code originally was to do the set and then the observe. The observe part is the part that does all of the checking so the set will actually go through an then you will get the error. Similarly there is no checking for downed nodes and I don't think we actually have the ability to do this at the moment, but I may be wrong.

On another note, one other thing I thing is wrong is returning an OperationFuture from all of the observe functions, but it isn't actually an asynchronous function.
Comment by Michael Nitschinger [ 30/Nov/12 ]
http://review.couchbase.org/#/c/22936/
Comment by Michael Nitschinger [ 03/Dec/12 ]
fixed and will be available in the beta release.




[JCBC-134] resubscriber IllegalArgumentException during topology changes Created: 19/Oct/12  Updated: 23/Jan/13  Resolved: 23/Jan/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.0, 1.1.1
Fix Version/s: 1.1.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Mark Nunberg Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Got this during a swap-rebalance. I don't have any other relevant information to reproduce for now (fwiw I've run this many times and it's the first time I'm seeing it.. though I've just seen it again on the very next run).

During the second run, the cluster rebalance is actually hanging..

Attachments: Text File daschl-4-rebealance_two_nodes.log     Text File daschl-4-restart.log     Zip Archive junit.zip     File log2.txt.bz2     File log2.txt.bz2    
Issue Links:
Dependency

 Description   
Exception in thread "couchbase cluster resubscriber - running" java.lang.IllegalArgumentException: Bucket name cannot be null and must never be re-set to a new object.
        at com.couchbase.client.vbucket.ConfigurationProviderHTTP.subscribe(ConfigurationProviderHTTP.java:240)
        at com.couchbase.client.vbucket.ConfigurationProviderHTTP.finishResubscribe(ConfigurationProviderHTTP.java:215)
        at com.couchbase.client.CouchbaseConnectionFactory$Resubscriber.run(CouchbaseConnectionFactory.java:322)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)


Unfortunately I don't have a whole lot more insight into what's happening, but the stack trace might be helpful to examine.. assigning to myself until I have more info..

 Comments   
Comment by Michael Nitschinger [ 08/Nov/12 ]
Hi Mark,

can you elaborate a bit more whats going on through the test? This particular exception can come up when the java sdk tries to subcribe to a new node when the old connection is closed. I think the scenario should give us a connection to what is happening at runtime in the java sdk.

Thanks,
Michael
Comment by Mark Nunberg [ 12/Dec/12 ]
I've run the Java SDKD tests several times already and cannot reproduce this. Will re-open it if i see it again
Comment by Mark Nunberg [ 21/Jan/13 ]
Seen again at:

http://review.couchbase.org/#/c/24092/
Comment by Mark Nunberg [ 21/Jan/13 ]
So as I mentioned in the bug, I closed it because I haven't seen this error. Just now, both me and Michael encountered this error while running the SDKD tests
Comment by Michael Nitschinger [ 22/Jan/13 ]
Attaching the logs for http://review.couchbase.org/#/c/24092 changeset 4 (prefixed with daschl-4-) on some test runs.

Deepti is currently running the changeset against the brun cluster, expect some info in 2 hours.
Comment by Michael Nitschinger [ 22/Jan/13 ]
./stester -i 20devcluster.ini --service ALL --svcaction RESTART --num_nodes 3 --no_fo 1 -c failover.Once --dsw_timeres 1 -d -o restart.log -C 127.0.0.1:8050
Comment by Michael Nitschinger [ 22/Jan/13 ]
./stester -i 20devcluster.ini -c rebalance.Once --mode out --rbcount 2 --dsw_timeres 1 -d -o rebealance_two_nodes.log -C 127.0.0.1:8050
Comment by Deepti Dawar [ 22/Jan/13 ]
Attaching the functional test results.
This was run against a local 2.0.0 node.
Pass rate is better this time - 92%.
Comment by Deepti Dawar [ 22/Jan/13 ]
For the Hybrid tests - failures are still coming.

Attaching the intermittent log.
Comment by Deepti Dawar [ 22/Jan/13 ]
The error that seems to be problematic in the unit test logs is this one -

'Timeout occurred. Please note the time in the report does not reflect the time until the timeout.
junit.framework.AssertionFailedError: Timeout occurred. Please note the time in the report does not reflect the time until the timeout.'

Most of the issues coming due to timeout.

Note : that these tests were run against a local cluster. Hence, such problems should not be occurring.
Comment by Michael Nitschinger [ 23/Jan/13 ]
Merged in today, right before the 1.1.1 release.




[JCBC-125] Don't cast view documents to strings Created: 03/Oct/12  Updated: 03/Dec/12  Resolved: 08/Nov/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.1-beta
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: view
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: All


 Description   
When objects are stored as non-strings in Couchbase and then loaded through a View, the SDK currently casts every document to a string. This works fine for JSON documents, but as soon as you want to store serialized objects it breaks.

Implicit casting is not needed in this place (see fix).

 Comments   
Comment by Michael Nitschinger [ 03/Oct/12 ]
Here is a quick sample on how to reproduce it, but I'm adding a correct test case to the code as well (the view used is just the default "emit key and null" view with no reduce).


import com.couchbase.client.CouchbaseClient;
import com.couchbase.client.CouchbaseConnectionFactory;
import com.couchbase.client.protocol.views.Query;
import com.couchbase.client.protocol.views.View;
import com.couchbase.client.protocol.views.ViewResponse;
import com.couchbase.client.protocol.views.ViewRow;
import java.io.IOException;
import java.net.URI;
import java.util.Arrays;
import java.util.Date;
import java.util.Iterator;


public class BinaryviewTest {

  /**
   * @param args the command line arguments
   */
  public static void main(String[] args) throws IOException {

    // Initialize Connection
    CouchbaseClient cb = new CouchbaseClient(
      new CouchbaseConnectionFactory(
        Arrays.asList(
          URI.create("http://192.168.1.105:8091/pools")
        ),
        "default",
        ""
      )
    );

    // Store binary objects
    Object ob1 = new Date();
    cb.set("date1", 0, ob1);

    Object result = cb.get("date1");
    System.out.println(result.getClass());

    Query query = new Query();
    query.setIncludeDocs(true);
    View view = cb.getView("testing", "binary");
    ViewResponse response = cb.query(view, query);

    Iterator<ViewRow> iterator = response.iterator();
    ViewRow row;
    while(iterator.hasNext()) {
      row = iterator.next();
      Object obj = row.getDocument();
      System.out.println(obj.getClass());
    }
    cb.shutdown();
  }
}

When run two times, this raises the exception as described here: http://www.couchbase.com/forums/thread/couchbase-2-view-non-json-docs

I'll update it when I have the fix and test ready.
Comment by Michael Nitschinger [ 03/Oct/12 ]
See http://review.couchbase.com/#/c/21305/
Comment by Michael Nitschinger [ 08/Nov/12 ]
pushed to master, will be available in dp5.




[JCBC-74] Implement observe command Created: 12/Jul/12  Updated: 22/Aug/12  Resolved: 22/Aug/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.1dp2
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Matt Ingenthron Assignee: Raghavan Srinivas (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Add the basic low level observe command

 Comments   
Comment by Raghavan Srinivas (Inactive) [ 22/Aug/12 ]
This is implemented in dp2 and will be enhanced post dp2.




[JCBC-97] documentation needs a discussion on JSON mapping, getting started needs an example of using JSON Created: 10/Aug/12  Updated: 22/Aug/12  Resolved: 22/Aug/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 1.1dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Raghavan Srinivas (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
At one point, I thought we had a discussion of JSON document usage in the getting started and on the web page. It's not there now, but we need to have a discussion about how a Java developer maps their high-level objects into JSON.

 Comments   
Comment by Raghavan Srinivas (Inactive) [ 22/Aug/12 ]
There were a couple of issues. The document changes were reverted back. That's fixed. The Getting Started has been enhanced to persist some JSON documents.




[JCBC-72] View query returns null Created: 05/Jul/12  Updated: 11/Jul/12  Resolved: 11/Jul/12

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1dp
Fix Version/s: 1.1dp2
Security Level: Public

Type: Bug Priority: Critical
Reporter: Sharon Barr (Inactive) Assignee: Mike Wiederhold
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
viewResponse return null, and i can't figure out why:

• I execute my code
CouchbaseClient c = new CouchbaseClient(baseURIs, "default", "");
View view = c.getView("dev_lenders", "country_count");

Query query = new Query();
Stale stale=Stale.OK;
query.setStale(stale);
query.setGroup(true);
ViewResponse viewResponse = c.query(view, query);

• My viewRespons is null.

• Checking the couchdb.1 log, I see:
[couchdb:info] [2012-07-04 21:40:50] [ns_1@127.0.0.1:<0.24865.33>:couch_log:info:39] 10.32.5.29 - - GET /default/_design/dev_lenders/_view/country_count?group=true&stale=ok 200

• Running this URL directly against my server I do get results:
http://10.2.1.12:8092/default/_design/dev_lenders/_view/country_count?group=true&stale=ok
{"rows":[
{"key":null,"value":4171},
{"key":"AE","value":5},
{"key":"AF","value":1},
{"key":"AM","value":2},
...


Not sure what logs i can provide to debug this further. let me know how i can help.

 Comments   
Comment by Sharon Barr (Inactive) [ 05/Jul/12 ]
On slight variation from the above issue, i did mange to get an error from the server via the HTTP request
{"error":"query_parse_error","reason":"Invalid URL parameter 'group' or 'group_level' for non-reduce view."}

but the client still returns null.
Comment by Mike Wiederhold [ 09/Jul/12 ]
http://review.couchbase.org/#change,18094




[JCBC-59] authentication not working correctly with CouchbaseConnectionFactoryBuilder Created: 04/Jun/12  Updated: 06/Aug/12  Resolved: 09/Jun/12

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.0.2
Fix Version/s: 1.0.3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Raghavan Srinivas (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The use of the CouchbaseConnectionFactoryBuilder like so yields a client that does not authenticate correctly.

CouchbaseConnectionFactoryBuilder cfb = new CouchbaseConnectionFactoryBuilder();
CouchbaseConnectionFactory cf = cfb.buildCouchbaseConnection(servers, "default_jOWPgw", "default_jOWPgw", "testing");
CouchbaseClient client = new CouchbaseClient(cf);

 Comments   
Comment by Matt Ingenthron [ 04/Jun/12 ]
http://review.couchbase.org/#change,16765




[JCBC-22] Spring Support for CouchbaseClient Created: 26/Jul/11  Updated: 31/Jan/13  Resolved: 31/Jan/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Mike Wiederhold Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We need to add support for the following Spring Integration Parts:

- General Info on how to use the Client with Spring Beans
- Caching Support through @Cacheable annotations
- Spring Data Support (yet to be defined).

Current progress of the (yet unofficial support) can be tracked here: https://github.com/couchbaselabs/couchbase-spring

 Comments   
Comment by osk [ 19/Nov/12 ]
Need any help on this?
Comment by Michael Nitschinger [ 31/Jan/13 ]
I'm going to close this ticket because it's not part of the couchbase client.

All ongoing efforts can be found here: https://github.com/couchbaselabs/spring-data-couchbase
Comment by Michael Nitschinger [ 31/Jan/13 ]
This is now part of a separate project.




[JCBC-480] replica read with NOT_EXISTS never completes Created: 25/Jun/14  Updated: 01/Jul/14  Resolved: 01/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.3
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-151] Client does timeout on connect in specific java environments (was believed to be java7 related). Created: 21/Nov/12  Updated: 21/Jul/14  Resolved: 31/Dec/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.3.0
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to JCBC-388 Refactor View Connection Management Resolved

 Description   
People report that the Client does not work with 1.7. Here is a sample stack trace:

2012-11-20 00:29:11.228 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-20 00:29:11.240 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@3f757322
2012-11-20 00:29:11.502 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-20 00:29:11.505 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-20 00:29:11.647 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-20 00:29:12.051 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-20 00:29:12.060 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
2012-11-20 00:29:17.111 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [localhost/127.0.0.1:8092(closed)]
java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:68)
at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:428)
at Example1.main(Example1.java:43)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:80)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:65)
... 2 more
2012-11-20 00:30:12.168 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-20 00:30:12.177 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Disconnected from the target VM, address: '127.0.0.1:65280', transport: 'socket'

Process finished with exit code 0


Also See http://www.couchbase.com/forums/thread/java-asyncconnectionmanager-timed-out-waiting-operation-please-help-console-log-included
And http://stackoverflow.com/questions/13466010/using-java-api-in-scala-to-query-views-in-couchbase-throws-timeout-exception?utm_source=twitterfeed&utm_medium=twitter

 Comments   
Comment by Quinn Slack [ 27/Nov/12 ]
This doesn't appear to be strictly related to Java7. I was able to get similar code to work on Java7. I even simulated loading a Play2 environment with other JARs that could potentially cause conflicts. It's possible that my simulation of the Play2 environment was insufficient and that Play2 does other stuff...

Code at https://github.com/sqs/couchbase-scala-example. Run with:

CBURL=http://localhost:8091/pools CBPASSWORD=mypassword sbt -Dconfig.file=conf/application.conf '~run'

Here is the output on my system that shows it's on Java7 (openjdk) and that shows the expected output from the views:

on java version 1.7.0_09
[info] play - Application started (Prod)
2012-11-27 01:34:27.115 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-27 01:34:27.120 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@6267e5a2
2012-11-27 01:34:27.147 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-27 01:34:27.149 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-27 01:34:27.176 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-27 01:34:27.282 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-27 01:34:27.288 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
Res = List(com.couchbase.client.protocol.views.ViewRowNoDocs@4667820f, com.couchbase.client.protocol.views.ViewRowNoDocs@358bcae5, com.couchbase.client.protocol.views.ViewRowNoDocs@6cb59bd9, com.couchbase.client.protocol.views.ViewRowNoDocs@70afb51, com.couchbase.client.protocol.views.ViewRowNoDocs@61f98673)
2012-11-27 01:34:27.396 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-27 01:34:27.403 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Comment by Michael Nitschinger [ 27/Nov/12 ]
Are you running on jdk7 on mac? Also, when you use play 2.1 it should work. Play2-related issues more were because of netty version incompatibilities.
Comment by Quinn Slack [ 27/Nov/12 ]
This is jdk7 on Arch Linux and Play 2.1-RC1.

I tried using a different netty, but no luck. What worked for me was moving calls to "new CouchbaseClient" out of static "object" initializers, since they appeared to be getting called in the Netty I/O thread (I got this exception in new CouchbaseClient: java.lang.IllegalStateException: await*() in I/O thread causes a dead lock or sudden performance drop. Use addListener() instead or call await*() from a different thread).
Comment by Michael Nitschinger [ 28/Nov/12 ]
Hi Quinn, this issue is a different one! The issue described here is that people seem to find connection timeouts with java 7.
Comment by Quinn Slack [ 28/Nov/12 ]
They may be related--or may just have similar symptoms. I was not getting timeout exceptions, but I was seeing view requests hang indefinitely until I made the fix described above.
Comment by dragos [ 13/Mar/13 ]
Hi Michael, can you help with this error ? I am trying to find a workaround or something to work. This is the only thing which is keeping us from using Couchbase. I also posted here : http://www.couchbase.com/forums/thread/couchbase-connectivity-problem-aws-vpc . As you can see in the times the timeout is received really fast.

2013-03-13 11:40:47.702 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.X.XXX:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 11:40:52.712 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@5cbfe9d
2013-03-13 11:40:52.840 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.X.XXX/10.0.X.XXX:11210
2013-03-13 11:40:57.860 INFO com.couchbase.client.ViewConnection: Added 10.0.X.XXX to connect queue
2013-03-13 11:40:57.863 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 11:40:58.070 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 11:40:58.077 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.X.XXX:8092 - Session request successful
2013-03-13 11:41:03.113 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.X.XXX/10.0.X.XXX:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:63)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi dragos,

can you please post the full code where you connect to couchbase? (including factories and such)

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Mike, this is the POC client for Couchbase:
package couchbase.training.view.poc;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URI;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.TimeUnit;

import org.apache.commons.io.IOUtils;
import org.json.JSONException;
import org.json.XML;

import com.couchbase.client.CouchbaseClient;
import com.couchbase.client.CouchbaseConnectionFactory;
import com.couchbase.client.CouchbaseConnectionFactoryBuilder;
import com.couchbase.client.protocol.views.ComplexKey;
import com.couchbase.client.protocol.views.Query;
import com.couchbase.client.protocol.views.Stale;
import com.couchbase.client.protocol.views.View;
import com.couchbase.client.protocol.views.ViewResponse;
import com.couchbase.client.protocol.views.ViewRow;

public class CouchbasePOC
{
public static void main(String[] args) throws FileNotFoundException, JSONException, IOException
{
System.out.println("Hello from CouchBase");

// Set the URIs and get a client
final List<URI> uris = new LinkedList<URI>();

// Connect to localhost or to the appropriate URI(s)
uris.add(URI.create("http://10.0.7.164:8091/pools"));

final String mappedName="problems";
long start = System.currentTimeMillis();

CouchbaseConnectionFactoryBuilder m_couchbaseConnectionFactoryBuilder = new CouchbaseConnectionFactoryBuilder();
m_couchbaseConnectionFactoryBuilder.setMaxReconnectDelay(10000);
m_couchbaseConnectionFactoryBuilder.setOpQueueMaxBlockTime(100);
m_couchbaseConnectionFactoryBuilder.setOpTimeout(20000);
m_couchbaseConnectionFactoryBuilder.setShouldOptimize(true);
m_couchbaseConnectionFactoryBuilder.setViewTimeout(300000);

CouchbaseConnectionFactory connFactory = m_couchbaseConnectionFactoryBuilder.buildCouchbaseConnection(uris, mappedName, "");

CouchbaseClient client = null;
try
{
client = new CouchbaseClient(connFactory);
}
catch (IOException e)
{
System.err.println("IOException connecting to Couchbase: " + e.getMessage());
System.exit(1);
}

final View view = client.getView(mappedName, mappedName);
final Query query = new Query();

final ComplexKey key = ComplexKey.of("problem",null,null,null,null);
query.setKey(key);
query.setStale( Stale.UPDATE_AFTER );
int counter=0;
try
{
final ViewResponse result = client.query(view, query);
for(ViewRow row : result)
{
counter++;
System.out.println(row.getId());
}
}
catch(java.lang.VerifyError e)
{
System.out.println("Error: " + e.getMessage());
}

System.out.println("time: " + (System.currentTimeMillis()-start) );
System.out.println("documents: " + counter );

client.shutdown(3, TimeUnit.SECONDS);
System.exit(0);
}
}
Comment by dragos [ 13/Mar/13 ]
I hope you would not mind that i called you Mike and not Michael.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hehe no worries.

Can you do me a favor and remove the factory for a second and just use CouchbaseClient() without anything else? Let me know if the problem still persists or not! Thanks
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
First i want to thank you for the quick feed-back. Your owesome !!!

Before using the factory i used the CouchbaseClient(uris, bucketName, "") which is also timing out. I added the factory to be able increase the timeout times for the connection but still the same problem.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hmm okay, so its not the factory (we had some issues with default values in the past, thats why I'm asking).

Can you pin me down your operating system and exact java version you're using?Thanks for your help!
Comment by Michael Nitschinger [ 13/Mar/13 ]
Also, since you're so responsive (I hope we can pin this down finally!).. Can you run the same code with DEBUG log enabled (see https://code.google.com/p/spymemcached/wiki/Logging).. just use Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST); instead of Logger.getLogger("net.spy.memcached").setLevel(Level.FINEST);

Thanks!
Comment by dragos [ 13/Mar/13 ]
This is my test environment for the client which will be similar for production.
OS Ubuntu Server 11.10 x64
java -version result:
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

My network infrastructure is like this:
In an AWS virtual private center (VPC) i have a subnet and in this subnet there are 2 machines. The client machine and the Couchbase server. The machines have identical OS and java version as mentioned above. I have open the ports and also checked the network ACL. The traffic is free to go,no restriction. If i am using a rest query with curl for the view i receive the result, the problem is only in the java SDK.
If needed more information please ask. I will gladly offer it.

Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

yes, if you can please rerun the script with DEBUG logging enabled (see last comment).. Thanks very much in advance!
Comment by dragos [ 13/Mar/13 ]
I've setted the logger to finest and this is the error log obtained. Also i tested again the connection with curl and no problemes.

Mar 13, 2013 1:30:14 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
2013-03-13 13:30:14.679 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 13:30:19.689 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@4e99353f
2013-03-13 13:30:19.810 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.7.164/10.0.7.164:11210
2013-03-13 13:30:24.828 INFO com.couchbase.client.ViewConnection: Added 10.0.7.164 to connect queue
2013-03-13 13:30:24.830 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 13:30:25.025 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 13:30:25.033 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.7.164:8092 - Session request successful
2013-03-13 13:30:30.083 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:68)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
        ... 2 more

curl result

curl http://10.0.7.164:8092/problems/_design/dev_problems/_view/problems?stale=false\&connection_timeout=60000\&limit=10\&skip=0
{"total_rows":0,"rows":[
]
}
[4]+ Done
Comment by dragos [ 13/Mar/13 ]
I am available for next steps in debuging. With what can i help next ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, hmm no there should be more output (DEBUG), not just info... you need to copy this whole snippet before your code actually executes:

        // Tell spy to use the SunLogger
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);


I guess you're running it from your IDE?
Comment by dragos [ 13/Mar/13 ]
I am building the client on local IDE then uploading it to EC2 test machine. The snipped of code word and now i have a full log.

Mar 13, 2013 1:53:10 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default?uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default/buckets?v=120523822&uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM net.spy.memcached.MemcachedConnection createConnections
INFO: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Updating nodesMap in VBucketNodeLocator.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Adding node with address 10.0.7.164:11210.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Node added is {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8}.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=false, c=true, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
INFO: Connection state changed for sun.nio.ch.SelectionKeyImpl@2c76e369
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added Cmd: 10 Opaque: 1 to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=1, topRop=null, topWop=null, toWrite=0, interested=8}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 24 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: Cmd: 10 Opaque: 1 and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added SASL auth operation to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=true, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=true, w=false, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=1, #Wops=0, #iq=0, topRop=SASL auth operation, topWop=null, toWrite=0, interested=1})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 37 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.auth.AuthThread$1 receivedStatus
INFO: Authenticated to 10.0.7.164/10.0.7.164:11210
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: SASL auth operation and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewConnection createConnections
INFO: Added 10.0.7.164 to connect queue
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseClient <init>
INFO: viewmode property isn't defined. Setting viewmode to production mode
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP subscribe
FINE: Subscribing an object for reconfiguration updates com.couchbase.client.CouchbaseClient
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686] OPEN


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is true and Channel State is OPEN.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] BOUND: /10.0.7.213:55729


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.213:55729 and Channel State is BOUND.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] CONNECTED: /10.0.7.164:8091


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.164:8091 and Channel State is CONNECTED.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: STATUS: 200 OK
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: VERSION: HTTP/1.1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Cache-Control = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Content-Type = application/json; charset=utf-8
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Date = Wed, 13 Mar 2013 13:55:42 GMT
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Pragma = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Server = Couchbase Server 2.0.0-1976-rel-enterprise
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Transfer-Encoding = chunked
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER:

Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: CHUNKED CONTENT {
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: {"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3066
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 2048
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: ],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3072
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 361
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketMonitor logFiner
FINER: Getting server list returns this last chunked response:
{"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@4cf3fdc4
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@fadb83cf
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager processConnectionRequests
INFO: Opening new Couchbase HTTP connection
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback completed
INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, thanks for the log..

interestingly, there is nothing unusual in this.. I'll investigate further and come back to you if I need more info..

thanks so far!
Comment by dragos [ 13/Mar/13 ]
Thank you for spending time on this. Looking forward for a fix :) .
Comment by Michael Nitschinger [ 13/Mar/13 ]
interestingly, the connection is open and then times out...

INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]

this sounds highly suspicious to me, maybe this is a bug in apache httpcore-nio...
Comment by dragos [ 13/Mar/13 ]
The time out is displayed after 5 seconds. As you suggested i removed the factory and the detailed logged above i obtained with using CouchbaseClient(uris, mappedName, ""). Do you have some way of testing the httpCore-nio ? Should i try to look for latest version of httpCore-nio.. ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi,

yes you could try that.. I ran the test suite with 4.2.3 which is the latest one and there were no errors, so our code should work with it. You need to exchange the httpcore and httpcore-nio with the latest 4.2.3 jars

I don't think this is 100% the issue, but it helps us get rid of one more factor!

Thanks!
Michael
Comment by Michael Nitschinger [ 13/Mar/13 ]
Okay, the 5 second period can come from this:

 HttpParams params = new SyncBasicHttpParams();
      params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 5000)
          .setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 5000)
....


SO_TIMEOUT is: Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).

and CONNECTION_TIMEOUT is: Determines the timeout in milliseconds until a connection is established.


Dragos, can it be the case that it takes more than 5 seconds to return a result when using curl or so? That would explain the "fail fast" timeout here.. not saying that these times are okay this way, just to find the root cause..

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
I've made a test with httpcore-nio-4.2.jar and with httpcore-4.2.1.jar and your intuition is good. The httpcore is not the problem because the timeout is still there. With curl the results is back fast, in less then half a second. So the problem must be somewhere in the java SDK in the way the view is retrieved.
Comment by dragos [ 13/Mar/13 ]
I've moved the Couchbase server to the same machine as the client to have all in same environment for testing. Now all is working fine but it is not possible for our product to stay on the same machine as Couchbase server for obvious reasons. In production we will have more then one app machine which we will try to access the Couchbase cluster.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

okay so its definitely something with those timeouts.. The thing is we can increase them for sure, but the question is if it would work for your application given the long network latency - and if your application stays performant that way.

When you're running curl on a request, how long does it take?
Comment by dragos [ 14/Mar/13 ]
The curl request is very fast (a few milliseconds, and less then 0.5 seconds i am sure) and the network latency it is not a problem. Some how the couchbase client is taking to much time connecting. When it need to connect to a network machine in the same lan it takes more then 10 seconds. I experimented this in EC2 environment and on my local machine with a VM. If couchbase is local with the test app, the couchbase client is connecting fast.
Do you have any points on how a connection should be setup for couchbase client ? I am trying to use the factory builder and the connection factory from java sdk but still with no success.
Comment by Tim Smith (Inactive) [ 14/Mar/13 ]
Dragos,

I can think of two separate things to try. One is to increase the SO_TIMEOUT to 8000, CONNECTION_TIMEOUT to 12000. Identify what's being hit.

Probably more effective would be to use Wireshark to capture all traffic on the client machine, and see what is really happening. Start the capture (with tshark command-line tool, for example), then run the test client, let it time out, then run the curl command, let it succeed, then stop the capture. Gzip the capture data and attach it to this issue.

Tim
Comment by dragos [ 14/Mar/13 ]
Hi Tim,
I've started some load tests to be able to evaluate Couchbase, so this means i moved Couchbase server on the same machine as the load test client. I tried to use a pool of couchbase clients always connected to the server, and after a while the test failed because couldn't create more connections. From my initial research i seen that couchbase client is managing his own connection can you tell me how this is done or what are the policies of shutting down connections ? Can you point me to some best practices for couchbase client connectivity ?
Comment by Tim Smith (Inactive) [ 14/Mar/13 ]
I will handle that question about connection pooling, etc., separately, since it is unrelated to this bug about timeouts.

On this specific bug report, the current state as I understand it is that: curl returns very quickly with the correct results. The Java client times out after 5 seconds without returning the results.

My suggestion is to set up a very simple Java client that performs a single view query, and to get a packet capture of both curl and the Java client, to identify what the two are doing differently and why the curl succeeds but the Java client fails.
Comment by Jahangir Zinedine [ 27/Mar/13 ]
Hi Michael,

Any update on this issue?
I faced the same issue and here is my environment:
JRuby 1.7.3
Oracle JDK 1.7(with 1.6 the same thing happened)
Couchbase 2.0.1
Java SDK 1.1.3(with 1.1.4 the same thing happened)
And latest netty and http-core and other jar files.

Appreciate your help.

Thanks,
Jani
Comment by dragos [ 01/Apr/13 ]
Hi All,
Finally after a long time i found the workaround needed for the java client to not timeout. As has been suggested by the Couchbase team to use a host name for the Couchabase server i applied the same fix but to the client machine. Meaning on the client machine were the java Couchbase client was timing out ( in Amazon VPC ) we added a hostname for the Couchbase server. Our test URI from http://10.0.7.164:8091/pools has become http://couchbase:8091/pools. This workaround is also fixing a 10 seconds connectivity for the client.

On Ubuntu the hostname file is at: /etc/hosts
On Windows 7 the hostname file is at: C:\Windows\system32\drivers\etc\hosts

Example of host name entry in client machine:
10.0.7.146 couchbase
Comment by Michael Nitschinger [ 02/Apr/13 ]
Thanks very much dragos for tracking this down!

Does this mean that in your environment after the change, you're not seeing any performance impact anymore? Everything works as expected and performant?

Thanks!
Comment by dragos [ 02/Apr/13 ]
Hi Michael,
In this moment the client does not time out and the time taken by java couchbase client to connect takes around 1.5 seconds from 10 seconds which we can consider a normal behavior. As for performance with this workaround we can continue the POC and evaluate the performance. As a first impression i can say that we see a good speed but i will keep my reservation until final POC is tested and we can achieve a full load test.

What i want to emphasize that this is a welcomed workaround but it is not a solution on a longer term for our cloud production site. In a scenario where we need to create new machines and maybe new couchbase clusters it involves manual or specific automatic changes to the hosts file and of course extra managing of this. Bottom line is that we are waiting the fix on the couchbase java sdk 1.1.5 which will make this workaround obsolete.

Also i feel the need to mention that this workaround was found in collaboration with the Couchbase team which provided the basic information for the workaround to be found, so thank you guys.
Comment by Michael Nitschinger [ 02/Apr/13 ]
Hi Dragos,

I'm not sure if there is a thing that we can fix inside the client when this is a DNS/hostname resolution issue with the OS and amazon?

We would definitely need to investigate further, but all that the client does is try to use the hostname/IP provided during bootstrap. If nothing comes back or the host can't be found, we can't do much about it. Anyway, I think we need to do some more investigation here to really see whats the problem.
Comment by aboyev [ 09/Sep/13 ]
Hello, Michael!
I have the same problem. Here is my setup:
- Couchbase 2.1.1
- Java SDK 1.1.9
- Java 1.6.0_27 (and also tried 1.7)
[ERROR] getView | Exception while doing getView: Timed out waiting for operation
[ERROR] Exception while doing getView: I/O reactor has been shut down
It surely has something to do with the client, because when I launch second client it works properly for a while.
Also, it seems to happen under high loads more frequently
Comment by gaurang jhawar [ 24/Oct/13 ]
How did u change the "Meaning on the client machine were the java Couchbase client was timing out ( in Amazon VPC ) we added a hostname for the Couchbase server. Our test URI from http://10.0.7.164:8091/pools has become http://couchbase:8091/pools. This workaround is also fixing a 10 seconds connectivity for the client." in AWS .. I have EC2 instance for which i want to set the hostname and connect through it like you did... Can you tell me the exact steps for that
Comment by Michael Nitschinger [ 25/Oct/13 ]
Hi Gaurang,

are you also exhibiting a delay on connect? Or during operations? I think these two may be related but should be looked at separately and defined. What environment, client, workload and so on are you running?
Comment by gaurang jhawar [ 25/Oct/13 ]
My env is JDK 1.7.0_09
Client is my local machine ...

Couchbase server installed on aws ..

Reconnecting due to failure to connect to {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:423)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:261)
at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:288)
2013-10-24 22:14:01.893 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}, attempt 1.
2013-10-24 22:14:05.903 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
2013-10-24 22:14:09.632 INFO com.couchbase.client.ViewConnection: Added 172.31.9.124 to connect queue
Comment by Michael Nitschinger [ 25/Oct/13 ]
One thing to note here is that this is not a good idea. You have "the internet" between your computer (client) and the server which adds latency. Can you try running the application also in an AWS instance and see if the error goes away?
Comment by gaurang jhawar [ 25/Oct/13 ]
I gave the elastic amazon IP in the url and also tried what dragos tried but that also didn`t work for me .. I made ALL TCP ports on firewall ON ... And I am not sure why it reconnects to the IP or fails to connect and load data in the bucket.

But its the same case as dragos ... for production that`s not viable .. since not everything can be on AWS.

Also I can connect to the same instance via telnet and http(browser) .. I don`t know why I am facing this issue with Java 1.7.

"2013-10-24 22:14:01.893 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}, attempt 1.
2013-10-24 22:14:05.903 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
2013-10-24 22:14:09.632 INFO com.couchbase.client.ViewConnection: Added 172.31.9.124 to connect queue
2013-10-24 22:14:09.632 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
in INIT **************.
Here is the entity set : users
2013-10-24 22:14:09.868 WARN com.couchbase.client.CouchbaseConnection: Node expected to receive data is inactive. This could be due to a failure within the cluster. Will check for updated configuration. Key without a configured node is: 0.
30 Seconds: Load is in progress
2013-10-24 22:14:09.993 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-24 22:14:12.037 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@6dae04e2
2013-10-24 22:14:12.037 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to failure to connect to {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
java.net.ConnectException: Connection timed out: no further information"

NOTE here it tried to reconnect to the client and says its inactive and I am not sure if this is a firewall issue or something else.
Comment by Michael Nitschinger [ 25/Oct/13 ]
Can you provide debug log information for the connect process? Maybe this helps pinning down where the issue is. (you can read here how to do it if you don't know already) :http://nitschinger.at/Logging-with-the-Couchbase-Java-Client

thanks
Comment by gaurang jhawar [ 25/Oct/13 ]
http://s000.tinyupload.com/index.php?file_id=46832814372422198765
Comment by Michael Nitschinger [ 25/Oct/13 ]
Thanks, can you also share the code you use?

In the code it looks like you are creating the CouchbaseClient object twice.
Please note that the bootstrapping succeeds (!) it is different from the ticket reported here.

Are you sure port 11210 is open from client to server? It looks like something is blocking it, http seems to work fine.
Comment by gaurang jhawar [ 25/Oct/13 ]
http://s000.tinyupload.com/index.php?file_id=34200463437173886967



ya the port is opened.

http://s000.tinyupload.com/index.php?file_id=06508245784961590671

Both inbound / outbound on aws as well as my local machine firewall ports are open
Comment by gaurang jhawar [ 26/Oct/13 ]
Any updateS?
Comment by couchbase-user [ 27/Oct/13 ]
This is a nightmare, can't believe no solution has been found yet for over a year now.....
Comment by Michael Nitschinger [ 28/Oct/13 ]
Hi couchbase-user, are you also running app/db over the internet or on a specific ec2 environment?
Comment by couchbase-user [ 28/Oct/13 ]
hey, actually I'm running them all on a local small network.
I am using ubuntu 12.4 on all machines and running java application on headnode to get view data and store them locally.
Comment by Michael Nitschinger [ 28/Oct/13 ]
Hm, I wonder why you are hitting this, I'm not sure its related.. do you get the same exception as the top of the ticket here? Can you provide debug logs so we can see at which point it is happening?

Maybe we need to tweak socket timeouts.
Comment by couchbase-user [ 28/Oct/13 ]
hey, here is my execution log:

28-Oct-2013 09:29:09 com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
2013-10-28 09:29:09.618 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.100:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:09.620 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:09.620 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.102:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:14.635 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@7f09fd93
2013-10-28 09:29:14.635 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@79a5f739
2013-10-28 09:29:14.636 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@68e6ff0d
2013-10-28 09:29:14.656 INFO com.couchbase.client.ViewConnection: Added Cloud-assistant.local to connect queue
2013-10-28 09:29:19.663 INFO com.couchbase.client.ViewConnection: Added 192.168.0.102 to connect queue
2013-10-28 09:29:19.666 INFO com.couchbase.client.ViewConnection: Added 192.168.0.100 to connect queue
2013-10-28 09:29:19.667 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-10-28 09:29:19.788 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-10-28 09:29:19.794 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /192.168.0.102:8092 - Session request successful
2013-10-28 09:29:24.815 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [192.168.0.102/192.168.0.102:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
    at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:483)
    at HelloWorld.Export_Couchbase_View(HelloWorld.java:85)
    at HelloWorld.main(HelloWorld.java:287)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
    at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)

I thought this is exactly the same as the error at the top of this thread....
Comment by Michael Nitschinger [ 28/Oct/13 ]
In your logs I can see that 192.168.0.101 is changed to Cloud-assistant.local. Could it be that this is a DNS issue in your environment?
Also please make sure that ports 8091, 8092 (!) and 11210 are reachable on each node.

Both the socket and connection timeout for views are set to 5 second which is already "quite high" if you want to run a real workload over it.
Comment by couchbase-user [ 28/Oct/13 ]
I'm not sure what the DNS has to do with it or how to check it, something to note here is that sometimes it works fine (very rare).
I have also just tried to change the switch but still the same problem.
I'm working on investigating the DNS and firewall issue. thanks
Comment by Michael Nitschinger [ 28/Oct/13 ]
Okay, if you go directly to the view url, how long does it take you to? like with curl... so we can see how far you are over the 5 seconds and if you are under it then there's something else going on.
Comment by couchbase-user [ 28/Oct/13 ]
I have added different servers names to hosts files in all servers and disabled the firewall on all of them and now it seems to work fine.
Thanks a lot for your help Michael.
Comment by Michael Nitschinger [ 28/Oct/13 ]
perfect, good to know it works now.
Comment by gaurang jhawar [ 28/Oct/13 ]
Any updates on my issue Michael Nitschinger. Thanks
Comment by Michael Nitschinger [ 29/Oct/13 ]
Since we fixed the suspected issue with "couchbase-user" in kind of the same way,
ca you first verify that

- All ports are reachable from all the boxes (11210, 8091, 8092)
- DNS settings are configured properly, this is especially important in EC2

If you look through the thread here, some of the people reporting this thing fixed once they made sure their environmental settings were okay. Can you double check?
Comment by gaurang jhawar [ 31/Oct/13 ]
Couchbase-user how did u configure hostname . can u tell me a step by step approach .. dont give me links to couchbase documentation that didn`t help ..

Michael .. the ports are open I am able to ping .. it connects via java and everything but the timeout is quite frequent ..



C:\Users\gaurang\Downloads>tcping 54.219.141.78 8091

Probing 54.219.141.78:8091/tcp - Port is open - time=21.692ms
Probing 54.219.141.78:8091/tcp - Port is open - time=19.357ms
Probing 54.219.141.78:8091/tcp - Port is open - time=15.414ms
Probing 54.219.141.78:8091/tcp - Port is open - time=16.225ms

Ping statistics for 54.219.141.78:8091
     4 probes sent.
     4 successful, 0 failed.
Approximate trip times in milli-seconds:
     Minimum = 15.414ms, Maximum = 21.692ms, Average = 18.172ms

C:\Users\gaurang\Downloads>tcping 54.219.141.78 11210\

Probing 54.219.141.78:11210/tcp - Port is open - time=18.236ms
Probing 54.219.141.78:11210/tcp - Port is open - time=23.513ms
Probing 54.219.141.78:11210/tcp - Port is open - time=1218.881ms
Probing 54.219.141.78:11210/tcp - Port is open - time=14.712ms

Ping statistics for 54.219.141.78:11210
     4 probes sent.
     4 successful, 0 failed.
Approximate trip times in milli-seconds:
     Minimum = 14.712ms, Maximum = 1218.881ms, Average = 318.836ms

Comment by CathodTech [ 13/Dec/13 ]
We have been facing the same issue for couples of months - with Couchbase on RHEL, and we have just found out the solution that works for us.

We have 2 Tomcat nodes running; 1 with Java 1.7 and another with 1.6. Couchbase was deployed on another 4 servers in the same network. We simply defined all Couchbase nodes in /etc/hosts file in both Tomcat nodes as you normally would (with x.x.x.x hostname), then it magically works for us - without having to restart Tomcat, nor Couchbase nodes.
Comment by Michael Nitschinger [ 13/Dec/13 ]
Thanks for pointing this out, I'll see if I can get it into the official docs to aid other people. To me this really looks like java-related and network-related. Interestingly only a few people are hitting it.
Comment by CathodTech [ 14/Dec/13 ]
For the issue I faced, it looks more like Java API related. Perhaps, something to do with DNS (e.g., something like gethostbyname() / gethostbyaddr()), and how the API handles the case when IP has no hostname. Note that we only use IP address to add nodes into Couchbase cluster and Java on other servers also connect to Couchbase with IP address, not hostname.

From the trace, we found that java code has no problem connecting (via connection pool) with Couchbase nodes that has IP/hostname pair in /etc/hosts. But for Couchbase nodes that were not defined in /etc/hosts, it attempted to connect, then the connection dropped, and returned timeout issue.

My team also found that later when he restarted java to re-deploy, Couchbase connection attempts at start-up were much faster (on both Java 1.6 and 1.7).

For this issue, we didn't face it when deploying Java and Couchbase on the same server as, perhaps, it is common to have "127.0.0.1 localhost" defined in /etc/hosts.

We hope that the solution we found would help you and others as well. Not sure if others are facing the same scenario, though.
Comment by David Borsos [ 18/Dec/13 ]
I've ran into a very similar/the same situation with the latest couchbase (2.2.0) server and java client (1.2.3), I thought I share my experiences.

The problem I faced was when trying to call CouchbaseClient.getView(). This apparently goes to one node in the couchbase cluster and queries some view metadata over HTTP (roughly). However I got "ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out" and that pretty much killed my client there. Seeing the solutions here with the hosts file workaround, I tried that and it indeed worked, but it's not really a suitable solution for us, so I decided to see into this a little bit more. Here is what I found.

(Disclaimer: this worked for me, and although I found other ways around this they may not work for you, and I did these attempting to solve this particular issue and didn't really care about anything else these changes may break, so be careful!)

(0) Baseline, the environment specific thing that actually causes the issue is that call java api call InetAddress.getHostFromNameService(InetAddress addr, boolean check) returns very slowly if there's no hostname to be found for a given IP. I have no idea why is that in my environment, I suppose could be because of a bunch of different reasons. Slow here means that it takes 4-5 secs for me for this call to return with an unknown hostname

This comes into the picture when the couchbase client tries to get a view. It uses Apache Http client to do this, and the following things happen:
(1) When the request for the view gets created, eventually we get to BaseIOReactor.execute which instantiates SessionHandle, and registers the time when this happened (this will cause the timeout)
(2) Somewhere in the request processing, we actually DO hit getHostFromNameService. More specifically this happens because a RequestTargetHost interceptor runs and this injects the "Host" header into the HTTP request.
(3) some time after (2) we hit a timeout check which compares the timestamp recorded in (1) and the current time, obviously resulting in a timeout and the error in the client if (2) takes a long time as indicated

The ideal solution would obviously be to fix whatever is wrong with my DNS configs, but I'm not (yet) sure what's wrong (running Ubuntu 13.10 btw). And also make sure that our live environments won't have the same issue...

One workaround I found that might be generally usable is to override the JVM's DNS resolver, by starting the client with: -Dsun.net.spi.nameservice.provider.1=dns,sun This worked for me, but I don't know what side-effects it may have (generally this may not be a great idea). Additionally this resulted in a significant increase of startup speed as the couchbase client seems to do a lot of ip->hostname lookups when it's starting (are these really necessary?)

Also I'd like to ask from the Couchbase folks if it's really necessary to do this reverse DNS lookup for every HTTP request. I cloned your client from git, and removed the RequestTargetHost interceptor from these calls; that also solved the timeout problem, and all seems to work fine. I didn't do exhaustive testing on it, and this, too, can be environment specific - running my couchbase cluster in local VMs at the moment.

Thanks.
Comment by Michael Nitschinger [ 19/Dec/13 ]
Hi David,

thanks for the heads-up, that gives me a better idea of whats going on. I basically revamped the whole view connection management for the next 1.3 release which hopefully should address this to. Would you want to try it out based on a preliminary build?

The thing is that you can't remove the RequestTargetHost, but it only does the DNS lookup if HTTP.TARGET_HOST is not set, but we can infer that from our configuration. So the new code basically sets it before the request is put in the queue, once we've determined which node to ask, then it should be skipped.

I already have that locally, so if you want to try it out we can quickly see if it fixes the issue.
Comment by Michael Nitschinger [ 19/Dec/13 ]
The issue will be fixed with the overall changeset of jcbc-388, which will appear in - as it stands now - 1.3, slated for jan 2014.
Comment by David Borsos [ 19/Dec/13 ]
Michael,

That sounds great. I'd be happy to help you verify your new code; send it over or let me know how can I get it, I'll switch to it and see if it solves the problem. If we can get the new version in early January (I saw it's scheduled for the 7th) that's cool too, we won't be doing much during the holidays anyway :)

Thanks
Comment by Sergii [ 19/Dec/13 ]
Hi Michael,

i have run into the same issue with couchbase 2.2 on mac - Java 1.7

Can i have the changed jar to try the fix?

Thanks
Comment by Michael Nitschinger [ 31/Dec/13 ]
A fix for this has been merged into master and will be available in the next (1.3.0) release.
Comment by Michael Nitschinger [ 31/Dec/13 ]
The new code minimizes dns lookups to workaround the issue, but the environment still needs to be fixed properly :)
Comment by Oded Hassidi [ 21/Jul/14 ]
Was this fixed, since we are having issues with timeot on Java SDK v1.4.x, and we are using JDK 7?
This is not happen all the time, if I run a thread that runs many request it occur every 2-4 min
Comment by Michael Nitschinger [ 21/Jul/14 ]
This issue happened during startup, are you experiencing issues during startup? Where do you get the timeouts?
Comment by Oded Hassidi [ 21/Jul/14 ]
Thanks for the quick response.
The timeout are not at startup it happens during runtime. every 2-3 minutes I get the followed:

SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:75)
        at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:778)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:93)
at com.couchbase.client.internal.ViewFuture.get(ViewFuture.java:65)
at com.couchbase.client.internal.ViewFuture.get(ViewFuture.java:49)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:72)
... 45 more




[JCBC-513] Implement bucket.close() Created: 18/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-500] CouchbaseClient doesn't timeout, hangs forever Created: 27/Jul/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Milan Simonovic Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
On a single node cluster:

1. open CouchbaseClient with opTimeout = 10sec (on a different machine)
2. write 50 docs (works fine)
3. make the server drop packets for 15sec (iptables -A INPUT -p tcp --dport 200:65535 -j DROP)
4. try to write another doc -> blocks forever

I've tried different timeout values: opTimeout =2.5sec, dropping packets for 5sec, but it also never times-out.

The code to reproduce this issue is here: https://github.com/mbsimonovic/jepsen-couchbase.
A more detailed explanation in my blog post: http://milansimonovic.com/2014/07/22/smashing-couchbase/

 Comments   
Comment by Milan Simonovic [ 28/Jul/14 ]
tested with Java SDK v1.4.3
Comment by Michael Nitschinger [ 14/Aug/14 ]
Hey Milan,

I cloned your project and adjusted it to my 3 node cluster. What I got was:

/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/bin/java -Didea.launcher.port=7540 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 13.app/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/michael/tmp/jepsen-couchbase/target/classes:/Users/michael/.m2/repository/com/couchbase/client/couchbase-client/1.4.3/couchbase-client-1.4.3.jar:/Users/michael/.m2/repository/io/netty/netty/3.5.5.Final/netty-3.5.5.Final.jar:/Users/michael/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/michael/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/michael/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar:/Users/michael/.m2/repository/net/spy/spymemcached/2.11.4/spymemcached-2.11.4.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore/4.3/httpcore-4.3.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore-nio/4.3/httpcore-nio-4.3.jar:/Users/michael/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/michael/.m2/repository/com/jcraft/jsch/0.1.51/jsch-0.1.51.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-api/1.5.10/slf4j-api-1.5.10.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-log4j12/1.5.10/slf4j-log4j12-1.5.10.jar:/Users/michael/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/michael/.m2/repository/javax/mail/mail/1.4/mail-1.4.jar:/Users/michael/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Applications/IntelliJ IDEA 13.app/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain com.milansimonovic.jepsen.CouchTimeoutTest
2014-08-14 10:19:38,106 INFO [com.milansimonovic.jepsen.RemoteConnection] - 192.168.56.101
2014-08-14 10:19:38,636 INFO [com.milansimonovic.jepsen.CouchbaseClientFactory] - CouchbaseClientFactory{192.168.56.101, default, }
2014-08-14 10:19:38,783 INFO [com.couchbase.client.vbucket.provider.BucketConfigurationProvider] - Could bootstrap through carrier publication.
2014-08-14 10:19:38,800 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,800 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.102:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,801 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.103:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,806 INFO [com.couchbase.client.CouchbaseClient] - CouchbaseConnectionFactory{bucket='default', nodes=[http://192.168.56.101:8091/pools], order=RANDOM, opTimeout=10000, opQueue=16384, opQueueBlockTime=10000, obsPollInt=10, obsPollMax=1000, obsTimeout=10000, viewConns=10, viewTimeout=75000, viewWorkers=1, configCheck=10, reconnectInt=1100, failureMode=Redistribute, hashAlgo=NATIVE_HASH, authWaitTime=2500}
2014-08-14 10:19:38,849 INFO [com.couchbase.client.CouchbaseClient] - viewmode property isn't defined. Setting viewmode to production mode
2014-08-14 10:19:38,849 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 10:19:39,869 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:39,870 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 10:19:39,919 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 1 in 43
2014-08-14 10:19:39,922 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 2 in 3
2014-08-14 10:19:39,923 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 3 in 1
2014-08-14 10:19:39,925 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 4 in 2
2014-08-14 10:19:39,927 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 5 in 2
2014-08-14 10:19:39,928 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 6 in 1
2014-08-14 10:19:39,930 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 7 in 2
2014-08-14 10:19:39,931 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 8 in 1
2014-08-14 10:19:39,933 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 9 in 2
2014-08-14 10:19:39,934 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 10 in 1
2014-08-14 10:19:39,936 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 11 in 2
2014-08-14 10:19:39,938 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 12 in 2
2014-08-14 10:19:39,939 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 13 in 1
2014-08-14 10:19:39,941 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 14 in 2
2014-08-14 10:19:39,943 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 15 in 2
2014-08-14 10:19:39,945 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 16 in 2
2014-08-14 10:19:39,946 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 17 in 1
2014-08-14 10:19:39,948 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 18 in 2
2014-08-14 10:19:39,949 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 19 in 1
2014-08-14 10:19:39,953 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 20 in 2
2014-08-14 10:19:39,955 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 21 in 2
2014-08-14 10:19:39,956 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 22 in 1
2014-08-14 10:19:39,958 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 23 in 2
2014-08-14 10:19:39,959 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 24 in 1
2014-08-14 10:19:39,961 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 25 in 2
2014-08-14 10:19:39,963 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 26 in 2
2014-08-14 10:19:39,964 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 27 in 1
2014-08-14 10:19:39,966 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 28 in 2
2014-08-14 10:19:39,968 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 29 in 2
2014-08-14 10:19:39,969 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 30 in 1
2014-08-14 10:19:39,971 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 31 in 2
2014-08-14 10:19:39,972 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 32 in 1
2014-08-14 10:19:39,974 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 33 in 2
2014-08-14 10:19:39,976 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 34 in 2
2014-08-14 10:19:39,977 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 35 in 1
2014-08-14 10:19:39,979 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 36 in 2
2014-08-14 10:19:39,981 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 37 in 2
2014-08-14 10:19:39,982 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 38 in 1
2014-08-14 10:19:39,984 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 39 in 2
2014-08-14 10:19:39,986 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 40 in 2
2014-08-14 10:19:39,988 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 41 in 1
2014-08-14 10:19:39,989 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 42 in 1
2014-08-14 10:19:39,991 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 43 in 1
2014-08-14 10:19:39,992 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 44 in 1
2014-08-14 10:19:39,993 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 45 in 1
2014-08-14 10:19:39,995 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 46 in 2
2014-08-14 10:19:39,996 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 47 in 1
2014-08-14 10:19:39,998 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 48 in 2
2014-08-14 10:19:39,999 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 49 in 1
2014-08-14 10:19:40,001 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 50 in 2
2014-08-14 10:19:40,001 INFO [com.milansimonovic.jepsen.Cluster] - creating 1 way partition
2014-08-14 10:19:41,003 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:41,003 INFO [com.milansimonovic.jepsen.Cluster] - creating partition DONE
2014-08-14 10:19:41,004 INFO [com.milansimonovic.jepsen.Cluster] - scheduled heal in 15 seconds
2014-08-14 10:19:56,005 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 10:19:57,007 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:57,008 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 10:19:57,008 INFO [com.milansimonovic.jepsen.Cluster] - closing connections

and the doc on the server looked like:

{
  "elements": [
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    11,
    12,
    13,
    14,
    15,
    16,
    17,
    18,
    19,
    20,
    21,
    22,
    23,
    24,
    25,
    26,
    27,
    28,
    29,
    30,
    31,
    32,
    33,
    34,
    35,
    36,
    37,
    38,
    39,
    40,
    41,
    42,
    43,
    44,
    45,
    46,
    47,
    48,
    49,
    50
  ]
}

I wonder what I did different? Does it happen for you only in the single-node case? Or does running the test just not cover the bug?
Comment by Michael Nitschinger [ 14/Aug/14 ]
Ah now I get it, there should be much more appended than just 50.
Comment by Michael Nitschinger [ 14/Aug/14 ]
Ah I fixed your code. Let me do a PR for your github project.

You need to move the gets into the try/catch block as well since it can also throw a timeout (and will if there is a partition):

                CASValue<Object> casValue = client.gets(key);
                String updatedValue = appendElement(casValue.getValue().toString(), nextVal);

When I move it inside I get:

/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:61775,suspend=y,server=n -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/michael/tmp/jepsen-couchbase/target/classes:/Users/michael/.m2/repository/com/couchbase/client/couchbase-client/1.4.3/couchbase-client-1.4.3.jar:/Users/michael/.m2/repository/io/netty/netty/3.5.5.Final/netty-3.5.5.Final.jar:/Users/michael/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/michael/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/michael/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar:/Users/michael/.m2/repository/net/spy/spymemcached/2.11.4/spymemcached-2.11.4.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore/4.3/httpcore-4.3.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore-nio/4.3/httpcore-nio-4.3.jar:/Users/michael/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/michael/.m2/repository/com/jcraft/jsch/0.1.51/jsch-0.1.51.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-api/1.5.10/slf4j-api-1.5.10.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-log4j12/1.5.10/slf4j-log4j12-1.5.10.jar:/Users/michael/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/michael/.m2/repository/javax/mail/mail/1.4/mail-1.4.jar:/Users/michael/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Applications/IntelliJ IDEA 13.app/lib/idea_rt.jar" com.milansimonovic.jepsen.CouchTimeoutTest
Connected to the target VM, address: '127.0.0.1:61775', transport: 'socket'
2014-08-14 11:02:47,408 INFO [com.milansimonovic.jepsen.RemoteConnection] - 192.168.56.101
2014-08-14 11:02:47,771 INFO [com.milansimonovic.jepsen.CouchbaseClientFactory] - CouchbaseClientFactory{192.168.56.101, default, }
2014-08-14 11:02:47,908 INFO [com.couchbase.client.vbucket.provider.BucketConfigurationProvider] - Could bootstrap through carrier publication.
2014-08-14 11:02:47,920 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 11:02:47,923 INFO [com.couchbase.client.CouchbaseClient] - CouchbaseConnectionFactory{bucket='default', nodes=[http://192.168.56.101:8091/pools], order=RANDOM, opTimeout=10000, opQueue=16384, opQueueBlockTime=10000, obsPollInt=10, obsPollMax=1000, obsTimeout=10000, viewConns=10, viewTimeout=75000, viewWorkers=1, configCheck=10, reconnectInt=1100, failureMode=Redistribute, hashAlgo=NATIVE_HASH, authWaitTime=2500}
2014-08-14 11:02:47,956 INFO [com.couchbase.client.CouchbaseClient] - viewmode property isn't defined. Setting viewmode to production mode
2014-08-14 11:02:47,956 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 11:02:48,970 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:02:48,970 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
counting down 943596453
counting down 99276090
2014-08-14 11:02:49,042 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 1 in 61
counting down 1076537872
2014-08-14 11:02:49,048 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 2 in 6
2014-08-14 11:02:49,054 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 3 in 6
counting down 1413217636
2014-08-14 11:02:49,060 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 4 in 6
counting down 491904845
2014-08-14 11:02:49,066 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 5 in 6
counting down 1312922328
2014-08-14 11:02:49,072 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 6 in 6
counting down 1529791274
2014-08-14 11:02:49,078 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 7 in 6
counting down 1349895670
2014-08-14 11:02:49,083 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 8 in 5
counting down 1129169666
2014-08-14 11:02:49,089 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 9 in 6
counting down 26740457
counting down 746758755
2014-08-14 11:02:49,094 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 10 in 5
counting down 1386645616
2014-08-14 11:02:49,100 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 11 in 5
2014-08-14 11:02:49,106 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 12 in 6
counting down 2145256693
2014-08-14 11:02:49,111 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 13 in 5
counting down 1051781801
2014-08-14 11:02:49,117 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 14 in 6
counting down 119068326
2014-08-14 11:02:49,123 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 15 in 6
counting down 1637559101
2014-08-14 11:02:49,128 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 16 in 5
counting down 1861050967
2014-08-14 11:02:49,134 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 17 in 6
counting down 1462839425
2014-08-14 11:02:49,140 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 18 in 6
counting down 383609127
2014-08-14 11:02:49,147 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 19 in 6
counting down 774334978
2014-08-14 11:02:49,156 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 20 in 8
counting down 735922763
2014-08-14 11:02:49,162 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 21 in 6
counting down 1970674321
2014-08-14 11:02:49,170 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 22 in 7
counting down 136298825
2014-08-14 11:02:49,177 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 23 in 7
counting down 494783794
2014-08-14 11:02:49,185 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 24 in 8
counting down 1546867652
2014-08-14 11:02:49,192 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 25 in 7
counting down 433354817
2014-08-14 11:02:49,199 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 26 in 7
counting down 1865938875
2014-08-14 11:02:49,207 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 27 in 8
counting down 483439410
2014-08-14 11:02:49,214 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 28 in 7
counting down 1513848351
2014-08-14 11:02:49,221 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 29 in 7
counting down 1612120937
2014-08-14 11:02:49,228 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 30 in 7
counting down 810400511
2014-08-14 11:02:49,235 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 31 in 7
counting down 689821415
2014-08-14 11:02:49,242 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 32 in 7
counting down 685219421
2014-08-14 11:02:49,248 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 33 in 6
counting down 1887689829
counting down 759433928
2014-08-14 11:02:49,254 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 34 in 6
2014-08-14 11:02:49,259 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 35 in 5
counting down 1781091177
2014-08-14 11:02:49,266 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 36 in 7
counting down 2123891975
2014-08-14 11:02:49,271 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 37 in 5
counting down 278306373
2014-08-14 11:02:49,278 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 38 in 6
counting down 495568085
2014-08-14 11:02:49,284 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 39 in 6
counting down 1029420558
2014-08-14 11:02:49,290 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 40 in 5
counting down 55913631
2014-08-14 11:02:49,296 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 41 in 6
counting down 664141318
2014-08-14 11:02:49,302 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 42 in 6
counting down 1953048909
2014-08-14 11:02:49,307 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 43 in 5
counting down 350362120
counting down 1048178143
2014-08-14 11:02:49,313 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 44 in 6
2014-08-14 11:02:49,318 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 45 in 5
counting down 907297240
2014-08-14 11:02:49,323 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 46 in 5
counting down 193602838
2014-08-14 11:02:49,329 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 47 in 6
counting down 255614005
2014-08-14 11:02:49,335 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 48 in 6
counting down 2128240408
2014-08-14 11:02:49,340 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 49 in 5
counting down 1783946138
2014-08-14 11:02:49,346 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 50 in 6
2014-08-14 11:02:49,347 INFO [com.milansimonovic.jepsen.Cluster] - creating 1 way partition
2014-08-14 11:02:50,348 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:02:50,348 INFO [com.milansimonovic.jepsen.Cluster] - creating partition DONE
2014-08-14 11:02:50,349 INFO [com.milansimonovic.jepsen.Cluster] - scheduled heal in 15 seconds
counting down 1330247343
2014-08-14 11:03:03,213 ERROR [com.milansimonovic.jepsen.CouchTimeoutTest] - timed out adding 51
2014-08-14 11:03:03,214 ERROR [com.milansimonovic.jepsen.CouchTimeoutTest] - error adding 51
net.spy.memcached.OperationTimeoutException: Timeout waiting for value
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1147)
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1211)
at com.milansimonovic.jepsen.CouchTimeoutTest.addElementToArray(CouchTimeoutTest.java:65)
at com.milansimonovic.jepsen.CouchTimeoutTest.appendToDocument(CouchTimeoutTest.java:102)
at com.milansimonovic.jepsen.CouchTimeoutTest.run(CouchTimeoutTest.java:49)
at com.milansimonovic.jepsen.CouchTimeoutTest.main(CouchTimeoutTest.java:39)
Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: 192.168.56.101/192.168.56.101:11210
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:167)
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1137)
... 5 more
2014-08-14 11:03:05,349 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 11:03:06,351 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:03:06,351 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 11:03:06,351 INFO [com.milansimonovic.jepsen.Cluster] - closing connections
counting down 1330247343
counting down 808247635
counting down 1562173079
2014-08-14 11:03:08,230 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 52 in 5014
counting down 1897031859
2014-08-14 11:03:08,236 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 53 in 6
2014-08-14 11:03:08,241 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 54 in 5
counting down 1029104896
2014-08-14 11:03:08,247 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 55 in 6
counting down 1789373824
2014-08-14 11:03:08,252 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 56 in 5
counting down 1374957093
2014-08-14 11:03:08,259 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 57 in 7
counting down 859935576
2014-08-14 11:03:08,265 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 58 in 5
counting down 959778517
2014-08-14 11:03:08,270 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 59 in 5
counting down 1856597982
2014-08-14 11:03:08,276 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 60 in 6
counting down 509403432
2014-08-14 11:03:08,283 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 61 in 7
counting down 565734120
2014-08-14 11:03:08,289 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 62 in 6
counting down 812288940
2014-08-14 11:03:08,295 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 63 in 6
counting down 362337760
2014-08-14 11:03:08,301 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 64 in 6
counting down 1815360474
counting down 238680990
2014-08-14 11:03:08,307 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 65 in 6
2014-08-14 11:03:08,312 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 66 in 5
counting down 1793734174
2014-08-14 11:03:08,318 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 67 in 6
counting down 965663094
2014-08-14 11:03:08,323 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 68 in 5
counting down 1961465726
2014-08-14 11:03:08,329 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 69 in 6
counting down 894265206
counting down 237825287
2014-08-14 11:03:08,334 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 70 in 5
2014-08-14 11:03:08,340 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 71 in 6
counting down 1093062877
2014-08-14 11:03:08,346 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 72 in 6
counting down 2043477472
2014-08-14 11:03:08,352 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 73 in 6
counting down 660886392
2014-08-14 11:03:08,358 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 74 in 6
counting down 1321850438
2014-08-14 11:03:08,363 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 75 in 5
counting down 966674298
2014-08-14 11:03:08,368 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 76 in 5
counting down 665030228
counting down 584709707
2014-08-14 11:03:08,373 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 77 in 5
2014-08-14 11:03:08,379 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 78 in 6
counting down 581070768
2014-08-14 11:03:08,384 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 79 in 5
counting down 1766611209
2014-08-14 11:03:08,390 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 80 in 6
counting down 1378265946
2014-08-14 11:03:08,395 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 81 in 5
counting down 1067480315
2014-08-14 11:03:08,400 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 82 in 5
counting down 240010064
2014-08-14 11:03:08,406 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 83 in 6
counting down 96589697
counting down 1650118680
2014-08-14 11:03:08,411 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 84 in 5
2014-08-14 11:03:08,417 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 85 in 6
counting down 1583441668
2014-08-14 11:03:08,422 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 86 in 5
counting down 575269723
2014-08-14 11:03:08,427 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 87 in 5
counting down 692018508
2014-08-14 11:03:08,434 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 88 in 6
counting down 500269667
2014-08-14 11:03:08,440 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 89 in 6
counting down 1214091162
2014-08-14 11:03:08,446 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 90 in 6
counting down 631482105
2014-08-14 11:03:08,452 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 91 in 6
counting down 1982159420
2014-08-14 11:03:08,458 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 92 in 6
counting down 1744182273
counting down 1131642426
2014-08-14 11:03:08,463 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 93 in 5
2014-08-14 11:03:08,469 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 94 in 6
counting down 2069859523
2014-08-14 11:03:08,475 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 95 in 6
counting down 662166830
2014-08-14 11:03:08,481 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 96 in 6
counting down 1882643413
2014-08-14 11:03:08,487 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 97 in 6
counting down 678230343
2014-08-14 11:03:08,493 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 98 in 6
counting down 793919773
2014-08-14 11:03:08,498 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 99 in 5
counting down 911566708
2014-08-14 11:03:08,505 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 100 in 7
counting down 1887173144
2014-08-14 11:03:08,510 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 101 in 5
counting down 1058177937
2014-08-14 11:03:08,516 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 102 in 6
counting down 41252807
2014-08-14 11:03:08,523 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 103 in 6
counting down 528932410
2014-08-14 11:03:08,531 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 104 in 8
counting down 805461224
2014-08-14 11:03:08,538 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 105 in 7
counting down 463949682
2014-08-14 11:03:08,546 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 106 in 8
counting down 851822734
2014-08-14 11:03:08,554 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 107 in 8
counting down 454797282
2014-08-14 11:03:08,561 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 108 in 7
counting down 962354549
2014-08-14 11:03:08,568 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 109 in 7
counting down 413223411
2014-08-14 11:03:08,574 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 110 in 6
counting down 79455185
2014-08-14 11:03:08,580 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 111 in 6
counting down 1113518099
2014-08-14 11:03:08,586 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 112 in 6
counting down 431139544
2014-08-14 11:03:08,592 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 113 in 6
counting down 198084300
2014-08-14 11:03:08,599 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 114 in 7
counting down 1821101711
2014-08-14 11:03:08,605 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 115 in 6
counting down 1499489637
2014-08-14 11:03:08,611 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 116 in 6
counting down 1435037947
2014-08-14 11:03:08,618 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 117 in 7
counting down 705688672
2014-08-14 11:03:08,624 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 118 in 6
counting down 407021592
2014-08-14 11:03:08,630 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 119 in 6
counting down 1121168696
2014-08-14 11:03:08,637 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 120 in 7
counting down 1347326641
counting down 87539756
2014-08-14 11:03:08,642 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 121 in 5
2014-08-14 11:03:08,647 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 122 in 5
counting down 137935610
2014-08-14 11:03:08,653 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 123 in 6
counting down 1474133160
2014-08-14 11:03:08,659 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 124 in 6
counting down 1931651064
2014-08-14 11:03:08,665 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 125 in 6
counting down 1211051792
2014-08-14 11:03:08,688 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 126 in 23
counting down 93172613
2014-08-14 11:03:08,693 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 127 in 5
counting down 965322656
2014-08-14 11:03:08,699 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 128 in 6
counting down 1601358350
2014-08-14 11:03:08,705 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 129 in 6
counting down 1855207775
2014-08-14 11:03:08,710 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 130 in 5
counting down 107851411
2014-08-14 11:03:08,716 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 131 in 6
counting down 1915600217
2014-08-14 11:03:08,721 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 132 in 5
counting down 1002847144
2014-08-14 11:03:08,727 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 133 in 6
counting down 1960289102
2014-08-14 11:03:08,732 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 134 in 5
counting down 530940529
2014-08-14 11:03:08,738 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 135 in 6
counting down 16593833
2014-08-14 11:03:08,744 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 136 in 6
counting down 183255909
2014-08-14 11:03:08,749 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 137 in 5
counting down 1139297445
counting down 1886255267
2014-08-14 11:03:08,755 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 138 in 5
2014-08-14 11:03:08,760 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 139 in 5
counting down 426044796
2014-08-14 11:03:08,765 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 140 in 5
counting down 1903915614
2014-08-14 11:03:08,770 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 141 in 5
counting down 1895269808
2014-08-14 11:03:08,776 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 142 in 6
counting down 1866519533
counting down 1366412404
2014-08-14 11:03:08,781 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 143 in 5
2014-08-14 11:03:08,786 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 144 in 5
counting down 2075311494
2014-08-14 11:03:08,792 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 145 in 6
counting down 925138245
2014-08-14 11:03:08,797 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 146 in 5
counting down 1578242039
2014-08-14 11:03:08,802 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 147 in 5
counting down 23034413
2014-08-14 11:03:08,808 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 148 in 6
counting down 1869588749
counting down 2067087344
2014-08-14 11:03:08,814 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 149 in 6
2014-08-14 11:03:08,820 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 150 in 6
counting down 899087062
2014-08-14 11:03:08,826 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 151 in 6
counting down 1340235867
2014-08-14 11:03:08,832 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 152 in 6
counting down 1574412139
2014-08-14 11:03:08,838 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 153 in 6
counting down 256897438
2014-08-14 11:03:08,844 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 154 in 6
counting down 227404523
2014-08-14 11:03:08,849 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 155 in 5
counting down 1370600377
2014-08-14 11:03:08,855 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 156 in 5
counting down 821838371
2014-08-14 11:03:08,861 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 157 in 6
counting down 1087346339
counting down 578287331
2014-08-14 11:03:08,866 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 158 in 5
2014-08-14 11:03:08,872 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 159 in 6
counting down 2114430319
2014-08-14 11:03:08,877 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 160 in 5
counting down 750263766
2014-08-14 11:03:08,883 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 161 in 6
counting down 285142124
2014-08-14 11:03:08,889 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 162 in 6
counting down 228835454
2014-08-14 11:03:08,895 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 163 in 6
counting down 1322000504
2014-08-14 11:03:08,900 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 164 in 5
counting down 578208474
2014-08-14 11:03:08,906 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 165 in 6
counting down 1158469492
counting down 1446993819
2014-08-14 11:03:08,911 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 166 in 5
2014-08-14 11:03:08,917 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 167 in 6
counting down 2036300484
2014-08-14 11:03:08,923 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 168 in 5
counting down 1839929127
2014-08-14 11:03:08,928 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 169 in 5
counting down 1321196177
2014-08-14 11:03:08,934 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 170 in 6
counting down 1657833306
2014-08-14 11:03:08,941 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 171 in 7
counting down 1616599198
2014-08-14 11:03:08,948 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 172 in 7
counting down 1434216861
2014-08-14 11:03:08,955 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 173 in 7
counting down 1697078001
2014-08-14 11:03:08,962 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 174 in 7
counting down 53209944
2014-08-14 11:03:08,969 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 175 in 7
counting down 957271051
2014-08-14 11:03:08,983 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 176 in 14
counting down 260602422
2014-08-14 11:03:08,991 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 177 in 8
counting down 1344752100
2014-08-14 11:03:08,996 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 178 in 5
counting down 210373011
2014-08-14 11:03:09,003 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 179 in 7
counting down 1135342237
counting down 255748372
2014-08-14 11:03:09,008 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 180 in 5
2014-08-14 11:03:09,013 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 181 in 5
counting down 1706850686
2014-08-14 11:03:09,018 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 182 in 5
counting down 958800587
2014-08-14 11:03:09,023 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 183 in 5
counting down 621084776
counting down 1494203157
2014-08-14 11:03:09,028 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 184 in 5
2014-08-14 11:03:09,034 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 185 in 6
counting down 1610501827
2014-08-14 11:03:09,039 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 186 in 5
counting down 1472601997
2014-08-14 11:03:09,046 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 187 in 6
counting down 1523401565
2014-08-14 11:03:09,052 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 188 in 6
counting down 2118032386
2014-08-14 11:03:09,058 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 189 in 6
counting down 1402690896
2014-08-14 11:03:09,063 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 190 in 5
counting down 877005529
counting down 69300391
2014-08-14 11:03:09,068 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 191 in 5
counting down 250950317
2014-08-14 11:03:09,074 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 192 in 6
2014-08-14 11:03:09,080 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 193 in 6
counting down 126717950
2014-08-14 11:03:09,086 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 194 in 6
counting down 60378388
2014-08-14 11:03:09,092 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 195 in 6
counting down 213037448
2014-08-14 11:03:09,098 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 196 in 6
counting down 737512169
2014-08-14 11:03:09,103 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 197 in 5
counting down 1089208854
2014-08-14 11:03:09,109 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 198 in 6
counting down 156501746
2014-08-14 11:03:09,115 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 199 in 6
counting down 284416161
2014-08-14 11:03:09,121 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 200 in 6
2014-08-14 11:03:09,123 INFO [com.couchbase.client.CouchbaseConnection] - Shut down Couchbase client
2014-08-14 11:03:09,125 INFO [com.couchbase.client.ViewConnection] - I/O reactor terminated
2014-08-14 11:03:09,125 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - DONE!
Disconnected from the target VM, address: '127.0.0.1:61775', transport: 'socket'

Process finished with exit code 0

Comment by Michael Nitschinger [ 14/Aug/14 ]
https://github.com/mbsimonovic/jepsen-couchbase/pull/2
Comment by Matt Ingenthron [ 18/Aug/14 ]
Milan: would be great if you could update your blog with a paragraph or so at the top clarifying that it did work so people don't have to read into the comments to find that it was a problem with the test.
Comment by Milan Simonovic [ 19/Aug/14 ]
Sure, as soon as I get back from holidays (next we latest). Ps the tracker is major pain in the a** to use on a smartphone..
Comment by Milan Simonovic [ 26/Aug/14 ]
updated, feel free to close the issue
Comment by Michael Nitschinger [ 26/Aug/14 ]
Not an issue.




[JCBC-472] Implement support for snappy compression where supported Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-471] Implement HELLO command Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-473] Flush support Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-464] Client never recovers when all nodes are shut down Created: 03/Jun/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File thread-dump.txt    
Issue Links:
Relates to

 Description   
When running this demo, if I disconnect the cluster entirely and restart it, the client reconnects but never recovers:
https://github.com/couchbaselabs/CBLiveDemo

 Comments   
Comment by Michael Nitschinger [ 14/Jul/14 ]
http://review.couchbase.org/#/c/39346/




[JCBC-545] Couchbase Connection leaks to a single node in the cluster Created: 05/Sep/14  Updated: 15/Sep/14  Due: 14/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.2
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Critical
Reporter: Justin Michaels Assignee: Michael Nitschinger
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified