[JCBC-566] CouchbaseConnectionFactory cannot schedule configuration update Created: 23/Sep/14  Updated: 23/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.0-dp1, 1.4.0-dp2, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Major
Reporter: Sergey Avseyev Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Java Source File Hello.java    

 Description   
client is working with a cluster
network from client to cluster is disrupted (sudo service couchbase-server stop)
network from client to cluster is later restored (sudo service couchbase-server start)

expected behavior:
  client recovers and traffic proceeds a short time after the network is back to working
observed behavior:
  client does not recover

 Comments   
Comment by Sergey Avseyev [ 23/Sep/14 ]
http://review.couchbase.org/41585
Comment by Sergey Avseyev [ 23/Sep/14 ]
I've written that it affects versions since 1.4.0-dp1 because http://review.couchbase.org/32589 was relesed in 1.4.0-dp1 first time. I haven't verified all these releases, just 1.4.4.




[JCBC-550] Java 2.0 SDK Backpressure Exception Created: 11/Sep/14  Updated: 23/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: Wei-Li Liu Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This is a known issue and a fix is coming soon.
I work around the issue by waiting a bit longer between ops. It does reduces the number of backpressue exceptions.

Will run the test again after the fix is pushed.


 Comments   
Comment by Michael Nitschinger [ 19/Sep/14 ]
Moving this to 2.0.1 and non blocker because it seems to be a bug in the user code.
Comment by Matt Ingenthron [ 23/Sep/14 ]
Sergey: While Michael is out, can you look at this with Wei-Li?




[JCBC-546] Expected Behavior of FailureMode.Cancel Created: 08/Sep/14  Updated: 22/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Jeff Dillon Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate

 Description   
Customer has questions on the behavior of FailureMode.Cancel:

1) Is TimeoutExceptionThreshold irrelevant when using FailureMode.Redistribute ? The node is not deemed down even when the number of consecutive failures exceed the threshold.
2) In what scenarios is it advisable to use Redistribute ?
3) Whats the .NET equivalent.

 Comments   
Comment by Jeff Dillon [ 22/Sep/14 ]
Just checking, any updates on this? thx




[JCBC-502] Sometimes couchbase client stuck Created: 29/Jul/14  Updated: 21/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Infrastructure
Affects Version/s: 2.0-dp2
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Oded Hassidi Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Mac OSX as client


 Description   
Sometimes I run the client like in the examples in the community blog and the client stuck and Doesn't return. To validate that I run the client several times one after the the other and sometimes it get stuck and some don't

 Comments   
Comment by Michael Nitschinger [ 11/Aug/14 ]
Can you provide logging with level set to FINEST? that would be of great help, thanks.
Comment by Oded Hassidi [ 12/Aug/14 ]
I am running my test in a main, and I have debug level entries. Is that enough?
Comment by Michael Nitschinger [ 12/Aug/14 ]
Yes send over whatever you have, and the code would also be good.
Comment by Oded Hassidi [ 12/Aug/14 ]
Code:


public class Main {
public static void main(String[] args) throws IOException {
      
        System.setProperty("com.couchbase.client.queryEnabled", "true");

        CouchbaseCluster cc = new CouchbaseCluster("obbe-centos3.tlv.lpnet.com", "obbe-centos1.tlv.lpnet.com", "obbe-centos2.tlv.lpnet.com");

        final Bucket[] bucket = new CouchbaseBucket[1];
        System.out.println("******** Opening the Bucket");
        cc.openBucket("visitors-odedh").subscribe(b -> {
            System.out.println("################### Bucket is opened: ");
                System.out.println(b.toString());
                bucket[0] = b;
        });


        System.out.println("enter something: ");
        System.in.read();
    }
}



log:

/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/bin/java -Didea.launcher.port=7534 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 14 EAP.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/odedh/Projects/visitorfeed/branches/n1ql/dal/target/classes:/Users/odedh/.m2/repository/ch/qos/logback/logback-classic/1.1.2/logback-classic-1.1.2.jar:/Users/odedh/.m2/repository/ch/qos/logback/logback-core/1.1.2/logback-core-1.1.2.jar:/Users/odedh/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/Users/odedh/Projects/visitorfeed/branches/n1ql/dao/target/classes:/Users/odedh/.m2/repository/com/liveperson/framework/rest-api-data-objects/2.0.2.11/rest-api-data-objects-2.0.2.11.jar:/Users/odedh/.m2/repository/com/liveperson/RestApiCommon/1.0.2-6/RestApiCommon-1.0.2-6.jar:/Users/odedh/.m2/repository/com/sun/jersey/jersey-core/1.9.1/jersey-core-1.9.1.jar:/Users/odedh/.m2/repository/com/sun/jersey/jersey-client/1.9.1/jersey-client-1.9.1.jar:/Users/odedh/.m2/repository/com/sun/jersey/jersey-server/1.9.1/jersey-server-1.9.1.jar:/Users/odedh/.m2/repository/asm/asm/3.1/asm-3.1.jar:/Users/odedh/.m2/repository/com/sun/jersey/jersey-json/1.9.1/jersey-json-1.9.1.jar:/Users/odedh/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/odedh/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/odedh/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/odedh/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/odedh/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/Users/odedh/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/odedh/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.3/jackson-core-asl-1.8.3.jar:/Users/odedh/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.3/jackson-mapper-asl-1.8.3.jar:/Users/odedh/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.3/jackson-jaxrs-1.8.3.jar:/Users/odedh/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.3/jackson-xc-1.8.3.jar:/Users/odedh/.m2/repository/org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.jar:/Users/odedh/.m2/repository/org/springframework/spring-context/4.0.6.RELEASE/spring-context-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/org/springframework/spring-aop/4.0.6.RELEASE/spring-aop-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/odedh/.m2/repository/org/springframework/spring-beans/4.0.6.RELEASE/spring-beans-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/org/springframework/spring-core/4.0.6.RELEASE/spring-core-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/org/springframework/spring-expression/4.0.6.RELEASE/spring-expression-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/com/couchbase/client/couchbase-client/2.0.0-dp2/couchbase-client-2.0.0-dp2.jar:/Users/odedh/.m2/repository/com/couchbase/client/core-io/0.2/core-io-0.2.jar:/Users/odedh/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.4.0/jackson-databind-2.4.0.jar:/Users/odedh/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.4.0/jackson-annotations-2.4.0.jar:/Users/odedh/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.4.0/jackson-core-2.4.0.jar:/Users/odedh/.m2/repository/com/lmax/disruptor/3.2.1/disruptor-3.2.1.jar:/Users/odedh/.m2/repository/com/netflix/rxjava/rxjava-core/0.19.2/rxjava-core-0.19.2.jar:/Users/odedh/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/Users/odedh/.m2/repository/io/netty/netty-all/4.0.21.Final/netty-all-4.0.21.Final.jar:/Users/odedh/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/odedh/.m2/repository/org/springframework/spring-test/4.0.6.RELEASE/spring-test-4.0.6.RELEASE.jar:/Users/odedh/.m2/repository/com/liveperson/SnmpMonitoring/4.0.0.1/SnmpMonitoring-4.0.0.1.jar:/Users/odedh/.m2/repository/com/liveperson/SnmpMonitoring-api/4.0.0.1/SnmpMonitoring-api-4.0.0.1.jar:/Users/odedh/.m2/repository/org/apache/xmlbeans/xmlbeans/2.4.0/xmlbeans-2.4.0.jar:/Users/odedh/.m2/repository/com/liveperson/ServiceStatus/3.0.0.2/ServiceStatus-3.0.0.2.jar:/Users/odedh/.m2/repository/com/liveperson/ServiceStatus-API/3.0.0.2/ServiceStatus-API-3.0.0.2.jar:/Applications/IntelliJ IDEA 14 EAP.app/Contents/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain com.liveperson.visitorfeed.dal.couchbase.CouchbaseVisitorDao
15:55:05.598 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
15:55:05.604 [main] DEBUG i.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 16
15:55:05.616 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
15:55:05.616 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
15:55:05.617 [main] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
15:55:05.617 [main] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
15:55:05.624 [main] DEBUG i.n.util.internal.PlatformDependent - UID: 733553586
15:55:05.624 [main] DEBUG i.n.util.internal.PlatformDependent - Java version: 8
15:55:05.625 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noUnsafe: false
15:55:05.625 [main] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
15:55:05.625 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noJavassist: false
15:55:05.625 [main] DEBUG i.n.util.internal.PlatformDependent - Javassist: unavailable
15:55:05.626 [main] DEBUG i.n.util.internal.PlatformDependent - You don't have Javassist in your class path or you don't have enough permission to load dynamically generated classes. Please check the configuration for better performance.
15:55:05.626 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /var/folders/5f/wk7ckrbx1f30521n8rtxndzwnvk8xk/T (java.io.tmpdir)
15:55:05.626 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
15:55:05.626 [main] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
15:55:05.640 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
15:55:05.640 [main] DEBUG io.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
15:55:05.712 [main] DEBUG c.c.c.c.config.ConfigurationProvider - Setting seed hosts to [obbe-centos1.tlv.lpnet.com/192.168.15.120, obbe-centos3.tlv.lpnet.com/192.168.15.190, obbe-centos2.tlv.lpnet.com/192.168.15.2]
******** Opening the Bucket
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 8
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 8
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize: 512
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
15:55:05.754 [RxComputationThreadPool-1] DEBUG i.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192
enter something:
15:55:05.780 [RxComputationThreadPool-1] DEBUG i.n.util.internal.ThreadLocalRandom - -Dio.netty.initialSeedUniquifier: 0x9729d41313197dd7
15:55:05.785 [RxComputationThreadPool-1] DEBUG i.n.channel.ChannelOutboundBuffer - -Dio.netty.threadLocalDirectBufferSize: 65536
15:55:05.786 [RxComputationThreadPool-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacity.default: 262144
15:55:05.795 [RxComputationThreadPool-1] DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
15:55:05.880 [nioEventLoopGroup-2-2] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetectionLevel: simple
15:55:05.981 [nioEventLoopGroup-2-2] DEBUG io.netty.util.internal.Cleaner0 - java.nio.ByteBuffer.cleaner(): available
15:55:05.983 [nioEventLoopGroup-2-2] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos1.tlv.lpnet.com/192.168.15.120:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-1] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos1.tlv.lpnet.com/192.168.15.120:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-8] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos2.tlv.lpnet.com/192.168.15.2:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-7] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos2.tlv.lpnet.com/192.168.15.2:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-5] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos3.tlv.lpnet.com/192.168.15.190:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-4] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos3.tlv.lpnet.com/192.168.15.190:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-6] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos3.tlv.lpnet.com/192.168.15.190:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-3] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos1.tlv.lpnet.com/192.168.15.120:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.983 [nioEventLoopGroup-2-8] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:11210
15:55:05.983 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:11210
15:55:05.983 [nioEventLoopGroup-2-1] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:11210
15:55:05.983 [nioEventLoopGroup-2-7] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:11210
15:55:05.983 [nioEventLoopGroup-2-5] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:11210
15:55:05.983 [nioEventLoopGroup-2-4] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:11210
15:55:05.984 [nioEventLoopGroup-2-6] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:11210
15:55:05.984 [nioEventLoopGroup-2-3] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:11210
15:55:05.984 [nioEventLoopGroup-2-2] INFO com.couchbase.client.core.node.Node - Disconnected from Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:05.984 [nioEventLoopGroup-2-8] INFO com.couchbase.client.core.node.Node - Disconnected from Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:05.984 [nioEventLoopGroup-2-5] INFO com.couchbase.client.core.node.Node - Disconnected from Node obbe-centos3.tlv.lpnet.com/192.168.15.190
15:55:05.984 [nioEventLoopGroup-2-2] DEBUG com.couchbase.client.core.node.Node - Disconnected (CONNECTING) from Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:05.984 [nioEventLoopGroup-2-8] DEBUG com.couchbase.client.core.node.Node - Disconnected (CONNECTING) from Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:05.985 [nioEventLoopGroup-2-5] DEBUG com.couchbase.client.core.node.Node - Disconnected (CONNECTING) from Node obbe-centos3.tlv.lpnet.com/192.168.15.190
15:55:05.985 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.service.Service - Connected (DEGRADED) to BinaryService obbe-centos1.tlv.lpnet.com
15:55:05.985 [nioEventLoopGroup-2-1] DEBUG c.c.c.c.e.binary.BinaryHelloClient - obbe-centos2.tlv.lpnet.com/192.168.15.2:11210 Hello not successful (Response Status: 129), falling back to no datatype.
15:55:05.985 [nioEventLoopGroup-2-5] DEBUG c.c.client.core.service.Service - Connected (DEGRADED) to BinaryService obbe-centos3.tlv.lpnet.com
15:55:05.985 [nioEventLoopGroup-2-2] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:05.985 [nioEventLoopGroup-2-1] DEBUG c.c.client.core.endpoint.Endpoint - Connected to BinaryEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:11210
15:55:05.985 [nioEventLoopGroup-2-5] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos3.tlv.lpnet.com/192.168.15.190
15:55:05.985 [nioEventLoopGroup-2-2] DEBUG com.couchbase.client.core.node.Node - Connected (DISCONNECTED) to Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:05.985 [nioEventLoopGroup-2-1] DEBUG c.c.client.core.service.Service - Connected (DEGRADED) to BinaryService obbe-centos2.tlv.lpnet.com
15:55:05.985 [nioEventLoopGroup-2-5] DEBUG com.couchbase.client.core.node.Node - Connected (DISCONNECTED) to Node obbe-centos3.tlv.lpnet.com/192.168.15.190
15:55:05.985 [nioEventLoopGroup-2-1] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:05.985 [nioEventLoopGroup-2-1] DEBUG com.couchbase.client.core.node.Node - Connected (DISCONNECTED) to Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:05.994 [pool-1-thread-1] DEBUG c.c.c.c.config.loader.CarrierLoader - Successfully loaded config through carrier.
15:55:05.996 [pool-1-thread-1] DEBUG c.c.c.c.config.loader.CarrierLoader - Successfully loaded config through carrier.
15:55:06.238 [RxComputationThreadPool-2] DEBUG c.c.c.c.config.ConfigurationProvider - Applying new configuration DefaultCouchbaseBucketConfig{name='visitors-odedh', locator=null, uri='/pools/default/buckets/visitors-odedh?bucket_uuid=b496dc8a7cbae31046c914fff06dea2f', streamingUri='/pools/default/bucketsStreaming/visitors-odedh?bucket_uuid=b496dc8a7cbae31046c914fff06dea2f', nodeInfo=[NodeInfo{viewUri='http://obbe-centos1.tlv.lpnet.com:8092/visitors-odedh', hostname=obbe-centos1.tlv.lpnet.com/192.168.15.120, configPort=8091, directServices={VIEW=8092, CONFIG=8091, BINARY=11210}, sslServices={}}, NodeInfo{viewUri='http://obbe-centos2.tlv.lpnet.com:8092/visitors-odedh', hostname=obbe-centos2.tlv.lpnet.com/192.168.15.2, configPort=8091, directServices={VIEW=8092, CONFIG=8091, BINARY=11210}, sslServices={}}, NodeInfo{viewUri='http://obbe-centos3.tlv.lpnet.com:8092/visitors-odedh', hostname=obbe-centos3.tlv.lpnet.com/192.168.15.190, configPort=8091, directServices={VIEW=8092, CONFIG=8091, BINARY=11210}, sslServices={}}], partitionInfo=PartitionInfo{numberOfReplicas=1, partitionHosts=[obbe-centos1.tlv.lpnet.com, obbe-centos2.tlv.lpnet.com, obbe-centos3.tlv.lpnet.com], partitions=[[m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [1]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 0, r: [2]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [0]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 1, r: [2]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [0]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]], [m: 2, r: [1]]], forwardPartitions=[]}, tainted=false, rev=2070}
15:55:06.256 [nioEventLoopGroup-2-7] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ConfigEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:8091
15:55:06.257 [nioEventLoopGroup-2-7] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ConfigService obbe-centos2.tlv.lpnet.com
15:55:06.257 [nioEventLoopGroup-2-4] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ConfigEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:8091
15:55:06.257 [nioEventLoopGroup-2-4] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ConfigService obbe-centos1.tlv.lpnet.com
15:55:06.257 [nioEventLoopGroup-2-6] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ViewEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:8092
15:55:06.257 [nioEventLoopGroup-2-6] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ViewService obbe-centos2.tlv.lpnet.com
15:55:06.258 [nioEventLoopGroup-2-1] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ViewEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:8092
15:55:06.258 [nioEventLoopGroup-2-1] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ViewService obbe-centos3.tlv.lpnet.com
15:55:06.258 [nioEventLoopGroup-2-3] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ViewEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:8092
15:55:06.259 [nioEventLoopGroup-2-3] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ViewService obbe-centos1.tlv.lpnet.com
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.endpoint.Endpoint - Connected to QueryEndpoint obbe-centos1.tlv.lpnet.com/192.168.15.120:8093
15:55:06.266 [nioEventLoopGroup-2-5] DEBUG c.c.client.core.endpoint.Endpoint - Connected to QueryEndpoint obbe-centos2.tlv.lpnet.com/192.168.15.2:8093
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to QueryService obbe-centos1.tlv.lpnet.com
15:55:06.266 [nioEventLoopGroup-2-5] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to QueryService obbe-centos2.tlv.lpnet.com
15:55:06.266 [nioEventLoopGroup-2-8] DEBUG c.c.client.core.endpoint.Endpoint - Connected to QueryEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:8093
15:55:06.266 [nioEventLoopGroup-2-2] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:06.266 [nioEventLoopGroup-2-5] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:06.266 [nioEventLoopGroup-2-8] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to QueryService obbe-centos3.tlv.lpnet.com
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG com.couchbase.client.core.node.Node - Connected (DEGRADED) to Node obbe-centos1.tlv.lpnet.com/192.168.15.120
15:55:06.266 [nioEventLoopGroup-2-5] DEBUG com.couchbase.client.core.node.Node - Connected (DEGRADED) to Node obbe-centos2.tlv.lpnet.com/192.168.15.2
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.endpoint.Endpoint - Connected to ConfigEndpoint obbe-centos3.tlv.lpnet.com/192.168.15.190:8091
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG c.c.client.core.service.Service - Connected (CONNECTING) to ConfigService obbe-centos3.tlv.lpnet.com
15:55:06.266 [nioEventLoopGroup-2-2] INFO com.couchbase.client.core.node.Node - Connected to Node obbe-centos3.tlv.lpnet.com/192.168.15.190
15:55:06.266 [nioEventLoopGroup-2-2] DEBUG com.couchbase.client.core.node.Node - Connected (DEGRADED) to Node obbe-centos3.tlv.lpnet.com/192.168.15.190
Comment by Michael Nitschinger [ 21/Aug/14 ]
Hi,

can you please retry with dp3 and see if the issue still persists? If so, please also provide the debug logs - thanks! Also, can you specify what "hangs" means? Your println does never show?
Comment by Oded Hassidi [ 24/Aug/14 ]
Thanks, yes my println doesn't ever show!
Will check it with the beta, sorry for the delay I was OOO couple of days :)
Comment by Michael Nitschinger [ 04/Sep/14 ]
Hey, any update on this?
Comment by Michael Nitschinger [ 15/Sep/14 ]
I've removed the fix version since it's not clear if its a bug or not anymore. Please provide any details if you can as soon as possible so we can fix it if needed thanks!
Comment by Oded Hassidi [ 21/Sep/14 ]
Hi Michael and thanks
Sorry for not being responsive in the last few weeks

I will try to check this and respond with latest findings in the next couple of days

Thanks again




[JCBC-564] Add release notes for 2.0 GA Created: 19/Sep/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Task Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-565] Adapt README with new location and getting started Created: 19/Sep/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-563] View query will lock up after first attempt Created: 18/Sep/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 2.0-beta, 2.0-beta2
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Critical
Reporter: casmeiron Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: deadlock, queryview
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: JAVA 1.8
MAC 10.9.4

Issue Links:
Dependency
depends on JVMCBC-40 View responses are stuck occasionally... Open

 Description   
If you try to run a simple query view twice at beta, you will notice that only #1 attempt will work, the second will wait indefinitely.

In beta-2, the query won't work at all, it will block right away!!

I've tested with async and sync mode in both releases.

Could you please take a look on that?
Thanks.


    @Test
    public void testAsync() throws Exception {
        CouchbaseCluster cluster = CouchbaseCluster.create(DefaultCouchbaseEnvironment.builder().build());
        Bucket bucket = cluster.openBucket("default");

        CountDownLatch latch = new CountDownLatch(10);

        for (int i = 0; i < 10; i++) {
            bucket
                    .async()
                    .query(ViewQuery.from("test", "test"))
                    .doOnNext(result -> {
                        if (!result.success()) {
                            System.err.println(result.error());
                        }
                    })
                    .flatMap(AsyncViewResult::rows)
                    .subscribe(row -> {
                        System.out.println(row.id());
                        latch.countDown();
                    });
        }

        latch.await();
    }




 Comments   
Comment by Matt Ingenthron [ 18/Sep/14 ]
Sergey: you may get to this before Michael
Comment by Michael Nitschinger [ 19/Sep/14 ]
Interesting, could you please share you view? When trying to reproduce this issue I found a different one, but it doesn't lock up immediately.

Thanks
Comment by Michael Nitschinger [ 19/Sep/14 ]
One thing, as a side note, I found in your view code is that you're counting down the latch for each row that returns, and that just once. You probably want more something like this?

                .flatMap(AsyncViewResult::rows)
                .doOnCompleted(latch::countDown)
                .subscribe(row -> System.out.println(row.id()));
Comment by Michael Nitschinger [ 19/Sep/14 ]
This appears to be a race condition if you actually do nothing with the results itself.

So if you do

while(true) {
      bucket.query(ViewQuery.from("beer", "brewery_beers")).toBlocking().single();
}

it will lock up, but if you add in a Thread.sleep(100) or do something with the results like printing it it will continue to work. Looking into it.
Comment by casmeiron [ 19/Sep/14 ]
Hi,
 The view emit all docs, exactly like the sample when you create a new view from console.
Thanks for improving my code with latch.
Comment by Michael Nitschinger [ 19/Sep/14 ]
I think the bug you were hitting is https://github.com/couchbase/couchbase-jvm-core/commit/805cb611deea9f36eb4cb6743733ff463e4b9f0c JVMCBC-39, but this uncovered another one which we're currently working on.
Comment by Michael Nitschinger [ 19/Sep/14 ]
One bug has been fixed, the other depends on JVMCBC-40. We are actively looking into fixing this for 2.0.1, for now please apply a lower timeout and retry the operation, this will help. 1.0.0 GA has an associated fix with a slightly different issue.
Comment by casmeiron [ 19/Sep/14 ]
Ok, I couldn't make it work setting the timeout for query and view. I will wait till you guys have a fix so we can try it again.
Thanks for the effort spent looking into this.
Best Regards.




[JCBC-558] Improve Javadoc for (Async)Bucket Created: 15/Sep/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-557] Improve Javadoc for the .document package Created: 15/Sep/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-534] Enhance documentation how JSON is handled Created: 28/Aug/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-405] Allow custom server ports in the test suite (was: apache http library throws 'badurl' error while running Java SDK ant test against cluster on non-default port.) Created: 23/Jan/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Venu Uppalapati Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: CentOS 6.4 64-bit
RAM:256GB
Dual 2.9GHz 8-core Xeon E5-2690 for 32 total cores (16 + hyperthreading)


 Description   
This is part of multi-instance testing for customer 'A'.

Steps to reproduce:
1)Setup a 3 instances per physical machine. configure a cluster of 5 such instances. The instances are modified to run on non-default ports by modifying static config file. for example:
{rest_port,9000}.
{mccouch_port,9001}.
{memcached_port,12000}.
{memcached_dedicated_port,12001}.
{moxi_port,12002}.
{short_name,"ns11"}.
{ssl_rest_port,11000}.
{ssl_capi_port,11001}.
{ssl_proxy_downstream_port,11002}.
{ssl_proxy_upstream_port,11003}.

2)clone java SDK project. change the server ip address and server port number(internal memcached port) in the build.xml file to reflect the server address and non-default internal memcached port (12001 in this case).
example:sysproperty key="server.address_v6" value="${server.address_v6}"/>
            <sysproperty key="server.port_number" value="12001"/>

3) the junit tests in the Java SDK contain hardcoded REST port, so modify all of them to non-default REST port:
find . -name "*.java" -print0 | xargs -0 sed -i '' -e 's/8091/9000/g'

4)Run 'ant test'. the ClusterManager tests fail with following apache http library error. the library methods are called from ClusterManager.java

2014-01-23 16:35:03.786 WARN com.couchbase.client.ClusterManager: Cluster Response failed with:
    [junit] java.net.UnknownHostException: badurl
    [junit] at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.validateAddress(DefaultConnectingIOReactor.java:245)
    [junit] at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processSessionRequests(DefaultConnectingIOReactor.java:265)
    [junit] at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:141)
    [junit] at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
    [junit] at com.couchbase.client.ClusterManager$2.run(ClusterManager.java:589)
    [junit] at java.lang.Thread.run(Thread.java:695)


 Comments   
Comment by Michael Nitschinger [ 23/Jan/14 ]
Alright, this will need some larger changes in the test suite, we didn't anticipate this back in the days.

Which ports do you need to have changed?
Comment by Venu Uppalapati [ 23/Jan/14 ]
All of the ports listed in step 1 of description need to be changed per instance.




[JCBC-407] Add Utility to load bootstrap URIs from DNS SRV Created: 29/Jan/14  Updated: 19/Sep/14

Status: In Progress
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-535] Add documentation on different document types Created: 28/Aug/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-533] Add documentation for 1.x migration Created: 28/Aug/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Matt Ingenthron [ 13/Sep/14 ]
disregard my earlier comment on this sergey.




[JCBC-520] Running SSL Test on 2.0 Client hangs when certificate is not valid Created: 20/Aug/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-dp3
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Wei-Li Liu Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The SSL test I make can be seen from
https://github.com/weilliu/couchbase-java-client/commit/97c3144692627d6e657a0cc157f5f9aa644de0f2
http://review.couchbase.org/#/c/40763/

Currently if we enable ssl, and when it hits a certificate which is not valid/not exist, the test would hang


11:38:16.868 [DEBUG] [TestEventLogger] com.couchbase.client.java.SSLTest > ClientCertificateRefresh STARTED
11:38:17.843 [DEBUG] [TestEventLogger]
11:38:17.843 [DEBUG] [TestEventLogger] com.couchbase.client.java.SSLTest > ClientCertificateRefresh STANDARD_OUT
11:38:17.843 [DEBUG] [TestEventLogger] File copied!
11:38:17.844 [DEBUG] [TestEventLogger] Reconnect with the Server
> 11:39:06.2 [ DEB] [org.gradle.process.internal.ExecHandleRunner] Abort requested. Destroying process: Gradle Worker 1.
> Building 85% > :integrationTest > 29 tests completed, 29 skippedWei-Lis-MacBook-Pro:couchbase-java-client wei-li$






[JCBC-385] Configurable number of threads for client Created: 02/Dec/13  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: Task Priority: Minor
Reporter: James Mauss Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Often the Client will create a number of threads for some features that the client does not plan on using, like views. This causes some extra overhead that is not needed.

It would be nice if the total number of threads that a client object creates is configurable.

 Comments   
Comment by Michael Nitschinger [ 19/Dec/13 ]
note that this will e fully fixed in 2.0, but in 1.3 we will be much better with that on views.




[JCBC-518] no intro page on the javadoc Created: 18/Aug/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-dp3
Fix Version/s: 2.0.1
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Goal is to have the index page for the javadoc give an introductory overview so a developer knows which class to begin with. The 1.4 javadoc does this already, but the 2.0 doesn't have introductory HTML getting built in to the project. At a minimum, it should describe accessing a cluster and a bucket with a code sample that can be cut and paste into a working program.

 Comments   
Comment by Matt Ingenthron [ 13/Sep/14 ]
Sergey: This should be an easy one. There is a similar change on the release14 branch.




[JCBC-478] Docs: Deferred view deployment instruction does not work Created: 17/Feb/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: 1.3.0
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Minor
Reporter: Don Stacy Assignee: Amy Kurtzman
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Section: http://docs.couchbase.com/couchbase-sdk-java-1.3/#preview-the-application
Area: Text under step 2
Issues: We say that we will deploy the views to production later. However, the app will not work unless the views are in production. We should include the steps here or at least tell the reader to publish while pointing them to the View documentation.




[JCBC-441] add SSL support in support of Couchbase Server 3.0 Created: 27/Mar/14  Updated: 19/Sep/14  Resolved: 19/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 2.0.0
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
blocks MB-10084 Sub-Task: Changes required for Data E... Open

 Description   
Add support to the client library in support of SSL. Behavior should be that the client will try SSL first, unless otherwise specified.

See other specifications and details on CCBC-344

 Comments   
Comment by Matt Ingenthron [ 11/Sep/14 ]
What's left on this? Done?




[JCBC-555] Add support for JsonValue.NULL on the transcoders and documents Created: 15/Sep/14  Updated: 19/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-562] Java 2.0 client feature test shows timeout exception on design document test Created: 17/Sep/14  Updated: 17/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Wei-Li Liu Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
This is the only failed integration test from beta2

Class com.couchbase.client.java.DesignDocumentTest

java.lang.RuntimeException: java.util.concurrent.TimeoutException
at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:481)
at rx.observables.BlockingObservable.single(BlockingObservable.java:348)
at com.couchbase.client.java.bucket.DefaultBucketManager.flush(DefaultBucketManager.java:138)
at com.couchbase.client.java.bucket.DefaultBucketManager.flush(DefaultBucketManager.java:60)
at com.couchbase.client.java.util.ClusterDependentTest.connect(ClusterDependentTest.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:355)
at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at rx.internal.operators.OperatorTimeoutBase$TimeoutSubscriber.onTimeout(OperatorTimeoutBase.java:169)
at rx.internal.operators.OperatorTimeout$1$1.call(OperatorTimeout.java:42)
at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:43)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 more




[JCBC-536] Interpretation and Performance Impact of Client Profiling Created: 29/Aug/14  Updated: 17/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Jeff Dillon Assignee: Sergey Avseyev
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to

 Description   
Customer is using client side profiling as suggested in:

http://blog.couchbase.com/whats-new-couchbase-java-sdk-12

Do we have documentation on the Meters, Counters and Histogram output? What would the performance impact be, if any? Can this be safely left in place in a Production environment, for example?

 Comments   
Comment by Jeff Dillon [ 02/Sep/14 ]
Here are specific asks:

1) We need documentation on counters, meters and histogram output.
2) Can we run clients with MetricType = PERFORMANCE for extended duration? What's the overhead?
3) Can we change the net.spy.metrics.reporter.type to hook it up to Graphite? The metrics-core library does offer support for a GraphiteReporter. Reference: http://metrics.dropwizard.io/manual/graphite/#manual-graphite
4) Is it possible to add custom metrics over and above what's being gathered by the Couchbase java client? If so, how?
Comment by Deepu Bhatia [ 09/Sep/14 ]
Hi Michael, Matt,
Can we please get an update on this?

Thanks,
-Deepu
Comment by Michael Nitschinger [ 10/Sep/14 ]
Hi,

sorry, I've been quite busy lately...

Currently there is not much more documentation available, and metrics can be used in production environments. That said, while the metrics library is designed to be used in production it is not "zero overhead" and should be used with care. PERFORMANCE just logs less information, only those needed to find performance infos.

wrt to graphite: yes this should be possible to do, but I've never tried graphite on my own together with metrics.

You can use the MetricCollector provided on the Factory and use the methods like add counter with a name and then you can increment the counter as well. This API is used in the core, but to my knowledge is not (widely) used out there in general, but it should definitely work. If it does not, its a bug.
Comment by Deepu Bhatia [ 11/Sep/14 ]
Hi Michael,
At the bare minimum, I'd at least like to get the definitions for each of the metrics so that there's no misunderstanding. Here's a list of files being created when we use CSV logging.

[MEM] Shutting Down Nodes (NodesToShutdown).csv
[MEM] Response Rate: Success.csv
[MEM] Response Rate: Retry.csv
[MEM] Response Rate: Failure.csv
[MEM] Response Rate: All (Failure + Success + Retry).csv
[MEM] Request Rate: All.csv
[MEM] Reconnecting Nodes (ReconnectQueue).csv
[MEM] Average Time on wire for operations (µs).csv
[MEM] Average Bytes written to OS per write.csv
[MEM] Average Bytes read from OS per read.csv

This is critical to understand to unblock us. I understand that it's not zero overhead. We'll benchmark the performance with PERFORMANCE mode enabled and decide accordingly.
Comment by Deepu Bhatia [ 12/Sep/14 ]
Michael, Matt,
This has become critical for us. We have client side profiling capability now but we don't have an exact definition of the metric. This is a show stopper for us.
Comment by Matt Ingenthron [ 13/Sep/14 ]
Depu: The ones mentioned as "rate" are counters, so you'll need to collect with regularity. The average time and byte counts are just that.

Is there a particular question you're trying to answer? Maybe with that we can give some more specific guidance. I understand it may have to do with latency and where it is? Does the average time on wire correlate to what you're seeing from cbstats timings would be the thing I would be checking for. I'd also be looking for reconnect to be non-zero and retry rate being non-zero.
Comment by Deepu Bhatia [ 16/Sep/14 ]
Hi Matt,
Here's what I am specifically looking for:

[MEM] Shutting Down Nodes (NodesToShutdown).csv - what does this mean?
[MEM] Response Rate: Success.csv - what does this mean?
[MEM] Response Rate: Retry.csv - What retries are this?
[MEM] Response Rate: Failure.csv - What failures does this cover? Is this tracking timeout exceptions?
[MEM] Response Rate: All (Failure + Success + Retry).csv
[MEM] Request Rate: All.csv
[MEM] Reconnecting Nodes (ReconnectQueue).cvs - What is this metric tracking?
[MEM] Average Time on wire for operations (µs).csv - Is this pure network time? Is it in microseconds?
[MEM] Average Bytes written to OS per write.csv
[MEM] Average Bytes read from OS per read.csv

Comment by Michael Nitschinger [ 17/Sep/14 ]
Hi Deepu,

here is more info on each specific metric:

[MEM] Shutting Down Nodes (NodesToShutdown).csv - what does this mean?

>> contains nodes which are currently pending shutdown in the IO client. This means that if a node is failovered or removed, at some point it needs to be shutdown. This mostly indicates that a cluster is currently undergoing some changes.

[MEM] Response Rate: Success.csv - what does this mean?

>> How many responses returned with a "success" state from the server. so no failures or retries.

[MEM] Response Rate: Retry.csv - What retries are this?

>> how many responses returned from the server which are to be retried. This is mostly covered by "not my vbucket" responses that are coming up during a rebalance operation. If such ops are to be seen, then the cluster is currently or recently undergoing a rebalance operation.

[MEM] Response Rate: Failure.csv - What failures does this cover? Is this tracking timeout exceptions?

>> Operations that had been failed from the server side. This should normally not be larger than 0, if so something is wrong. This can indicate all forms of errors and the logs need to be investigated further.

[MEM] Response Rate: All (Failure + Success + Retry).csv

>> the response rate for all those responses combined (to get an aggregated view on the response rate)

[MEM] Request Rate: All.csv

>> rate of outgoing requests

[MEM] Reconnecting Nodes (ReconnectQueue).cvs - What is this metric tracking?

>> the number of all nodes that are currently waiting to be reconnected. this could be due to too many failing/timing out ops (issuing a manual reconnect), or the socket is closed upon us. A reconnecting node is not able to process operations, so this is indicating some form of unstable/fluent state.

[MEM] Average Time on wire for operations (µs).csv - Is this pure network time? Is it in microseconds?

>> this is nearly network time, and yes in microseconds. It is measured once we put the op over to the JVM and the second time when we start reading it off the JVM NIO parts. So it does not include only networking time, but also OS and JVM socket processing. But it cuts out other parts like transcoding.

[MEM] Average Bytes written to OS per write.csv

>> this is an indication of how well the syscalls are used, the higher the batch size the better the utilization. Lower utilization means more overhead for packets and less efficient network handling.

[MEM] Average Bytes read from OS per read.csv

>> see above, just for reads.




[JCBC-544] Does not throw error when querying a non existent view/designdoc Created: 05/Sep/14  Updated: 16/Sep/14  Resolved: 16/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Bug Priority: Major
Reporter: Subhashni Balakrishnan Assignee: Sergey Avseyev
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Here is the code I used. This doesn't call onError.
 

ViewQuery queryTemplate = ViewQuery.from("test","dummy");
    queryTemplate.stale(Stale.FALSE);
    Observable<ViewRow> viewRowObservable = bucket.query(queryTemplate).flatMap(new Func1<ViewResult, Observable<ViewRow>>() {
      public Observable<ViewRow> call(ViewResult res) {
        return res.rows();
      }
    });

   viewRowObservable.subscribe(new Observer<ViewRow>() {
     @Override
     public void onCompleted() {
      System.out.println("Completed");
     }

     @Override
     public void onError(Throwable throwable) {
      System.out.println(throwable.getMessage());
     }

     @Override
     public void onNext(ViewRow viewRow) {
     }
   });

 Comments   
Comment by Matt Ingenthron [ 09/Sep/14 ]
Sergey: Can you see if this is reproducible off the current master and if so investigate?
Comment by Michael Nitschinger [ 09/Sep/14 ]
Hi,

some remarks: it's just not implemented. The core should send the response back as a failure (see the status). Chain in an onNext and check for the error. if it is, throw a ViewDoesNotExist execption. Put the exception in the error namespace, the others are also in there.
Comment by Sergey Avseyev [ 12/Sep/14 ]
http://review.couchbase.org/41369
Comment by Michael Nitschinger [ 16/Sep/14 ]
I've removed the blocker, because right now you can get access to the error on the observable.

We need to work through some more tricky semantics, but this will not break the API at this point. Sergey will update this ticket once there is more progress, but its not holding us back for the beta2.
Comment by Michael Nitschinger [ 16/Sep/14 ]
Okay one step back, we'll add an exception for this in beta2 (ViewDoesNotExistException) if 404 is returned. For more tricky cases we'll handle it in separate tickets.




[JCBC-560] Add jar info in line with core io Created: 16/Sep/14  Updated: 16/Sep/14  Resolved: 16/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-561] Rename binary to kv for less ambiguity Created: 16/Sep/14  Updated: 16/Sep/14  Resolved: 16/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-525] Add support for full JSON compat docs & common flags Created: 22/Aug/14  Updated: 16/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-545] Couchbase Connection leaks to a single node in the cluster Created: 05/Sep/14  Updated: 15/Sep/14  Due: 14/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.2
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Critical
Reporter: Justin Michaels Assignee: Michael Nitschinger
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We are observing 2x the number of connections open on a single node in a given Couchbase cluster. All connections are on port 11210.

The development team believes they have identified a bug that creates this behavor during a cluster change and a new map being distributed to the Couchbase clients. Below is their observation.
... there in the list to randomize the nodes, but there’s a bug in the constructor of Couchbase SDK class com.couchbase.client.vbucket.provider.BucketConfigurationProvider

Here’s the code:

  public BucketConfigurationProvider(final List<URI> seedNodes,
    final String bucket, final String password,
    final CouchbaseConnectionFactory connectionFactory) {
    config = new AtomicReference<Bucket>();
    configurationParser = new ConfigurationParserJSON();
    httpProvider = new AtomicReference<ConfigurationProviderHTTP>(
      new ConfigurationProviderHTTP(seedNodes, bucket, password)
    );
    refreshingHttp = new AtomicBoolean(false);
    pollingBinary = new AtomicBoolean(false);
    observers = Collections.synchronizedList(new ArrayList<Reconfigurable>());
    binaryConnection = new AtomicReference<CouchbaseConnection>();

    this.seedNodes = Collections.synchronizedList(new ArrayList<URI>(seedNodes));
    this.bucket = bucket;
    this.password = password;
    this.connectionFactory = connectionFactory;
    potentiallyRandomizeNodeList(seedNodes);

    disableCarrierBootstrap = Boolean.parseBoolean(
      CouchbaseProperties.getProperty("disableCarrierBootstrap", "false"));
    disableHttpBootstrap = Boolean.parseBoolean(
      CouchbaseProperties.getProperty("disableHttpBootstrap", "false"));
  }


The potentiallyRandomizeNodeList method will randomize the list passed to it and in fact, in our clients we ARE randomizing the two-element list we are passing.
Unfortunately, the wrong list is being passed to this method. The code should be randomizing the list stored as a member variable.
The line should read
potentiallyRandomizeNodeList(this.seedNodes);

because four lines above a completely new list is constructed by making a shallow copy of the input argument, seedNodes.

The list later used to get the configuration information is always used ordered and that’s why all our management connections are to the same node, the first one in our list.

I found this in v1.4.2 of the client, but v1.4.4 seems to have the same code so I don’t think it’s been fixed yet.

 Comments   
Comment by Dan Douglas [ 06/Sep/14 ]
I found this issue.

The symptom is that the configuration information is always obtained from the first node in seed list passed and the shuffle is not done.

The line that does the shuffle in the constructor needs to reference the seedNodes member variable, not the argument. Needs to add "this." like so

potentiallyRandomizeNodeList(this.seedNodes);






[JCBC-559] Retry view responses if possible Created: 15/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
depends on JVMCBC-35 Expose View retry codes in response s... Open




[JCBC-556] Add support for optional decompression on legacy decodes Created: 15/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-554] Provide backoff handling utility classes for the user Created: 15/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.1.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-538] Consider adding bulk APIs Created: 01/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.1.0
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-542] Handle TMPFAIL and expose it Created: 04/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-539] Consider a way to expose target key nodes for user-level circuit breakers Created: 01/Sep/14  Updated: 15/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0.1
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Java Source File HelloWorld.java     Java Source File NodeLocator.java    
Issue Links:
Relates to
relates to CCBC-505 Allow user-side modification of node'... Open

 Comments   
Comment by Dan Douglas [ 02/Sep/14 ]
Attached hacked/reflected workaround I used to get locator information and simple HelloWorld showing usage.

As indicated, real usage is for creating Hystrix command wrappers around CB functions to do circuit breakers for resiliancy in the face of failing (but not yet failed over) nodes.
Comment by Matt Ingenthron [ 02/Sep/14 ]
Thanks Dan! This is something we've been discussing a bit and I really wonder if in general we want a "fast-fail" behavior to be defined. Maybe we need another word. Basic idea is that there are some use cases where people don't want to wait for the timeout value while the connection is known to be down. They want to fast-fail while that connection is attempted to be rebuilt. I didn't read all the code, but is this the kind of behavior you were going for by checking node status?
Comment by Dan Douglas [ 03/Sep/14 ]
Fail fast is one aspect.

However, our plan also calls for a policy driven fallback

- readFromReplica
- read/write to secondary cluster
- user supplied fallback

What I want is access to the locator data so I can associate a key with a node (both primary and replicas).

I don't intend to check the client's status of the node through any CB API. I'll identify the iffy nodes using the outcome of operations associated with them by wrapping them in Hystrix commands. The locator info is just to creat the name of the command.

For even more context, here’s a small deck describing the work I’m doing to add more resiliency to the DUKES wrapper we use for configuration and application monitoring of Couchbase. It includes a short six-minute embedded video demonstrating how the proposed implementation shields applications from badness when a CB node is failing, but has not yet failed over.
https://www.dropbox.com/s/8hue2za1kjje7zu/DUKESv2-Resiliant-Hystrix-Commands.pptx?dl=0 (156 MB)

Some folks have said that they’ve had trouble running the embedded video from the PPT. Here’s a direct link to the .mp4
https://www.dropbox.com/s/zaek5042bk3rhl0/DukesV2Demo.mp4?dl=0 (153 MB)

The NodeLocator implementation attached is what I’ll need to do in order to implement DUKESv2 onto of Couchbase SDK 2.0. I’m not really looking forward to maintaining the hack. I’m hoping to convince you to add back some sort of locator information to the SDK so that I won’t need to.
Comment by Mark Nunberg [ 03/Sep/14 ]
If I may chime in here a bit. The same feature has been requested for the C client library. I'd also say that rather than stuffing all the logic into the client library, providing the facilities and tools for a user to manage such a "circuit breaker" functionality.




[JCBC-481] Integration Tests hang up after 85% progress Created: 25/Jun/14  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
[root@pomelo-11017 couchbase-java-client]# ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava UP-TO-DATE
:processIntegrationResources UP-TO-DATE
:integrationClasses UP-TO-DATE
> Building 85% > :integrationTest

 Comments   
Comment by Wei-Li Liu [ 18/Aug/14 ]
The process hangs at 85% due to view query test.
Just runs 2.0.0-dp3 integration test against the new beta2 build, and all the tests completes.
Comment by Deepti Dawar [ 21/Aug/14 ]
Following error appears with latest DP3 -

[root@pomelo-11017 couchbase-java-client]# ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
1 warning
:processIntegrationResources
:integrationClasses
:integrationTest

com.couchbase.client.java.BinaryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.BinaryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.DesignDocumentTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.DesignDocumentTest > classMethod FAILED
    java.util.NoSuchElementException

4 tests completed, 4 failed
:integrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':integrationTest'.
> There were failing tests. See the report at: file:///root/couchbase/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 22.09 secs
Comment by Michael Nitschinger [ 21/Aug/14 ]
Does it also happen on master?
Comment by Wei-Li Liu [ 21/Aug/14 ]
I test it on master and only see JCBC-479




[JCBC-551] Implement blocking API around async one Created: 11/Sep/14  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Task Priority: Blocker
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-523] Add support for memcached buckets Created: 22/Aug/14  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Sergey Avseyev
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Matt Ingenthron [ 13/Sep/14 ]
Sergey: I believe this is done. Can you verify?
Comment by Sergey Avseyev [ 15/Sep/14 ]
Verified, memcache buckets supported with exception of APIs which are not available there (like locking)




[JCBC-228] a no-args constructor and an init method are needed Created: 31/Jan/13  Updated: 15/Sep/14  Resolved: 15/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1.0, 1.1.1
Fix Version/s: 2.0.0
Security Level: Public

Type: New Feature Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Currently, the CouchbaseClient does a number of things that are not so pretty, like spinning up a thread from it's ctor. It would be better to add an additional, optional no-args constructor which expects an init method to be called with the other params needed, start thread, etc.

 Comments   
Comment by Michael Nitschinger [ 31/Jan/13 ]
Assigning towards 1.1.2 but we can defer it to 1.1.3 as well.
Comment by Michael Nitschinger [ 19/Dec/13 ]
We'll have something like this in 2.0
Comment by Matt Ingenthron [ 11/Sep/14 ]
Sergey, I think this is true now in 2.0, can you verify and close this if so?
Comment by Sergey Avseyev [ 12/Sep/14 ]
Yes, we have static method create() which does not have any arguments, but it is just provides defaults to full constructor, which does blocking connection anyway.

We might extract connect() method to the public interface, and modify static constructors somehow they will allow to do at least asynchronous connection. I did some sketch here (it just extracts connect() method, but still call it from constructor) http://review.couchbase.org/41397

The patch is not meant to merged, just demonstrate my question.

Matt, could you expand a little bit what we need to achieve?

Does it spawn thread in constructor? No
Does it do blocking connection? Yes
Comment by Matt Ingenthron [ 12/Sep/14 ]
Assigning this to Michael as he's the architecture owner here.

The reason I filed this way back at -228 was that in general, a recommendation with Java is to not do anything requiring IO or threads in the ctor and separate out the actions of setting things up in a separate method. This is often helpful in some frameworks with DI, etc. Imagine for a moment that you throw an IOException from the ctor(). Chances are whatever created that object inline isn't ready to handle that exception right there. However, calling .init() on it and seeing an explicit throws IOException gives the app developer a cleaner way of abstracting the set up of the things and the initializing of the things.

http://www.javapractices.com/topic/TopicAction.do?Id=254

All of these things change over time though, so I trust Michael to pick a solution he can live with.
Comment by Sergey Avseyev [ 12/Sep/14 ]
Michael, I also think that current implementation is good. Abandoned my patch. Please resolve the ticket, or reassign back to me
Comment by Michael Nitschinger [ 15/Sep/14 ]
Yes, we have static factory methods now and should be good on that.

This was actually a relict out of changing the 1.* SDK when we didn't know that 2.0 was a breaking change. So I'm going to close this out.
Comment by Michael Nitschinger [ 15/Sep/14 ]
Requirements have changed since 2.0 is a complete rewrite its not needed anymore




[JCBC-553] 2.0.0.beta hangs on 2nd query Created: 12/Sep/14  Updated: 12/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-beta
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Sean Colonello Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Client is Windows 8.1, server is CentOS 6 running 3.0 beta.

Attachments: Text File two-queries-hang.log    
Issue Links:
Duplicate
is duplicated by JVMCBC-28 ViewHandler can end up not completing... Resolved

 Description   
The following code hangs on the 2nd query.

CouchbaseCluster cluster = CouchbaseCluster.fromConnectionString("couchbase://192.168.1.75");
Bucket bucket = cluster.openBucket("trade-unittest", "blah").toBlocking().first();
bucket
.query(ViewQuery.from("devOnly", "allDocs"))
.flatMap(result -> result.rows())
.toBlocking()
.forEach(row -> logger.log(Level.INFO, row.id()));
bucket
.query(ViewQuery.from("devOnly", "allDocs"))
.flatMap(result -> result.rows())
.toBlocking()
.forEach(row -> logger.log(Level.INFO, row.id()));
cluster.disconnect();





[JCBC-549] Implement setHashAlg() / Implement ability to change hashing algorithm Created: 11/Sep/14  Updated: 12/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: New Feature Priority: Major
Reporter: Mark Woosey Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Gantt: start-finish
Relates to
relates to PYCBC-250 support for distributing data by hashkey Open

 Description   
CouchbaseConnectionFactoryBuilder.setHashAlg() is currently not operational.




[JCBC-552] Support other sources for key store Created: 12/Sep/14  Updated: 12/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-beta
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Cihan Biyikoglu Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-532] Implement Common Flags Created: 27/Aug/14  Updated: 11/Sep/14  Resolved: 11/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Task Priority: Major
Reporter: Brett Lawson Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Implement the Common Flags formatting as defined by the specification.

 Comments   
Comment by Michael Nitschinger [ 11/Sep/14 ]
dups 525




[JCBC-384] couchbase java libs should use log4j or slf4j not JUL Created: 11/Sep/13  Updated: 11/Sep/14  Resolved: 11/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Dependencies
Affects Version/s: 1.4.0
Fix Version/s: 2.0-dp3
Security Level: Public

Type: Bug Priority: Major
Reporter: Timothy Ehlers Assignee: Michael Nitschinger
Resolution: Fixed Votes: 2
Labels: usability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
log4j has more appends and is widely used, it is also more enterprise friendly thanks to the SocketAppender.

 Comments   
Comment by Timothy Ehlers [ 11/Sep/13 ]
A dependency you pull in uses log4j

net/spy/memcached/compat/log/Log4JLogger.java:import org.apache.log4j.Logger;

Then the couchbase lib uses JUL, this seems counter production
com/couchbase/client/CouchbaseClient.java:import java.util.logging.Logger;
Comment by Matt Ingenthron [ 27/Nov/13 ]
Note that we manage/maintain the net.spy.memcached project as well.

We have support for SLF4J, Log4J, and JUL. Because of how things are packaged, all dependencies are described via maven, but they're not all pulled in.
Comment by Matt Ingenthron [ 27/Nov/13 ]
Michael: do you think we can make our packaging better in the 2.0 client?
Comment by Michael Nitschinger [ 28/Nov/13 ]
That's right since 1.2 you can use slf4j. 2.0 will be slf4j only like most other projects out there, only shipping with the binding and you plug in your own impl.
Comment by Matt Ingenthron [ 11/Sep/14 ]
Michael: I think this is a resolved as of 2.0, right?
Comment by Michael Nitschinger [ 11/Sep/14 ]
it is!




[JCBC-529] Add support for spatial Created: 22/Aug/14  Updated: 11/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-548] user agent string is incorrect and doesn't work with some older HTTP core Created: 10/Sep/14  Updated: 10/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
I found that the current client can't be used in CDH5 because a JavaLangNoSuchMethodError will be thrown with org.apache.http.protocol.RequestUserAgent.<init>(Ljava/lang/String;)V since the older 4.2 Apache httpcore does not have that method of defining a string and it takes precedence in the classpath.

In the process, I found that this header wasn't being set correctly anyway.

Filing this issue to remove the header. If we get a chance, it should be fixed by re-adding with reflection or whatever method 4.2 provides.




[JCBC-547] Paginator hasNext() works incorrectly on small results Created: 09/Sep/14  Updated: 09/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Views
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: raycardillo Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: javaclient
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
While working with the Paginator in our QA system I traced down a bug that will only happen when the page size is greater than the number of items in the underlying view/query. We wanted to call hasNext() multiple times because one layer of code validates that it has a good Paginator (due to other problems we were having with our cluster). However, because the actual response is fetched during hasNext(), and because the FINISHED state is set as a result of that, only the first hasNext() will return true even though we have not retrieved the response with next() yet.

For example, we should be able to call, hasNext() multiple times, and then state is only changed to indicate FINISHED after we actually retrieve the response using next().

After tracing this bug down, I can see that hasNext() returns false right away if FINISHED is true, and I can see that the FINISHED state is set during the fetch that happens during hasNext(). So when there is less than one page of data available, the first hasNext() will return true, and the next hasNext() will return false, which is a problem if the consumer has not actually called next() to fetch the response.

I think the simple fix here is to only set the FINISHED state in next() so you know that the data has actually been retrieved by the consumer. That way the consumer is free to ask hasNext() as many times as required for their logic, and it will not effect internal state until next() is called.




[JCBC-540] Enable SDK 1.x & 2.x to be deployed side by side in a single JVM. Consider renaming couchbase-client to java-client Created: 01/Sep/14  Updated: 09/Sep/14  Resolved: 09/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Dan Douglas [ 02/Sep/14 ]
Thanks for considering this.

Renaming the artifact is a start. However, the package names should be different, too, to make sure that the class namespace has no clashes.

The goal should be coexistence of both 1.4 SDK and 2.0 SDK clients in the same JVM.

Motivation is Couchbase use by large, distributed teams that will not all migrate in lockstep, yet contribute code that runs in the same JVM.

Lack of separation at both Maven artifact AND class namespace level will impede migration to CB Java 2.0 SDK.
Comment by Michael Nitschinger [ 02/Sep/14 ]
Hm, they both have com.couchbase.client in common, but the new one is now under com.couchbase.client.java and the core is com.couchbase.client.core

Is that sufficient or would the new one be completely different and not under com.couchbase.client at all?
Comment by Michael Nitschinger [ 02/Sep/14 ]
I just tried adding both as dependencies (I locally renamed the new one to java-client and pushed a new artifact) and it seems to work nicely:

    public static void main(String... args) throws Exception {

        CouchbaseClient client = new CouchbaseClient(
            Arrays.asList(URI.create("http://127.0.0.1:8091/pools")),
            "default",
            ""
        );

        Cluster cluster = CouchbaseCluster.create();
        Bucket bucket = cluster.openBucket().toBlocking().single();

        client.set("id", "value1234").get();
        System.out.println(bucket.get("id", LegacyDocument.class).toBlocking().single());

    }
Comment by Dan Douglas [ 02/Sep/14 ]
Doesn't have to be completely different.... just as long as there are no clashes in ANY of the class names.

Different prefixes are ok.

So as long as the SDK 2.0 packages always start with

com.couchbase.client.java
com.couchbase.client.core
com.couchbase.client.deps

and the SDK 1.4 maintenance never adds those paractular packages, it should be ok.

So far, the 1.4 version uses these packages...

com.couchbase.client
com.couchbase.client.clustermanager
com.couchbase.client.http
com.couchbase.client.internal
com.couchbase.client.protocol
com.couchbase.client.vbucket

It seems a little ripe for error, however. Should someone maintaining 1.4 ever forget and slip in .java, .core or .deps ... but like you said, so far so good.

Taking a step back, though, since the 2.0 SDK doesn't actaully have a "client" class (uses Cluster and Bucket objects instead) and the use of "java" seems somewhat superfluous here, perhaps you'd consider dropping "java" and changing the 2.0 SDK packages to

com.couchbase.sdk

or

com.couchbase.jsdk

or

com.couchbase.jcbc

or

something else....



Just a thought. What you have works, although it seems a little subtle and open to creating confusion.

There's precedent for changing package/artifact names together, for example Apache Commons Lang 3 (http://commons.apache.org/proper/commons-lang/javadocs/api-3.3.2/)
Comment by Cihan Biyikoglu [ 02/Sep/14 ]
added the goal to the title for clarity: side by side deployment of 1.x and 2.x SDKs in a JVM.
Comment by Michael Nitschinger [ 04/Sep/14 ]
I'm actually very open to rename the artifact to java-client (http://review.couchbase.org/#/c/41194/), but I'm not sure its a good idea to rename the namespace. Btw "java" is in there to align with "core" for "core-io" and also in the future for other language bindings like "scala" and so forth.

Since the 1.* branch is mostly in bugfix mode right now, I don't think there will be much room for conflict, since all classes added by the new one are under one level deeper and the only thing we need to make sure is not add a "java" or "core" namespace in the current stable SDK.

Would that make sense?
Comment by Dan Douglas [ 04/Sep/14 ]
Michael, you're the man. If you think it's going to be ok to just rename the artifact, then I'll agree with you. If someone introduces a clash later in a "fix", it'll just be a bad fix that will need to be redone. Perhaps you can introduce something in your static analysis or automated test suite to catch that.
Comment by Michael Nitschinger [ 09/Sep/14 ]
Merged!




[JCBC-541] incr() returns -1 Created: 02/Sep/14  Updated: 09/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.2
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: RICHARDS PETER Assignee: Matt Ingenthron
Resolution: Unresolved Votes: 0
Labels: info-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Hi,

I am using incr() of couchbase client to increase a counter key in a bucket. I found this method very useful in some of our usecases. Around 150 threads from four machines increments current value of counter by 10 and use the values for some internal logic. Initially everything worked as expected. However a few days back I realized that couchbase returns -1 for some of the incr() method calls. That is how I noticed that incr() in MemcachedClient could return -1 if there is a failure to update the counter:
http://www.couchbase.com/autodocs/couchbase-java-client-1.4.3/net/spy/memcached/MemcachedClient.html

I have a few queries about this return value:
1. Could you please tell me the cases in which incr() would return -1? I identified one case. If the RAM quota allocated to the bucket is exhausted, we are likely to get -1. During this time addition of key-value pairs into the same bucket using add() will return false through the OperationFuture object. What are the other cases that would return -1 in the incr()?

2. Could you please tell me how the following method call works?
http://www.couchbase.com/autodocs/couchbase-java-client-1.4.3/net/spy/memcached/MemcachedClient.html#incr%28java.lang.String,%20long,%20long,%20int%29

I would like to know when the default value would be returned and when -1 would be returned. I think the default value would be returned if it is possible to insert the value into couchbase and -1 would be returned if it is not possible to retrieve the counter and update it. Am I correct?

3. Till now my project did not have a use case to increment negative values. However if somebody wants to increment -2 by 1 then the result would be -1. How will the client call be able to distinguish failure and actual increment in this case? Wouldn't it be better to throw a checked exception from incr() than returning -1? I think with checked exception we can provide more information about the cause of failure.

Richards Peter.




[JCBC-537] Setting Reader/Writer Worker value on Server causes ParseException on Client Created: 29/Aug/14  Updated: 09/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.4
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Minor
Reporter: Jeff Dillon Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File output.log    
Issue Links:
Dependency
depends on MB-12104 Carrier Config missing after R/W Conc... Open
Duplicate

 Description   
When setting the "Number of suggested reader/writer workers" in the Admin UI to 8, the 1.4.4 client then begins to see ParseException errors during the bootstrap phase. The client appears to continue as expected after that, but the error is concerning. The exception does not occur in the .NET or C client libraries.

To repro:

* Set the worker value to 8 in the UI
* Run the HelloWorld example from:

http://www.couchbase.com/communities/java/getting-started

Relevant code is:

// (Subset) of nodes in the cluster to establish a connection
    List<URI> hosts = Arrays.asList(
      new URI("http://127.0.0.1:8091/pools")
    );
 
    // Name of the Bucket to connect to
    String bucket = "default";
 
    // Password of the bucket (empty) string if none
    String password = "";
 
    // Connect to the Cluster
    CouchbaseClient client = new CouchbaseClient(hosts, bucket, password); // <- exception here

When running the example, the response is consistently:

2014-08-29 15:36:21.979 INFO net.spy.memcached.auth.AuthThread: Authenticated to /<ip_address>:11210
2014-08-29 15:36:22.014 WARN com.couchbase.client.vbucket.provider.BucketConfigurationProvider: Could not parse config, retrying bootstrap.
java.text.ParseException: A JSONObject text must begin with '{' at character 0 of
at com.couchbase.client.vbucket.config.ConfigurationParserJSON.parseBucket(ConfigurationParserJSON.java:148)



 Comments   
Comment by Matt Ingenthron [ 29/Aug/14 ]
If you crank up the log level, it should print out what the cluster is returning. Maybe the .NET and C clients are probably seeing the same behavior but not logging the warning?
Comment by Jeff Dillon [ 29/Aug/14 ]
In .NET I used the following log settings, and didn't see the error:

      <priority value="ALL" />
      <level value="DEBUG" />
Comment by Matt Ingenthron [ 29/Aug/14 ]
Yep, might be a bug that .NET doesn't log a warning on bad response and goes on to the next node.

The parse error comes from Jettison. https://github.com/couchbase/couchbase-java-client/blob/669c9580674082b2acccd769d9da5e1b3bb3e417/src/main/java/com/couchbase/client/vbucket/config/ConfigurationParserJSON.java#L148

It'd be really helpful if you could get the log to tell us what the cluster is returning.
Comment by Jeff Dillon [ 29/Aug/14 ]
Attaching Java log using:

logging.properties
handlers = java.util.logging.FileHandler
java.util.logging.FileHandler.pattern=output.log
java.util.logging.FileHandler.level = ALL
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
com.couchbase.client.vbucket.level = FINEST
com.couchbase.client.vbucket.config.level = FINEST
com.couchbase.client.level = FINEST
Comment by Michael Nitschinger [ 01/Sep/14 ]
Interesting, it could also be causing a latent parsing bug in the handler which has not been uncovered yet. I'll try to reproduce it.
Comment by Michael Nitschinger [ 01/Sep/14 ]
Wow, this looks like a server issue!

When you change the number of r/w threads the binary get config for CCCP returns an empty response.
Comment by Michael Nitschinger [ 01/Sep/14 ]
Please see http://www.couchbase.com/issues/browse/MB-12104 for more info
Comment by Michael Nitschinger [ 01/Sep/14 ]
While this is a server bug, I think we can do better on the binary bootstrap in the SDK to not try to apply a config like this and keep moving on, falling back to http.




[JCBC-543] IllegalStateException: Node not found for requestcom.couchbase.client.core.message.binary.UpsertRequest Created: 05/Sep/14  Updated: 05/Sep/14

Status: Open
Project: Couchbase Java Client
Component/s: Core, Infrastructure
Affects Version/s: 2.0-beta
Fix Version/s: None
Security Level: Public

Type: Task Priority: Major
Reporter: Jeff Morris Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This could be user error, but when trying the couchbae hellworld tutorial: http://docs.couchbase.com/prebuilt/java-sdk-2.0-beta/topics/hello-couchbase.html, I receive the following exception:

Sep 05, 2014 10:30:02 AM com.couchbase.client.core.node.CouchbaseNode$5 call
INFO: Connected to Node 127.0.0.1
Sep 05, 2014 10:30:07 AM com.couchbase.client.core.node.CouchbaseNode$5 call
INFO: Disconnected from Node 127.0.0.1
Sep 05, 2014 10:30:07 AM com.couchbase.client.core.node.CouchbaseNode$5 call
INFO: Connected to Node jmorris-pc
Sep 05, 2014 10:30:07 AM com.couchbase.client.deps.com.lmax.disruptor.FatalExceptionHandler handleEventException
SEVERE: Exception processing: 2 com.couchbase.client.core.RequestEvent@7bcfceb7
java.lang.IllegalStateException: Node not found for requestcom.couchbase.client.core.message.binary.UpsertRequest@9c94115
at com.couchbase.client.core.node.locate.BinaryLocator.locateForCouchbaseBucket(BinaryLocator.java:126)
at com.couchbase.client.core.node.locate.BinaryLocator.locate(BinaryLocator.java:63)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:148)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:68)
at com.couchbase.client.deps.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

Exception in thread "pool-1-thread-2" java.lang.RuntimeException: java.lang.IllegalStateException: Node not found for requestcom.couchbase.client.core.message.binary.UpsertRequest@9c94115
at com.couchbase.client.deps.com.lmax.disruptor.FatalExceptionHandler.handleEventException(FatalExceptionHandler.java:45)
at com.couchbase.client.deps.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:147)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.IllegalStateException: Node not found for requestcom.couchbase.client.core.message.binary.UpsertRequest@9c94115
at com.couchbase.client.core.node.locate.BinaryLocator.locateForCouchbaseBucket(BinaryLocator.java:126)
at com.couchbase.client.core.node.locate.BinaryLocator.locate(BinaryLocator.java:63)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:148)
at com.couchbase.client.core.RequestHandler.onEvent(RequestHandler.java:68)
at com.couchbase.client.deps.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
... 3 more

Additional logging included. I am connecting to localhost CB server 2.5.




[JCBC-528] Add shortcut JsonObject and JsonArray factories Created: 22/Aug/14  Updated: 04/Sep/14  Resolved: 04/Sep/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta2
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-509] Client doesn't shut down Created: 17/Aug/14  Updated: 27/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-dp3
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Adam Honen Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Java 87 update 11.
Windows 7 Service Pack 1 & Linux.

Attachments: Text File disconnection.log    
Issue Links:
Dependency

 Description   
Despite calling cluster.disconnect(), there are remaining non daemon (netty) threads which prevent the JVM from shutting down.

A thread dump from my machine when the program should have terminated:

Full thread dump Java HotSpot(TM) Client VM (25.11-b03 mixed mode):

"DestroyJavaVM" #22 prio=5 os_prio=0 tid=0x0016d800 nid=0x1f38 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"RxComputationThreadPool-4" #21 daemon prio=5 os_prio=0 tid=0x15255400 nid=0x17ac waiting on condition [0x18c7f000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0a02bd00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

"RxComputationThreadPool-3" #20 daemon prio=5 os_prio=0 tid=0x18390400 nid=0x7d0 waiting on condition [0x18d4f000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0a044918> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

"nioEventLoopGroup-2-3" #12 prio=10 os_prio=2 tid=0x154b4c00 nid=0x1ba0 runnable [0x1730f000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x0a02e818> (a com.couchbase.client.deps.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x0a03b060> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0a02e7a0> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)

"nioEventLoopGroup-2-2" #11 prio=10 os_prio=2 tid=0x154a4800 nid=0x1524 runnable [0x1735f000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x0a02e400> (a com.couchbase.client.deps.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x0a038df8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0a02e388> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)

"RxComputationThreadPool-2" #19 daemon prio=5 os_prio=0 tid=0x15374000 nid=0x1748 waiting on condition [0x14f9f000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0a0447a8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

"threadDeathWatcher-3-1" #18 daemon prio=1 os_prio=-2 tid=0x15336c00 nid=0x1ffc waiting on condition [0x15dcf000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at com.couchbase.client.deps.io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:137)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)

"nioEventLoopGroup-2-1" #10 prio=10 os_prio=2 tid=0x152f4400 nid=0x1934 runnable [0x15c3f000]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x0a02dfe8> (a com.couchbase.client.deps.io.netty.channel.nio.SelectedSelectionKeySet)
        - locked <0x0a036b90> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0a02df70> (a sun.nio.ch.WindowsSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)

"RxComputationThreadPool-1" #16 daemon prio=5 os_prio=0 tid=0x15284800 nid=0x180c waiting on condition [0x15adf000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0a044860> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00f74c00 nid=0xefc runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread0" #6 daemon prio=9 os_prio=2 tid=0x00f65800 nid=0x1600 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" #5 daemon prio=5 os_prio=2 tid=0x00f5f800 nid=0x1a78 waiting on condition [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=2 tid=0x00f5cc00 nid=0x1e54 runnable [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=1 tid=0x00f20c00 nid=0x1e30 in Object.wait() [0x14cff000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09de3798> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)
        - locked <0x09de3798> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x00f1a800 nid=0x1154 in Object.wait() [0x047bf000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x09de3938> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
        - locked <0x09de3938> (a java.lang.ref.Reference$Lock)

"VM Thread" os_prio=2 tid=0x00f17400 nid=0x1d7c runnable

"VM Periodic Task Thread" os_prio=2 tid=0x00f7e400 nid=0x1264 waiting on condition

JNI global references: 242


Code to reproduce:

    final ViewQuery view = ViewQuery
        .from("views", "myView")
        .startKey(JsonArray.from(5))
        .endKey(JsonArray.from(6))
        .reduce(false);

    logger.debug("view=" + view);

    final Observable<ViewResult> query = bucket.query(view);
    logger.debug("query=" + query);

    query
        .doOnNext(item -> logger.debug("item={}; item.success()={}", item, item.success()))
        .filter(item -> item.success())
        .doOnNext(item -> logger.debug("getting rows"))
        .flatMap(ViewResult::rows)
        .doOnNext(item -> logger.debug("view row={}", item))
        .filter(item -> choose(item))
        .take(howMany)
        .doOnNext(item -> log(item))
        .doOnError(e -> e.printStackTrace())
        .doOnTerminate(() -> {
          System.out.println("done");
          cluster.disconnect();
        })
        .subscribe();
    logger.debug("subscribed");

    Thread.sleep(1000);
    cluster.disconnect();

(This is called by the main thread after connecting to the cluster)

This will print done, but never terminate.

 Comments   
Comment by Michael Nitschinger [ 18/Aug/14 ]
thanks for reporting it, I'll look into it soon.
Comment by Michael Nitschinger [ 21/Aug/14 ]
Found the issue, from the client the stateful environment was not properly shutdown. Should be fixed on master and is in the beta release. Let me know if it still not works, thanks!
Comment by Adam Honen [ 24/Aug/14 ]
Actually it did not work.
I tried with the beta jar, but I still see those Netty IO (non daemon) threads hanging around.

I tried also adding a call to bucket.close(), but that didn't help either.

Comment by Adam Honen [ 24/Aug/14 ]
Attached is the TRACE log that is emitted after I call the following two lines of code:

bucket.close();
cluster.disconnect();
Comment by Michael Nitschinger [ 27/Aug/14 ]
Ah, are you passing in a custom environment? Your demo code doesn't show the init path. If so, you need to shutdown() it manually. We can't do that on cluster.disconnect() because it could be shared across more cluster objects.
Comment by Michael Nitschinger [ 27/Aug/14 ]
Note that I made a change for beta2 so that 1) the owned pools are properly named (prefixed with cb) and are all daemonized, so even if the user doesn't call shutdown as he should at least the JVM exits.




[JCBC-500] CouchbaseClient doesn't timeout, hangs forever Created: 27/Jul/14  Updated: 26/Aug/14  Resolved: 26/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Milan Simonovic Assignee: Michael Nitschinger
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
On a single node cluster:

1. open CouchbaseClient with opTimeout = 10sec (on a different machine)
2. write 50 docs (works fine)
3. make the server drop packets for 15sec (iptables -A INPUT -p tcp --dport 200:65535 -j DROP)
4. try to write another doc -> blocks forever

I've tried different timeout values: opTimeout =2.5sec, dropping packets for 5sec, but it also never times-out.

The code to reproduce this issue is here: https://github.com/mbsimonovic/jepsen-couchbase.
A more detailed explanation in my blog post: http://milansimonovic.com/2014/07/22/smashing-couchbase/

 Comments   
Comment by Milan Simonovic [ 28/Jul/14 ]
tested with Java SDK v1.4.3
Comment by Michael Nitschinger [ 14/Aug/14 ]
Hey Milan,

I cloned your project and adjusted it to my 3 node cluster. What I got was:

/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/bin/java -Didea.launcher.port=7540 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA 13.app/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/michael/tmp/jepsen-couchbase/target/classes:/Users/michael/.m2/repository/com/couchbase/client/couchbase-client/1.4.3/couchbase-client-1.4.3.jar:/Users/michael/.m2/repository/io/netty/netty/3.5.5.Final/netty-3.5.5.Final.jar:/Users/michael/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/michael/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/michael/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar:/Users/michael/.m2/repository/net/spy/spymemcached/2.11.4/spymemcached-2.11.4.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore/4.3/httpcore-4.3.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore-nio/4.3/httpcore-nio-4.3.jar:/Users/michael/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/michael/.m2/repository/com/jcraft/jsch/0.1.51/jsch-0.1.51.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-api/1.5.10/slf4j-api-1.5.10.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-log4j12/1.5.10/slf4j-log4j12-1.5.10.jar:/Users/michael/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/michael/.m2/repository/javax/mail/mail/1.4/mail-1.4.jar:/Users/michael/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Applications/IntelliJ IDEA 13.app/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain com.milansimonovic.jepsen.CouchTimeoutTest
2014-08-14 10:19:38,106 INFO [com.milansimonovic.jepsen.RemoteConnection] - 192.168.56.101
2014-08-14 10:19:38,636 INFO [com.milansimonovic.jepsen.CouchbaseClientFactory] - CouchbaseClientFactory{192.168.56.101, default, }
2014-08-14 10:19:38,783 INFO [com.couchbase.client.vbucket.provider.BucketConfigurationProvider] - Could bootstrap through carrier publication.
2014-08-14 10:19:38,800 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,800 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.102:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,801 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.103:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 10:19:38,806 INFO [com.couchbase.client.CouchbaseClient] - CouchbaseConnectionFactory{bucket='default', nodes=[http://192.168.56.101:8091/pools], order=RANDOM, opTimeout=10000, opQueue=16384, opQueueBlockTime=10000, obsPollInt=10, obsPollMax=1000, obsTimeout=10000, viewConns=10, viewTimeout=75000, viewWorkers=1, configCheck=10, reconnectInt=1100, failureMode=Redistribute, hashAlgo=NATIVE_HASH, authWaitTime=2500}
2014-08-14 10:19:38,849 INFO [com.couchbase.client.CouchbaseClient] - viewmode property isn't defined. Setting viewmode to production mode
2014-08-14 10:19:38,849 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 10:19:39,869 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:39,870 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 10:19:39,919 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 1 in 43
2014-08-14 10:19:39,922 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 2 in 3
2014-08-14 10:19:39,923 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 3 in 1
2014-08-14 10:19:39,925 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 4 in 2
2014-08-14 10:19:39,927 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 5 in 2
2014-08-14 10:19:39,928 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 6 in 1
2014-08-14 10:19:39,930 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 7 in 2
2014-08-14 10:19:39,931 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 8 in 1
2014-08-14 10:19:39,933 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 9 in 2
2014-08-14 10:19:39,934 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 10 in 1
2014-08-14 10:19:39,936 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 11 in 2
2014-08-14 10:19:39,938 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 12 in 2
2014-08-14 10:19:39,939 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 13 in 1
2014-08-14 10:19:39,941 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 14 in 2
2014-08-14 10:19:39,943 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 15 in 2
2014-08-14 10:19:39,945 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 16 in 2
2014-08-14 10:19:39,946 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 17 in 1
2014-08-14 10:19:39,948 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 18 in 2
2014-08-14 10:19:39,949 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 19 in 1
2014-08-14 10:19:39,953 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 20 in 2
2014-08-14 10:19:39,955 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 21 in 2
2014-08-14 10:19:39,956 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 22 in 1
2014-08-14 10:19:39,958 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 23 in 2
2014-08-14 10:19:39,959 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 24 in 1
2014-08-14 10:19:39,961 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 25 in 2
2014-08-14 10:19:39,963 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 26 in 2
2014-08-14 10:19:39,964 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 27 in 1
2014-08-14 10:19:39,966 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 28 in 2
2014-08-14 10:19:39,968 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 29 in 2
2014-08-14 10:19:39,969 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 30 in 1
2014-08-14 10:19:39,971 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 31 in 2
2014-08-14 10:19:39,972 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 32 in 1
2014-08-14 10:19:39,974 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 33 in 2
2014-08-14 10:19:39,976 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 34 in 2
2014-08-14 10:19:39,977 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 35 in 1
2014-08-14 10:19:39,979 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 36 in 2
2014-08-14 10:19:39,981 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 37 in 2
2014-08-14 10:19:39,982 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 38 in 1
2014-08-14 10:19:39,984 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 39 in 2
2014-08-14 10:19:39,986 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 40 in 2
2014-08-14 10:19:39,988 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 41 in 1
2014-08-14 10:19:39,989 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 42 in 1
2014-08-14 10:19:39,991 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 43 in 1
2014-08-14 10:19:39,992 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 44 in 1
2014-08-14 10:19:39,993 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 45 in 1
2014-08-14 10:19:39,995 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 46 in 2
2014-08-14 10:19:39,996 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 47 in 1
2014-08-14 10:19:39,998 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 48 in 2
2014-08-14 10:19:39,999 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 49 in 1
2014-08-14 10:19:40,001 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 50 in 2
2014-08-14 10:19:40,001 INFO [com.milansimonovic.jepsen.Cluster] - creating 1 way partition
2014-08-14 10:19:41,003 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:41,003 INFO [com.milansimonovic.jepsen.Cluster] - creating partition DONE
2014-08-14 10:19:41,004 INFO [com.milansimonovic.jepsen.Cluster] - scheduled heal in 15 seconds
2014-08-14 10:19:56,005 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 10:19:57,007 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 10:19:57,008 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 10:19:57,008 INFO [com.milansimonovic.jepsen.Cluster] - closing connections

and the doc on the server looked like:

{
  "elements": [
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    11,
    12,
    13,
    14,
    15,
    16,
    17,
    18,
    19,
    20,
    21,
    22,
    23,
    24,
    25,
    26,
    27,
    28,
    29,
    30,
    31,
    32,
    33,
    34,
    35,
    36,
    37,
    38,
    39,
    40,
    41,
    42,
    43,
    44,
    45,
    46,
    47,
    48,
    49,
    50
  ]
}

I wonder what I did different? Does it happen for you only in the single-node case? Or does running the test just not cover the bug?
Comment by Michael Nitschinger [ 14/Aug/14 ]
Ah now I get it, there should be much more appended than just 50.
Comment by Michael Nitschinger [ 14/Aug/14 ]
Ah I fixed your code. Let me do a PR for your github project.

You need to move the gets into the try/catch block as well since it can also throw a timeout (and will if there is a partition):

                CASValue<Object> casValue = client.gets(key);
                String updatedValue = appendElement(casValue.getValue().toString(), nextVal);

When I move it inside I get:

/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:61775,suspend=y,server=n -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/lib/tools.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_05.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Users/michael/tmp/jepsen-couchbase/target/classes:/Users/michael/.m2/repository/com/couchbase/client/couchbase-client/1.4.3/couchbase-client-1.4.3.jar:/Users/michael/.m2/repository/io/netty/netty/3.5.5.Final/netty-3.5.5.Final.jar:/Users/michael/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/michael/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/michael/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar:/Users/michael/.m2/repository/net/spy/spymemcached/2.11.4/spymemcached-2.11.4.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore/4.3/httpcore-4.3.jar:/Users/michael/.m2/repository/org/apache/httpcomponents/httpcore-nio/4.3/httpcore-nio-4.3.jar:/Users/michael/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/michael/.m2/repository/com/jcraft/jsch/0.1.51/jsch-0.1.51.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-api/1.5.10/slf4j-api-1.5.10.jar:/Users/michael/.m2/repository/org/slf4j/slf4j-log4j12/1.5.10/slf4j-log4j12-1.5.10.jar:/Users/michael/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/michael/.m2/repository/javax/mail/mail/1.4/mail-1.4.jar:/Users/michael/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Applications/IntelliJ IDEA 13.app/lib/idea_rt.jar" com.milansimonovic.jepsen.CouchTimeoutTest
Connected to the target VM, address: '127.0.0.1:61775', transport: 'socket'
2014-08-14 11:02:47,408 INFO [com.milansimonovic.jepsen.RemoteConnection] - 192.168.56.101
2014-08-14 11:02:47,771 INFO [com.milansimonovic.jepsen.CouchbaseClientFactory] - CouchbaseClientFactory{192.168.56.101, default, }
2014-08-14 11:02:47,908 INFO [com.couchbase.client.vbucket.provider.BucketConfigurationProvider] - Could bootstrap through carrier publication.
2014-08-14 11:02:47,920 INFO [com.couchbase.client.CouchbaseConnection] - Added {QA sa=/192.168.56.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2014-08-14 11:02:47,923 INFO [com.couchbase.client.CouchbaseClient] - CouchbaseConnectionFactory{bucket='default', nodes=[http://192.168.56.101:8091/pools], order=RANDOM, opTimeout=10000, opQueue=16384, opQueueBlockTime=10000, obsPollInt=10, obsPollMax=1000, obsTimeout=10000, viewConns=10, viewTimeout=75000, viewWorkers=1, configCheck=10, reconnectInt=1100, failureMode=Redistribute, hashAlgo=NATIVE_HASH, authWaitTime=2500}
2014-08-14 11:02:47,956 INFO [com.couchbase.client.CouchbaseClient] - viewmode property isn't defined. Setting viewmode to production mode
2014-08-14 11:02:47,956 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 11:02:48,970 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:02:48,970 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
counting down 943596453
counting down 99276090
2014-08-14 11:02:49,042 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 1 in 61
counting down 1076537872
2014-08-14 11:02:49,048 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 2 in 6
2014-08-14 11:02:49,054 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 3 in 6
counting down 1413217636
2014-08-14 11:02:49,060 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 4 in 6
counting down 491904845
2014-08-14 11:02:49,066 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 5 in 6
counting down 1312922328
2014-08-14 11:02:49,072 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 6 in 6
counting down 1529791274
2014-08-14 11:02:49,078 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 7 in 6
counting down 1349895670
2014-08-14 11:02:49,083 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 8 in 5
counting down 1129169666
2014-08-14 11:02:49,089 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 9 in 6
counting down 26740457
counting down 746758755
2014-08-14 11:02:49,094 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 10 in 5
counting down 1386645616
2014-08-14 11:02:49,100 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 11 in 5
2014-08-14 11:02:49,106 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 12 in 6
counting down 2145256693
2014-08-14 11:02:49,111 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 13 in 5
counting down 1051781801
2014-08-14 11:02:49,117 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 14 in 6
counting down 119068326
2014-08-14 11:02:49,123 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 15 in 6
counting down 1637559101
2014-08-14 11:02:49,128 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 16 in 5
counting down 1861050967
2014-08-14 11:02:49,134 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 17 in 6
counting down 1462839425
2014-08-14 11:02:49,140 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 18 in 6
counting down 383609127
2014-08-14 11:02:49,147 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 19 in 6
counting down 774334978
2014-08-14 11:02:49,156 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 20 in 8
counting down 735922763
2014-08-14 11:02:49,162 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 21 in 6
counting down 1970674321
2014-08-14 11:02:49,170 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 22 in 7
counting down 136298825
2014-08-14 11:02:49,177 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 23 in 7
counting down 494783794
2014-08-14 11:02:49,185 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 24 in 8
counting down 1546867652
2014-08-14 11:02:49,192 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 25 in 7
counting down 433354817
2014-08-14 11:02:49,199 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 26 in 7
counting down 1865938875
2014-08-14 11:02:49,207 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 27 in 8
counting down 483439410
2014-08-14 11:02:49,214 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 28 in 7
counting down 1513848351
2014-08-14 11:02:49,221 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 29 in 7
counting down 1612120937
2014-08-14 11:02:49,228 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 30 in 7
counting down 810400511
2014-08-14 11:02:49,235 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 31 in 7
counting down 689821415
2014-08-14 11:02:49,242 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 32 in 7
counting down 685219421
2014-08-14 11:02:49,248 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 33 in 6
counting down 1887689829
counting down 759433928
2014-08-14 11:02:49,254 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 34 in 6
2014-08-14 11:02:49,259 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 35 in 5
counting down 1781091177
2014-08-14 11:02:49,266 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 36 in 7
counting down 2123891975
2014-08-14 11:02:49,271 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 37 in 5
counting down 278306373
2014-08-14 11:02:49,278 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 38 in 6
counting down 495568085
2014-08-14 11:02:49,284 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 39 in 6
counting down 1029420558
2014-08-14 11:02:49,290 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 40 in 5
counting down 55913631
2014-08-14 11:02:49,296 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 41 in 6
counting down 664141318
2014-08-14 11:02:49,302 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 42 in 6
counting down 1953048909
2014-08-14 11:02:49,307 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 43 in 5
counting down 350362120
counting down 1048178143
2014-08-14 11:02:49,313 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 44 in 6
2014-08-14 11:02:49,318 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 45 in 5
counting down 907297240
2014-08-14 11:02:49,323 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 46 in 5
counting down 193602838
2014-08-14 11:02:49,329 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 47 in 6
counting down 255614005
2014-08-14 11:02:49,335 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 48 in 6
counting down 2128240408
2014-08-14 11:02:49,340 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 49 in 5
counting down 1783946138
2014-08-14 11:02:49,346 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 50 in 6
2014-08-14 11:02:49,347 INFO [com.milansimonovic.jepsen.Cluster] - creating 1 way partition
2014-08-14 11:02:50,348 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:02:50,348 INFO [com.milansimonovic.jepsen.Cluster] - creating partition DONE
2014-08-14 11:02:50,349 INFO [com.milansimonovic.jepsen.Cluster] - scheduled heal in 15 seconds
counting down 1330247343
2014-08-14 11:03:03,213 ERROR [com.milansimonovic.jepsen.CouchTimeoutTest] - timed out adding 51
2014-08-14 11:03:03,214 ERROR [com.milansimonovic.jepsen.CouchTimeoutTest] - error adding 51
net.spy.memcached.OperationTimeoutException: Timeout waiting for value
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1147)
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1211)
at com.milansimonovic.jepsen.CouchTimeoutTest.addElementToArray(CouchTimeoutTest.java:65)
at com.milansimonovic.jepsen.CouchTimeoutTest.appendToDocument(CouchTimeoutTest.java:102)
at com.milansimonovic.jepsen.CouchTimeoutTest.run(CouchTimeoutTest.java:49)
at com.milansimonovic.jepsen.CouchTimeoutTest.main(CouchTimeoutTest.java:39)
Caused by: net.spy.memcached.internal.CheckedOperationTimeoutException: Timed out waiting for operation - failing node: 192.168.56.101/192.168.56.101:11210
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:167)
at net.spy.memcached.MemcachedClient.gets(MemcachedClient.java:1137)
... 5 more
2014-08-14 11:03:05,349 INFO [com.milansimonovic.jepsen.Cluster] - healing partition
2014-08-14 11:03:06,351 DEBUG [com.milansimonovic.jepsen.RemoteConnection] - exit-status: 0
2014-08-14 11:03:06,351 INFO [com.milansimonovic.jepsen.Cluster] - healing partition - DONE
2014-08-14 11:03:06,351 INFO [com.milansimonovic.jepsen.Cluster] - closing connections
counting down 1330247343
counting down 808247635
counting down 1562173079
2014-08-14 11:03:08,230 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 52 in 5014
counting down 1897031859
2014-08-14 11:03:08,236 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 53 in 6
2014-08-14 11:03:08,241 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 54 in 5
counting down 1029104896
2014-08-14 11:03:08,247 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 55 in 6
counting down 1789373824
2014-08-14 11:03:08,252 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 56 in 5
counting down 1374957093
2014-08-14 11:03:08,259 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 57 in 7
counting down 859935576
2014-08-14 11:03:08,265 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 58 in 5
counting down 959778517
2014-08-14 11:03:08,270 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 59 in 5
counting down 1856597982
2014-08-14 11:03:08,276 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 60 in 6
counting down 509403432
2014-08-14 11:03:08,283 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 61 in 7
counting down 565734120
2014-08-14 11:03:08,289 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 62 in 6
counting down 812288940
2014-08-14 11:03:08,295 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 63 in 6
counting down 362337760
2014-08-14 11:03:08,301 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 64 in 6
counting down 1815360474
counting down 238680990
2014-08-14 11:03:08,307 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 65 in 6
2014-08-14 11:03:08,312 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 66 in 5
counting down 1793734174
2014-08-14 11:03:08,318 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 67 in 6
counting down 965663094
2014-08-14 11:03:08,323 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 68 in 5
counting down 1961465726
2014-08-14 11:03:08,329 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 69 in 6
counting down 894265206
counting down 237825287
2014-08-14 11:03:08,334 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 70 in 5
2014-08-14 11:03:08,340 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 71 in 6
counting down 1093062877
2014-08-14 11:03:08,346 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 72 in 6
counting down 2043477472
2014-08-14 11:03:08,352 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 73 in 6
counting down 660886392
2014-08-14 11:03:08,358 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 74 in 6
counting down 1321850438
2014-08-14 11:03:08,363 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 75 in 5
counting down 966674298
2014-08-14 11:03:08,368 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 76 in 5
counting down 665030228
counting down 584709707
2014-08-14 11:03:08,373 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 77 in 5
2014-08-14 11:03:08,379 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 78 in 6
counting down 581070768
2014-08-14 11:03:08,384 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 79 in 5
counting down 1766611209
2014-08-14 11:03:08,390 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 80 in 6
counting down 1378265946
2014-08-14 11:03:08,395 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 81 in 5
counting down 1067480315
2014-08-14 11:03:08,400 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 82 in 5
counting down 240010064
2014-08-14 11:03:08,406 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 83 in 6
counting down 96589697
counting down 1650118680
2014-08-14 11:03:08,411 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 84 in 5
2014-08-14 11:03:08,417 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 85 in 6
counting down 1583441668
2014-08-14 11:03:08,422 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 86 in 5
counting down 575269723
2014-08-14 11:03:08,427 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 87 in 5
counting down 692018508
2014-08-14 11:03:08,434 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 88 in 6
counting down 500269667
2014-08-14 11:03:08,440 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 89 in 6
counting down 1214091162
2014-08-14 11:03:08,446 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 90 in 6
counting down 631482105
2014-08-14 11:03:08,452 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 91 in 6
counting down 1982159420
2014-08-14 11:03:08,458 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 92 in 6
counting down 1744182273
counting down 1131642426
2014-08-14 11:03:08,463 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 93 in 5
2014-08-14 11:03:08,469 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 94 in 6
counting down 2069859523
2014-08-14 11:03:08,475 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 95 in 6
counting down 662166830
2014-08-14 11:03:08,481 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 96 in 6
counting down 1882643413
2014-08-14 11:03:08,487 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 97 in 6
counting down 678230343
2014-08-14 11:03:08,493 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 98 in 6
counting down 793919773
2014-08-14 11:03:08,498 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 99 in 5
counting down 911566708
2014-08-14 11:03:08,505 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 100 in 7
counting down 1887173144
2014-08-14 11:03:08,510 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 101 in 5
counting down 1058177937
2014-08-14 11:03:08,516 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 102 in 6
counting down 41252807
2014-08-14 11:03:08,523 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 103 in 6
counting down 528932410
2014-08-14 11:03:08,531 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 104 in 8
counting down 805461224
2014-08-14 11:03:08,538 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 105 in 7
counting down 463949682
2014-08-14 11:03:08,546 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 106 in 8
counting down 851822734
2014-08-14 11:03:08,554 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 107 in 8
counting down 454797282
2014-08-14 11:03:08,561 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 108 in 7
counting down 962354549
2014-08-14 11:03:08,568 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 109 in 7
counting down 413223411
2014-08-14 11:03:08,574 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 110 in 6
counting down 79455185
2014-08-14 11:03:08,580 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 111 in 6
counting down 1113518099
2014-08-14 11:03:08,586 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 112 in 6
counting down 431139544
2014-08-14 11:03:08,592 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 113 in 6
counting down 198084300
2014-08-14 11:03:08,599 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 114 in 7
counting down 1821101711
2014-08-14 11:03:08,605 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 115 in 6
counting down 1499489637
2014-08-14 11:03:08,611 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 116 in 6
counting down 1435037947
2014-08-14 11:03:08,618 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 117 in 7
counting down 705688672
2014-08-14 11:03:08,624 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 118 in 6
counting down 407021592
2014-08-14 11:03:08,630 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 119 in 6
counting down 1121168696
2014-08-14 11:03:08,637 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 120 in 7
counting down 1347326641
counting down 87539756
2014-08-14 11:03:08,642 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 121 in 5
2014-08-14 11:03:08,647 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 122 in 5
counting down 137935610
2014-08-14 11:03:08,653 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 123 in 6
counting down 1474133160
2014-08-14 11:03:08,659 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 124 in 6
counting down 1931651064
2014-08-14 11:03:08,665 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 125 in 6
counting down 1211051792
2014-08-14 11:03:08,688 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 126 in 23
counting down 93172613
2014-08-14 11:03:08,693 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 127 in 5
counting down 965322656
2014-08-14 11:03:08,699 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 128 in 6
counting down 1601358350
2014-08-14 11:03:08,705 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 129 in 6
counting down 1855207775
2014-08-14 11:03:08,710 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 130 in 5
counting down 107851411
2014-08-14 11:03:08,716 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 131 in 6
counting down 1915600217
2014-08-14 11:03:08,721 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 132 in 5
counting down 1002847144
2014-08-14 11:03:08,727 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 133 in 6
counting down 1960289102
2014-08-14 11:03:08,732 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 134 in 5
counting down 530940529
2014-08-14 11:03:08,738 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 135 in 6
counting down 16593833
2014-08-14 11:03:08,744 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 136 in 6
counting down 183255909
2014-08-14 11:03:08,749 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 137 in 5
counting down 1139297445
counting down 1886255267
2014-08-14 11:03:08,755 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 138 in 5
2014-08-14 11:03:08,760 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 139 in 5
counting down 426044796
2014-08-14 11:03:08,765 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 140 in 5
counting down 1903915614
2014-08-14 11:03:08,770 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 141 in 5
counting down 1895269808
2014-08-14 11:03:08,776 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 142 in 6
counting down 1866519533
counting down 1366412404
2014-08-14 11:03:08,781 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 143 in 5
2014-08-14 11:03:08,786 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 144 in 5
counting down 2075311494
2014-08-14 11:03:08,792 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 145 in 6
counting down 925138245
2014-08-14 11:03:08,797 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 146 in 5
counting down 1578242039
2014-08-14 11:03:08,802 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 147 in 5
counting down 23034413
2014-08-14 11:03:08,808 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 148 in 6
counting down 1869588749
counting down 2067087344
2014-08-14 11:03:08,814 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 149 in 6
2014-08-14 11:03:08,820 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 150 in 6
counting down 899087062
2014-08-14 11:03:08,826 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 151 in 6
counting down 1340235867
2014-08-14 11:03:08,832 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 152 in 6
counting down 1574412139
2014-08-14 11:03:08,838 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 153 in 6
counting down 256897438
2014-08-14 11:03:08,844 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 154 in 6
counting down 227404523
2014-08-14 11:03:08,849 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 155 in 5
counting down 1370600377
2014-08-14 11:03:08,855 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 156 in 5
counting down 821838371
2014-08-14 11:03:08,861 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 157 in 6
counting down 1087346339
counting down 578287331
2014-08-14 11:03:08,866 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 158 in 5
2014-08-14 11:03:08,872 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 159 in 6
counting down 2114430319
2014-08-14 11:03:08,877 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 160 in 5
counting down 750263766
2014-08-14 11:03:08,883 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 161 in 6
counting down 285142124
2014-08-14 11:03:08,889 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 162 in 6
counting down 228835454
2014-08-14 11:03:08,895 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 163 in 6
counting down 1322000504
2014-08-14 11:03:08,900 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 164 in 5
counting down 578208474
2014-08-14 11:03:08,906 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 165 in 6
counting down 1158469492
counting down 1446993819
2014-08-14 11:03:08,911 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 166 in 5
2014-08-14 11:03:08,917 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 167 in 6
counting down 2036300484
2014-08-14 11:03:08,923 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 168 in 5
counting down 1839929127
2014-08-14 11:03:08,928 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 169 in 5
counting down 1321196177
2014-08-14 11:03:08,934 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 170 in 6
counting down 1657833306
2014-08-14 11:03:08,941 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 171 in 7
counting down 1616599198
2014-08-14 11:03:08,948 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 172 in 7
counting down 1434216861
2014-08-14 11:03:08,955 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 173 in 7
counting down 1697078001
2014-08-14 11:03:08,962 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 174 in 7
counting down 53209944
2014-08-14 11:03:08,969 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 175 in 7
counting down 957271051
2014-08-14 11:03:08,983 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 176 in 14
counting down 260602422
2014-08-14 11:03:08,991 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 177 in 8
counting down 1344752100
2014-08-14 11:03:08,996 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 178 in 5
counting down 210373011
2014-08-14 11:03:09,003 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 179 in 7
counting down 1135342237
counting down 255748372
2014-08-14 11:03:09,008 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 180 in 5
2014-08-14 11:03:09,013 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 181 in 5
counting down 1706850686
2014-08-14 11:03:09,018 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 182 in 5
counting down 958800587
2014-08-14 11:03:09,023 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 183 in 5
counting down 621084776
counting down 1494203157
2014-08-14 11:03:09,028 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 184 in 5
2014-08-14 11:03:09,034 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 185 in 6
counting down 1610501827
2014-08-14 11:03:09,039 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 186 in 5
counting down 1472601997
2014-08-14 11:03:09,046 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 187 in 6
counting down 1523401565
2014-08-14 11:03:09,052 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 188 in 6
counting down 2118032386
2014-08-14 11:03:09,058 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 189 in 6
counting down 1402690896
2014-08-14 11:03:09,063 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 190 in 5
counting down 877005529
counting down 69300391
2014-08-14 11:03:09,068 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 191 in 5
counting down 250950317
2014-08-14 11:03:09,074 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 192 in 6
2014-08-14 11:03:09,080 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 193 in 6
counting down 126717950
2014-08-14 11:03:09,086 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 194 in 6
counting down 60378388
2014-08-14 11:03:09,092 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 195 in 6
counting down 213037448
2014-08-14 11:03:09,098 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 196 in 6
counting down 737512169
2014-08-14 11:03:09,103 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 197 in 5
counting down 1089208854
2014-08-14 11:03:09,109 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 198 in 6
counting down 156501746
2014-08-14 11:03:09,115 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 199 in 6
counting down 284416161
2014-08-14 11:03:09,121 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - added 200 in 6
2014-08-14 11:03:09,123 INFO [com.couchbase.client.CouchbaseConnection] - Shut down Couchbase client
2014-08-14 11:03:09,125 INFO [com.couchbase.client.ViewConnection] - I/O reactor terminated
2014-08-14 11:03:09,125 INFO [com.milansimonovic.jepsen.CouchTimeoutTest] - DONE!
Disconnected from the target VM, address: '127.0.0.1:61775', transport: 'socket'

Process finished with exit code 0

Comment by Michael Nitschinger [ 14/Aug/14 ]
https://github.com/mbsimonovic/jepsen-couchbase/pull/2
Comment by Matt Ingenthron [ 18/Aug/14 ]
Milan: would be great if you could update your blog with a paragraph or so at the top clarifying that it did work so people don't have to read into the comments to find that it was a problem with the test.
Comment by Milan Simonovic [ 19/Aug/14 ]
Sure, as soon as I get back from holidays (next we latest). Ps the tracker is major pain in the a** to use on a smartphone..
Comment by Milan Simonovic [ 26/Aug/14 ]
updated, feel free to close the issue
Comment by Michael Nitschinger [ 26/Aug/14 ]
Not an issue.




[JCBC-531] Expose Diagnostics and debug dump them on startup Created: 26/Aug/14  Updated: 26/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.5
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-521] VBucketNodeLocator.java getPrimary() and getReplica() log messages are homologous Created: 21/Aug/14  Updated: 25/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Improvement Priority: Minor
Reporter: Mark Woosey Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate

 Description   
VBucketNodeLocator.java has these two methods:
- getPrimary(String k)
- getReplica(String key, int index)

These two write almost the same warning log message when it can't access a node which holds the key.

getPrimary() logs: "for which no server is responsible in the cluster map (-1)."
while
getReplica() logs: "for which no server is responsible in the cluster map."
(without -1 in the message)

 Comments   
Comment by Koji Kawamura [ 21/Aug/14 ]
These two methods log almost the same warning messages, but since those two are indicating different level of severity, those better to be differentiated.
Comment by Michael Nitschinger [ 25/Aug/14 ]
I think it would be okay to push the replica one down to INFO level.Would that be sufficient? I'd rather not change the message itself in a bugfix release.
Comment by Koji Kawamura [ 25/Aug/14 ]
Changing log level of missing replica node to INFO sounds reasonable. Thank you!




[JCBC-530] Exception when using java client both in Spark and MapReduce Created: 25/Aug/14  Updated: 25/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Critical
Reporter: Sun Chen Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: Hadoop,Dependencies,HttpCore, Spark,
Remaining Estimate: 40h
Time Spent: Not Specified
Original Estimate: 40h
Environment: Haddop2.4.1,Spark0.13,Java1.7,Ubuntu12.04


 Description   
I am using Couchbase java client(version 1.4.4),I writed some simple mapreaduce job and spark job to write and read data from Couchbase.But I always got an exception.Here is the exception information

-------------------------------------------------------------------------------------------------------------------------------------------------
2014-08-25 13:12:56,048 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: org.apache.http.protocol.RequestUserAgent.<init>(Ljava/lang/String;)V
at com.couchbase.client.ViewConnection.<init>(ViewConnection.java:156)
at com.couchbase.client.CouchbaseConnectionFactory.createViewConnection(CouchbaseConnectionFactory.java:254)
at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:266)
at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:194)
at com.eyespage.dc.etl.CouchbaseImportExtractor.extract(CouchbaseImportExtractor.java:31)
at com.eyespage.dc.etl.CouchbaseImportExtractor.extract(CouchbaseImportExtractor.java:20)
at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:96)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
-------------------------------------------------------------------------------------------------------------------------------------------

14/08/09 10:19:32 WARN Utils: Your hostname, precise64 resolves to a loopback address: 127.0.1.1; using 192.168.33.10 instead (on interface eth1) 14/08/09 10:19:32 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address 14/08/09 10:19:38 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2014-08-09 10:19:40.569 INFO net.spy.memcached.auth.AuthThread: Authenticated to /127.0.0.1:11210 2014-08-09 10:19:40.677 INFO com.couchbase.client.vbucket.provider.BucketConfigurationProvider: Could bootstrap through carrier publication. 2014-08-09 10:19:40.693 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=localhost/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue 2014-08-09 10:19:40.702 INFO com.couchbase.client.CouchbaseClient: CouchbaseConnectionFactory{bucket='Result', nodes=[http://127.0.0.1:8091/pools], order=RANDOM, opTimeout=2500, opQueue=16384, opQueueBlockTime=10000, obsPollInt=10, obsPollMax=500, obsTimeout=5000, viewConns=10, viewTimeout=75000, viewWorkers=1, configCheck=10, reconnectInt=1100, failureMode=Redistribute, hashAlgo=NATIVE_HASH, authWaitTime=2500} Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.protocol.RequestUserAgent.<init>(Ljava/lang/String;)V at com.couchbase.client.ViewConnection.<init>(ViewConnection.java:156) at com.couchbase.client.CouchbaseConnectionFactory.createViewConnection(CouchbaseConnectionFactory.java:254) at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:266) at com.couchbase.client.CouchbaseClient.<init>(CouchbaseClient.java:194) at com.eyespage.dc.etl.c2s.Run$.main(main.scala:16) at com.eyespage.dc.etl.c2s.Run.main(main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:303) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 2014-08-09 10:19:40.836 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210 - See more at: http://www.couchbase.com/communities/q-and-a/exception-when-using-scala-call-java-sdk%EF%BC%8Cand-run-spark#sthash.CmL5dcRz.dpuf








[JCBC-485] JsonObject and all its similars Created: 06/Jul/14  Updated: 24/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0.0
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Oded Hassidi Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: json
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Why should I work with JsonObject, JsonArray and all of that? Why can't I just send a String and you guys do with it what ever you want? Or at least give us a utility to convert our Pojo to your JsonObject?

If there is such utility I take it back :)

 Comments   
Comment by Michael Nitschinger [ 07/Jul/14 ]
Hi,

yes you are right, and we will provide that functionality as well once we get to feature completeness on 2.0. There has already been a user contribution to a converter which will act like the one on 1.4 so you can then pass in a string and it will be stored. So you get the both of two worlds.

Does that work for you?
Comment by Michael Nitschinger [ 21/Aug/14 ]
We have implemented a LegacyDocument where you can pass in a string and it works like the old one. We probably add more documents (string,...) directly so you can have more fun with it.

I'll close this one for now since it works with beta/dp3 already using the legacy codec and open more issues for specific transcoders/documents.
Comment by Oded Hassidi [ 24/Aug/14 ]
Thanks




[JCBC-526] Add support for bucket management operations Created: 22/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-527] Add cluster info Created: 22/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-524] Add support for custom transcoders Created: 22/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-513] Implement bucket.close() Created: 18/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-522] Implement bucket info. Created: 22/Aug/14  Updated: 22/Aug/14  Resolved: 22/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-479] ComparisonFailure in running unit tests for 2.0 client Created: 24/Jun/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Infrastructure
Affects Version/s: 2.0.0
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Deepti Dawar Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency

 Description   
1 warning
:processTestResources UP-TO-DATE
:testClasses
:test

com.couchbase.client.java.document.json.JsonObjectTest > shouldExportMix FAILED
    org.junit.ComparisonFailure at JsonObjectTest.java:27

com.couchbase.client.java.query.ViewQueryTest > shouldHandleEndKey FAILED
    org.junit.ComparisonFailure at ViewQueryTest.java:178

77 tests completed, 2 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///root/couchbase/sdkd-2.0/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED


 Comments   
Comment by Wei-Li Liu [ 15/Jul/14 ]
I am getting different failure running unit test

===================================================================================
1 warning
:processTestResources UP-TO-DATE
:testClasses
:test

com.couchbase.client.java.convert.JacksonJsonConverterTest > shouldEncodeSimpleJsonObject FAILED
    org.junit.ComparisonFailure at JacksonJsonConverterTest.java:70

com.couchbase.client.java.convert.JacksonJsonConverterTest > shouldEncodeMixedNestedJsonValues FAILED
    org.junit.ComparisonFailure at JacksonJsonConverterTest.java:177

77 tests completed, 2 failed
:test FAILED

FAILURE: Build failed with an exception.
Comment by Wei-Li Liu [ 15/Jul/14 ]
Failure running integration tests


============================================================================
wei-lis-mbp:couchbase-java-client wei-li$ ./gradlew integrationTest
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileIntegrationJava UP-TO-DATE
:processIntegrationResources UP-TO-DATE
:integrationClasses UP-TO-DATE
:integrationTest

com.couchbase.client.java.BinaryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.BinaryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.QueryTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.QueryTest > classMethod FAILED
    java.util.NoSuchElementException

com.couchbase.client.java.ViewTest > classMethod FAILED
    com.couchbase.client.core.config.ConfigurationException
        Caused by: java.lang.IllegalStateException
            Caused by: rx.exceptions.OnErrorThrowable$OnNextValue

com.couchbase.client.java.ViewTest > classMethod FAILED
    java.util.NoSuchElementException

6 tests completed, 6 failed
:integrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':integrationTest'.
> There were failing tests. See the report at: file:///Users/wei-li/repo/couchbase/sdkd-2.0/couchbase-java-client/build/reports/tests/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED
Comment by Wei-Li Liu [ 15/Jul/14 ]
Disregard the above integration test failure. It requires to create "default" bucket and set no password to run.
Comment by Wei-Li Liu [ 18/Aug/14 ]
Running 2.0-dp3 feature test against 1118 server, and all of the unit test and integration tests passed. I am resolving this ticket.
Comment by Deepti Dawar [ 19/Aug/14 ]
Hi Michael, Please close this when you push the fix for it.

Thanks !
Comment by Michael Nitschinger [ 19/Aug/14 ]
Should be fixed in master, please reopen if not.
Comment by Wei-Li Liu [ 20/Aug/14 ]
I run int test on master today and there is 1 failure on shouldUpsertLegacyObjectDocument

com.couchbase.client.java.BinaryTest STANDARD_ERROR
    log4j:WARN No appenders could be found for logger (com.couchbase.client.core.logging.CouchbaseLoggerFactory).
    log4j:WARN Please initialize the log4j system properly.
    log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument FAILED
    java.lang.NullPointerException
        at com.couchbase.client.java.BinaryTest.shouldUpsertLegacyObjectDocument(BinaryTest.java:295)
Gradle Worker 1 finished executing tests.

28 tests completed, 1 failed
Comment by Michael Nitschinger [ 21/Aug/14 ]
Hm there might be an issue with the new legacy document, a different test is failing for me.

Is there any other info in the logs what's going on?
Comment by Deepti Dawar [ 21/Aug/14 ]
The unit tests now run fine.
Tried with the latest of DP3.
Comment by Wei-Li Liu [ 21/Aug/14 ]
stacktrace for legacy document test

11:40:54.879 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument STARTED
11:40:54.918 [DEBUG] [TestEventLogger]
11:40:54.918 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldUpsertLegacyObjectDocument FAILED
11:40:54.918 [DEBUG] [TestEventLogger] java.lang.NullPointerException
11:40:54.918 [DEBUG] [TestEventLogger] at com.couchbase.client.java.BinaryTest.shouldUpsertLegacyObjectDocument(BinaryTest.java:295)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.919 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.919 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
11:40:54.919 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
11:40:54.920 [DEBUG] [TestEventLogger] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
11:40:54.921 [DEBUG] [TestEventLogger] at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:86)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69)
11:40:54.921 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.921 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.922 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
11:40:54.922 [DEBUG] [TestEventLogger] at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
11:40:54.922 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:105)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
11:40:54.922 [DEBUG] [TestEventLogger] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
11:40:54.923 [DEBUG] [TestEventLogger] at java.lang.reflect.Method.invoke(Method.java:483)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:355)
11:40:54.923 [DEBUG] [TestEventLogger] at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
11:40:54.923 [DEBUG] [TestEventLogger] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
11:40:54.923 [DEBUG] [TestEventLogger] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
11:40:54.923 [DEBUG] [TestEventLogger] at java.lang.Thread.run(Thread.java:745)
11:40:54.925 [DEBUG] [TestEventLogger]
11:40:54.925 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldInsertAndGet STARTED
11:40:54.925 [DEBUG] [TestEventLogger]
11:40:54.925 [DEBUG] [TestEventLogger] com.couchbase.client.java.BinaryTest > shouldInsertAndGet PASSED
11:40:57.110 [QUIET] [system.out] 11:40:57.110 [
11:40:57.110 [DEBUG] [TestEventLogger]
11:40:57.110 [QUIET] [system.out] DEBUG] [org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor] Executing test class com.couchbase.client.java.DesignDocumentTest
Comment by Wei-Li Liu [ 21/Aug/14 ]
This exception is caused by integration test, which is somewhat a duplicate of JCBC-481.
unit test works fine for me too.




[JCBC-499] Test SSL Feature with Java 2.0 SDK Created: 28/Jul/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Closed
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 2.0-dp2
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Test Blocker
Reporter: Wei-Li Liu Assignee: Michael Nitschinger
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File ssl.pcapng    

 Description   
Michael, here is the SSL Feature Test I created
https://github.com/couchbase/couchbase-java-client/pull/6/files
I know there are suff should be in core, and we can move it later

There are a couple issues in this feature tests

1. view and query test hangs
2. the ssl test I created is not able to catch exceptions such as sun.security.validator.ValidatorException, javax.net.ssl.SSLHandshakeException. etc

We can’t disable SSL from the server side after starting the server. SSL certificate automatically generated and self signed, so we can’t modify expired date.

I also test 2.5 server connect with java client, which does not work with SSL.

Attached file is wireshark packet capture fro FeatureTest with SSL. If you decode it with SSL in Wireshark, you can see the data and TLS protocol.
  



 Comments   
Comment by Wei-Li Liu [ 30/Jul/14 ]
java sdk's commit on observe relies on core
https://github.com/weilliu/couchbase-java-client/commit/6f8d672ff9a600780fd5d7d47d5ba6ce52bd083d

import com.couchbase.client.core.message.binary.ObserveRequest;
import com.couchbase.client.core.message.binary.ObserveResponse;

Currently Core can't be built on master

====================================
[root@centos-64-x64 couchbase-jvm-core]# ./gradlew install
:compileJava
Note: /root/sdkd-2.0/couchbase-jvm-core/src/main/java/com/couchbase/client/core/CouchbaseCore.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
:processResources
:classes
:jar
:javadoc
java.lang.UnsupportedClassVersionError: ch/raffael/doclets/pegdown/PegdownDoclet : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:643)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
at com.sun.tools.javadoc.DocletInvoker.<init>(DocletInvoker.java:96)
at com.sun.tools.javadoc.Start.setDocletInvoker(Start.java:414)
at com.sun.tools.javadoc.Start.parseAndExecute(Start.java:213)
at com.sun.tools.javadoc.Start.begin(Start.java:162)
at com.sun.tools.javadoc.Main.execute(Main.java:59)
at com.sun.tools.javadoc.Main.main(Main.java:49)
javadoc: error - fatal error
1 error
:javadoc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':javadoc'.
> Javadoc generation failed.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 24.87 secs
====================================================

Comment by Wei-Li Liu [ 04/Aug/14 ]
Please disregard my above error, which is triggered by inconsistent jdk version on run time and compile time.
Comment by Don Pinto [ 11/Aug/14 ]
Wei-li, can you update this ticket?
Comment by Wei-Li Liu [ 11/Aug/14 ]
@Michael looking at your most recent commit, I don't think the catching exception issue has been fixed yet. I believe you just changed the fix version to 2.0-beta.
I also have some issues build couchbase-java-client from master after I pull both sdk and the core. I knew Deepti has the same issue earlier, but not sure if it has been resolved yet.
Comment by Matt Ingenthron [ 18/Aug/14 ]
Michael: this is blocking completion of testing, can you aim to resolve it this week.
Comment by Wei-Li Liu [ 18/Aug/14 ]
After getting some help from Deepti, I am able to get dp3 build with ./gradlew publish and ./gradlew publishToMavenLocal
Integration test has been changed in dp3, so some modifications required for SSL feature tests on my end.
Comment by Wei-Li Liu [ 20/Aug/14 ]
Report bug JCBC-520
Comment by Michael Nitschinger [ 21/Aug/14 ]
So are we tracking it in JCBC-520 and closing this?
Comment by Wei-Li Liu [ 21/Aug/14 ]
okay




[JCBC-514] Only shutdown environment if not shared Created: 18/Aug/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-498] I can't connect to an authenticated bucket Created: 28/Jul/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 2.0-dp2
Fix Version/s: 2.0-beta
Security Level: Public

Type: Task Priority: Major
Reporter: Gareth Powell Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: customer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: MacOS X

Attachments: Text File trace.txt    

 Description   
I can't connect to a password-protected bucket with SDK 2-dp2.

"Similar" code can connect with SDK 1.4.

Here is my approximate code:

cluster = new CouchbaseCluster(couchConfig.fromHost);

Bucket bucket = cluster.openBucket(couchConfig.bucket, couchConfig.passwd).toBlocking().single();

if (couchConfig.clearDB)

bucket.flush().toBlocking().single();

where "fromHost" is set to "warburton", "bucket" is ziggrid-baseball and password is "". (Yes, that is the password on the bucket in development).

 Comments   
Comment by Michael Nitschinger [ 04/Aug/14 ]
thanks, I'll look into it soon. Let's see if I can fix that for dp3
Comment by Gareth Powell [ 20/Aug/14 ]
I just tried this and in dp3 I can "kind of" get it to work.

The thing is, I couldn't use the code above (presumably due to API changes) and that suggests the code above might always have been invalid.

I changed:

cluster = new CouchbaseCluster(couchConfig.fromHost);

to

cluster = CouchbaseCluster.create(couchConfig.fromHost);

and

bucket.flush().toBlocking().single();

to

bucket.bucketManager().toBlocking().single().flush().toBlocking().single();
Comment by Michael Nitschinger [ 20/Aug/14 ]
Yup, your changes look good. What do you mean by kind of? Does it work with the changes?
Comment by Gareth Powell [ 20/Aug/14 ]
Yes, it works now. At the time I made the comment, I was able to connect and clear out the bucket, but nothing else appeared to be working.

The problem I'm having is the usual migration one: I have so many errors in my code from the API changing that it's hard to tell what is going wrong. It turns out that my line of execution was being sabotaged by a broken dependency that wasn't getting caught. When I was able to fix that, I was able to get to the stage of loading a design document.

I now have design documents and regular documents loading.
Comment by Michael Nitschinger [ 21/Aug/14 ]
Perfect, thanks. Let me know if you run into more issues.




[JCBC-516] All internal code should use the provided scheduler from the environment Created: 18/Aug/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Comments   
Comment by Michael Nitschinger [ 21/Aug/14 ]
fixed in the core.




[JCBC-497] reduce in ViewQuery doesn't work Created: 27/Jul/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Views
Affects Version/s: 2.0-dp2
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Oded Hassidi Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
When querying using views, setting the reduce(false) seems to have no influence on the query and always reduces the query.

Am I missing something?

 Comments   
Comment by Michael Nitschinger [ 28/Jul/14 ]
Thanks, I'll look into it.
Comment by Michael Nitschinger [ 21/Aug/14 ]
Works in dp3 and on master!

Make sure that if you have a view with a reduce function, set .reduce(false) on the DSL to disable it optionally.




[JCBC-512] $HOST in Refresher against localhost Created: 18/Aug/14  Updated: 21/Aug/14  Resolved: 21/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Bug Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
carrier (and maybe also http refresher) don't replace $HOST

 Comments   
Comment by Michael Nitschinger [ 21/Aug/14 ]
Done in core.




[JCBC-519] Timeout Not Honored on First Call to Java asyncGetBulk Created: 11/Aug/14  Updated: 20/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.4
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: Jeff Dillon Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Java Source File App.java    
Issue Links:
Duplicate

 Description   
This was observed on Couchbase Server 2.2.0. The same behavior is not evident on Couchbase Server 2.5.1. It appears to be independent of any Java client version. The very first call of asyncGetBulk does not respect the timeout value, and continues to process and wait. Subsequent calls do properly respect the timeout value. One interesting note is that this only occurs when the timeout value is "close" to the actual execution time. That is, a very short timeout is indeed honored.

Steps to reproduce:

* Download a Java client, such as http://packages.couchbase.com/clients/java/1.4.3/Couchbase-Java-Client-1.4.3.zip

* Unzip

* Copy the attached App.java into the unzipped directory

* Compile and run:

javac -cp couchbase-client-1.4.3.jar:spymemcached-2.11.4.jar App.java

java -cp .:couchbase-client-1.4.3.jar:spymemcached-2.11.4.jar:jettison-1.1.jar:netty-3.5.5.Final.jar:commons-codec-1.5.jar:httpcore-4.3.jar:httpcore-nio-4.3.jar App > 143.txt

* Note the following output in the first group. It would be expected that the first call would timeout, but does not:

Timeout future is: false
Fetched 72 keys
Timeout set to:16
getMulti : Time taken=108ms

Timeout future is: false
Fetched 72 keys
Timeout set to:16
getMulti : Time taken=17ms

Timeout future is: false
Fetched 72 keys
Timeout set to:16
getMulti : Time taken=18ms

Timeout future is: false
Fetched 72 keys
Timeout set to:16
getMulti : Time taken=16ms


Exception:********************
Timeout future is: true
Operation timed out. - failing nodes: 10.4.2.107/10.4.2.107:11210, cbhost.domain/10.4.2.57:11210
Timeout set to:16
getMulti : Time taken=18ms

Timeout future is: false
Fetched 72 keys
Timeout set to:16
getMulti : Time taken=17ms


ASK - Determine why the first call does not honor the timeout, and ideally suggest a workaround for Couchbase Server 2.2.0, short of upgrading to Couchbase Server 2.5.1 (where the issue does not appear exist, and therefore looks to be a server-side issue)




 Comments   
Comment by Aleksey Kondratenko [ 11/Aug/14 ]
Not honoring timeouts cannot be server side issue. Passing to Michael who owns couchbase java sdk
Comment by Jeff Dillon [ 12/Aug/14 ]
Please disregard my previous comments with respect to being apparently fixed with Couchbase Server 2.5.1. This behavior evidences itself across all CB server versions, and also reproduces with CB Java client 1.4.4. In my latest 2.5.1 test, I set the timeout to 30 ms, and see the same incorrect behavior. So this is indeed a client issue (as noted), apologies on the initial misinformation.
Comment by Michael Nitschinger [ 14/Aug/14 ]
A few remarks:

- The simple benchmark also profiles logging and so forth.
- System.currentTimeMillis() precision is sometimes beyond all joke, always use nanotime.

So something like this is more accurate:

        while (low == 10) {
            try {
                startTime = System.nanoTime();
                future = client.asyncGetBulk(Main.getKeys());
                future.get(timeout, TimeUnit.MILLISECONDS);
                endTime = System.nanoTime();
                System.out.println("Timeout future is: " + future.isTimeout());
                System.out.println("Fetched " + Main.getKeys().size() + " keys");
            } catch (Exception e) {
                System.out.println("\nException:********************\nTimeout future is: " + future.isTimeout());
                System.out.println(e.getMessage());
            }
            operationTime = endTime - startTime;
            System.out.println("Timeout set to:" + timeout + "\ngetMulti : Time taken=" + TimeUnit.NANOSECONDS.toMillis(operationTime) + "ms\n");
        }

Now also note that profiling on the blocking .get() call not only waits on the latch where the timeout is applied, but it also puts all data together and so forth. So as far as I can see it the timeout is honored, its just not what you benchmark that close. Keep in mind that the first few calls are subject t JVM warmup, optimizations and so forth which can take some time. It's not a huge surprise that the first few calls are slightly slower, but I guess thats not what the JVM optimizes for compared to a long-running server.

Does that help?
Comment by Jeff Dillon [ 14/Aug/14 ]
I am seeing the same behavior when using nanoTime. And it's not slightly slower, it's an order of magnitude difference. And I don't think it's JVM warmup either, as different timeout values show consistent differences every time. Using a short time out value works as expected, but is not the timeout value they desire. When the timeout is "close" to the actual execution time, it experiences this behavior. This is what I'm seeing in my tests, and is what the customer is seeing also.
Comment by Matt Ingenthron [ 20/Aug/14 ]
Jeff Dillon: I understand from James this has been observed on 2.5.1 now as well. Can you confirm and update it appropriately? Also, I've moved it to JCBC, since the investigation has lead here.
Comment by Jeff Dillon [ 20/Aug/14 ]
Yes, please see my previous comment from 12/Aug/14 1:47 PM




[JCBC-517] Add overloads for Persist/Replicate only Created: 18/Aug/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-515] Simplify environment and collapse properties Created: 18/Aug/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-beta
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-494] Support touch and unlock Created: 16/Jul/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-492] Support get + lock and get + touch Created: 15/Jul/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-491] Support counter Created: 15/Jul/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-489] Support Replica Read Created: 10/Jul/14  Updated: 18/Aug/14  Resolved: 18/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp3
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-510] Allow optional non-persistent view connections Created: 18/Aug/14  Updated: 18/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.5
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-246] Docs: No CAS+durability in API reference table Created: 12/Feb/13  Updated: 14/Aug/14  Resolved: 14/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Documentation
Affects Version/s: 1.1.2
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Major
Reporter: Perry Krug Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
The table here: http://www.couchbase.com/docs/couchbase-sdk-java-1.1/api-reference-summary.html

Doesn't contain CAS methods with durability constraints.

 Comments   
Comment by Matt Ingenthron [ 14/Aug/14 ]
Starting with the 1.2 client, we removed this section and went to straight javadoc. Should be resolved with that.




[JCBC-504] Couchbase hangs during tomcat shutdown Created: 30/Jul/14  Updated: 11/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.2, 1.4.3
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Major
Reporter: Benoit Beaudet Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Tomcat 7.0.45.


 Description   
Using couchbase lib inside a WAR. All clients connections are stopped by the application but Tomcat can't stop.
Tomcat stop process is blocked by a CouchbaseConfigConnection thread (deamon=false)

{code}
  java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
    - locked <0x0000000795b80af0> (a sun.nio.ch.Util$2)
    - locked <0x0000000795b80b00> (a java.util.Collections$UnmodifiableSet)
    - locked <0x0000000795b80aa8> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
    at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:420)
    at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:321)
{code}


Looking a the bootstrap code from com.couchbase.client.vbucket.provider.BucketConfigurationProvider class.
A CouchbaseConfigConnection is created but not closed if everything is working fine.
{code}
private boolean tryBinaryBootstrapForNode(InetSocketAddress node)
    throws Exception

  connection = new CouchbaseConfigConnection(
        cf.getReadBufSize(), fact, Collections.singletonList(node),
        initialObservers, cf.getFailureMode(),
        cf.getOperationFactory()
      );

{code}



Hack to resolve the issue.
{code}
  private void killConfigConnection(){
        ThreadGroup rootGroup = Thread.currentThread( ).getThreadGroup( );
        ThreadGroup parentGroup;
        while ( ( parentGroup = rootGroup.getParent() ) != null ) {
            rootGroup = parentGroup;
        }

        Thread[] threads = new Thread[ rootGroup.activeCount() ];
        while ( rootGroup.enumerate( threads, true ) == threads.length ) {
            threads = new Thread[ threads.length * 2 ];
        }

        for (int i = 0; i < threads.length; i++){
            final Thread thread = threads[i];
            if(thread instanceof CouchbaseConfigConnection){
                final CouchbaseConfigConnection configConn = (CouchbaseConfigConnection) thread;
                try {
                    configConn.shutdown();
                } catch (final IOException e) {
                   //
                }
            }

        }
   
{code}

 Comments   
Comment by Michael Nitschinger [ 04/Aug/14 ]
Is that a duplicate of 503?
Comment by Benoit Beaudet [ 04/Aug/14 ]
This issue isn't a duplicate. A non deamon thread is still active in Tomcat. The CouchbaseConfigConnection used at boostrap should be shutdown.
Comment by Michael Nitschinger [ 05/Aug/14 ]
Hmm maybe I don't understand it. Do you call client.shutdown() or just shutdown tomcat? Because they are by intention non-daemon threads and you need to shutdown the client manually. or do you shut it down but some resources are not freed?
Comment by Benoit Beaudet [ 05/Aug/14 ]
I call client.shutdown in web context destroy event, but the CouchbaseConfigConnection isn't under my control and not freed.




[JCBC-507] Provide compatibility with 3.0 spatial views Created: 11/Aug/14  Updated: 11/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.5
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In 3.0, the returned result doesn't contain a "bbox" property anymore. It was renamed to "key" to be more along the lines of the mapreduce views. The structure also changes a bit. The "bbox" was an array with 4 elements, with [min_x, min_y, max_x, max_y]. The key is an array containing the bounds of the dimensions, so it is: [[min_x, max_x], [min_x, max_y]].




[JCBC-508] PersistTo.TWO doesn't fail with single node cluster. Created: 11/Aug/14  Updated: 11/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.5
Security Level: Public

Type: Bug Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In the observe poll part, the persist counter gets decremented which leads to a non-failing op on a single node cluster with PersistTo.TWO




[JCBC-488] Getting Failed to Access the View Exception Created: 08/Jul/14  Updated: 07/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.0, 1.4.1
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Major
Reporter: meet_ravip Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Dependency
Relates to

 Description   
Caused by: java.lang.RuntimeException: Failed to access the view
        at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:787)


Caused by: java.util.concurrent.ExecutionException: OperationException: GENERAL
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:98)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:82)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:72)
        at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:780)
        ... 9 more
Caused by: OperationException: GENERAL
        at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:74)
        at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:103)
        at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:51)
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
        at org.apache.http.nio.protocol.HttpAsyncRequester$RequestExecutionCallback.completed(HttpAsyncRequester.java:376)
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
        at org.apache.http.nio.protocol.BasicAsyncClientExchangeHandler.responseCompleted(BasicAsyncClientExchangeHandler.java:179)


 Comments   
Comment by Michael Nitschinger [ 08/Jul/14 ]
Hi,

can you please give us more information on when you are seeing the exception (what code are you executing, whats going on at the server - nothing, rebalance, failover,..) Thanks!
Comment by meet_ravip [ 08/Jul/14 ]
Hi,
  We use couchbase 1.4 client and its dependent binaries and below is the code snippet. The failure happens in client.query method. We are running 200 concurrent request to perform the below operation. When couchbase running for more than few and if we start this test it fails, after restarting couchbase when this issue happens and restart the test then it works. Our's 3 node cluster setup. At that time failure there is no rebalance or failover. Lets know if you need any further details.
                        

CouchbaseConnectionFactoryBuilder cfb = new CouchbaseConnectionFactoryBuilder();
cfb.setOpTimeout(opTimeOut); //100000
cfb.setOpQueueMaxBlockTime(queueMaxBlockTime);30000
cfb.setTimeoutExceptionThreshold(timeoutExceptionThreshold);//998
cfb.setReadBufferSize(readBufSize);//16384
cfb.setShouldOptimize(shouldOptimize);//false
cfb.setMaxReconnectDelay(maxReconnectDelay);//30
cfb.setObsPollInterval(obsPollInterval);//10
cfb.setObsTimeout(obsTimeOut);//5
cfb.setViewTimeout(viewTimeout);//300
cfb.setViewWorkerSize(viewWorkerSize);//1
cfb.setViewConnsPerNode(viewConnectionsPerNode);//100

CouchbaseClient client = new CouchbaseClient(cfb.buildCouchbaseConnection(
hosts, bucketName, userName, password));


Query query = new Query();
query.setReduce(false);
// query.setIncludeDocs(true);
query.setIncludeDocs(false);
query.setStale(Stale.FALSE);


View view = client.getView(designDocumentName, viewName);
if (view != null) {
ViewResponse result = client.query(view, query);
if (result != null) {
List<String> keys = new ArrayList<String>();
for (ViewRow row : result) {
if (row.getKey() != null && !row.getKey().isEmpty()) {

keys.add(row.getKey());
}
}
if (keys.size() > 0) {

documents = client.getBulk(keys);
}
}
}

View Definition:-

function (doc,meta) { emit(meta.id,null);
Comment by Vladislav Koroteev [ 08/Jul/14 ]
I have same issue. I noticed that this exception occured when I do request to view and view is indexing.
Comment by RICHARDS PETER [ 09/Jul/14 ]
In our use case, data update in the couchbase bucket happens only once in a day. During the remaining day, all our clients only read the data from these buckets and still we face this issue. Could you please tell us whether it is possible to control indexing of the view?
Comment by Michael Nitschinger [ 09/Jul/14 ]
Thanks folks,

could you also tell me which server versions you are using? the Error: General points to a 500 response from the server, for whatever reason.
Comment by RICHARDS PETER [ 09/Jul/14 ]
Hi Michael,
We are using 2.1.1 community edition (build-764).
Comment by RICHARDS PETER [ 09/Jul/14 ]
Hi Michael,
Could you also tell me the server log files that should be analyzed to find more details about this error?
Comment by Michael Nitschinger [ 09/Jul/14 ]
Peter,

there is a cbcollect_info command that you can use to aggregate this info, but I would be interested in something else if you can take the time: enable FINEST logging (detailed info here http://nitschinger.at/Logging-with-the-Couchbase-Java-Client) and can you share that log? in there we should have more info from the actual server response and see whats going on.

Also, you could set a breakpoint at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:74) and print the original exception https://github.com/couchbase/couchbase-java-client/blob/release14/src/main/java/com/couchbase/client/protocol/views/ViewOperationImpl.java#L74

maybe we should rethrow that.
Comment by Vladislav Koroteev [ 09/Jul/14 ]
I tried reproduce "java.lang.RuntimeException: Failed to access the view", but I can't. I do workload on Couchbase, so I can observer permanent indexing at web-interface. From JavaSDK I do query with stale=false. Earlier I get "java.lang.RuntimeException: Failed to access the view", now I have

java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:75)
at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:781)
at my.pack.MainView.main(MainView.java:29)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:93)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:82)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:72)
... 2 more

I tried it on Couchbase Server 2.2 and Couchbase Server 2.5.1 EE, on Java SDK 1.3.2, 1.4.1, 1.4.3.

Maybe this exception occures while view is compacting, not indexing?
Comment by Michael Nitschinger [ 14/Jul/14 ]
I have a change pending that will at least print out the unparsable/unexpected JSON body so it will be easier to debug: http://review.couchbase.org/#/c/39347/
Comment by RICHARDS PETER [ 15/Jul/14 ]
Hi Michael,

Sorry for the delay in giving more details. A system upgrade at our side delayed further testing of this issue. I downloaded the source code of the couchbase client and added some log statements before the new exception is set. The code looks as follows:

************************************************************************************
  @Override
  public void handleResponse(HttpResponse response) {
    String json = getEntityString(response);
    int errorcode = response.getStatusLine().getStatusCode();
    try {
      OperationStatus status = parseViewForStatus(json, errorcode);
      ViewResponse vr = null;
      if (status.isSuccess()) {
        vr = parseResult(json);
      } else {
        parseError(json, errorcode);
      }

      ((ViewCallback) callback).gotData(vr);
      callback.receivedStatus(status);
    } catch (ParseException e) {
System.out.println("ViewOperationImpl handleResponse()"+e.getMessage());
e.printStackTrace();
      LOGGER.log(Level.WARNING,"ViewOperationImpl handleResponse() "+e.getMessage(),e);
setException(new OperationException(OperationErrorType.GENERAL,
          "Error parsing JSON"));
    }
    callback.complete();
  }

************************************************************************************

I was able to reproduce the issue with the new jar file packaged with the above code change. Please find the logs in our application:

2014-07-15 12:11:28 STDIO [INFO] ViewOperationImpl handleResponse()Cannot read json:
2014-07-15 12:11:28 STDIO [ERROR] java.text.ParseException: Cannot read json:
2014-07-15 12:11:28 STDIO [ERROR] at com.couchbase.client.protocol.views.HttpOperationImpl.parseViewForStatus(HttpOperationImpl.java:140)
2014-07-15 12:11:28 STDIO [ERROR] at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:64)
2014-07-15 12:11:28 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:103)
2014-07-15 12:11:28 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:51)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequester$RequestExecutionCallback.completed(HttpAsyncRequester.java:376)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.nio.protocol.BasicAsyncClientExchangeHandler.responseCompleted(BasicAsyncClientExchangeHandler.java:179)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:349)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:236)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:267)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:165)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:51)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
2014-07-15 12:11:28 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:584)
2014-07-15 12:11:28 STDIO [ERROR] at java.lang.Thread.run(Thread.java:744)

Please let me know what could be going wrong. The issue does not occur always. It mostly happens after some restarts of our application and only a few clients are getting this issue.

Thanks,
Richards Peter.
Comment by Michael Nitschinger [ 15/Jul/14 ]
Hey, could you do me a favor and print both the HttpResponse and or the raw content body? from the log message I cannot see what actually got returned from the server only that we were unable to parse it.
Comment by RICHARDS PETER [ 16/Jul/14 ]
Hi Michael,

I added more print statements in the catch block to capture the request and response. Please find the modified code and the logs:

 public void handleResponse(HttpResponse response) {
   //retained the same code till here.
    } catch (ParseException e) {
System.out.println("ViewOperationImpl handleResponse()"+e.getMessage());
e.printStackTrace();
System.out.println("ViewOperationImpl handleResponse() HttpRequest: "+getRequest().toString());
System.out.println("ViewOperationImpl handleResponse() HttpResponse: "+response.toString()+" json:"+json+" errorcode:"+ errorcode);
      LOGGER.log(Level.WARNING,"ViewOperationImpl handleResponse() "+e.getMessage(),e);
setException(new OperationException(OperationErrorType.GENERAL,
          "Error parsing JSON"));
    }
    callback.complete();
  }

The issue was reproduced today morning. Please find our application log:

2014-07-16 10:36:50 STDIO [INFO] ViewOperationImpl handleResponse()Cannot read json:
2014-07-16 10:36:50 STDIO [ERROR] java.text.ParseException: Cannot read json:
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.protocol.views.HttpOperationImpl.parseViewForStatus(HttpOperationImpl.java:140)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:64)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:103)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:51)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequester$RequestExecutionCallback.completed(HttpAsyncRequester.java:376)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.BasicAsyncClientExchangeHandler.responseCompleted(BasicAsyncClientExchangeHandler.java:179)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:349)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:236)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:267)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:165)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:51)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:584)
2014-07-16 10:36:50 STDIO [ERROR] at java.lang.Thread.run(Thread.java:744)


2014-07-16 10:36:50 STDIO [INFO] ViewOperationImpl handleResponse()Cannot read json:
2014-07-16 10:36:50 STDIO [ERROR] java.text.ParseException: Cannot read json:
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.protocol.views.HttpOperationImpl.parseViewForStatus(HttpOperationImpl.java:140)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.protocol.views.ViewOperationImpl.handleResponse(ViewOperationImpl.java:64)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:103)
2014-07-16 10:36:50 STDIO [ERROR] at com.couchbase.client.http.HttpResponseCallback.completed(HttpResponseCallback.java:51)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequester$RequestExecutionCallback.completed(HttpAsyncRequester.java:376)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.BasicAsyncClientExchangeHandler.responseCompleted(BasicAsyncClientExchangeHandler.java:179)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:349)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:236)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:267)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:165)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.DefaultHttpClientIODispatch.onInputReady(DefaultHttpClientIODispatch.java:51)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:113)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:159)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:338)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:316)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:277)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:105)
2014-07-16 10:36:50 STDIO [ERROR] at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:584)
2014-07-16 10:36:50 STDIO [ERROR] at java.lang.Thread.run(Thread.java:744)

2014-07-16 10:36:50 STDIO [INFO] ViewOperationImpl handleResponse() HttpRequest: GET /COMMON_DIMENSION/_design/COMMON_DIMENSION/_view/COMMON_DIMENSION?reduce=false&stale=false [Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Connection: Keep-Alive, User-Agent: JCBC/1.2, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q
2014-07-16 10:36:50 STDIO [INFO] 09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09
2014-07-16 10:36:50 STDIO [INFO] NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NT
2014-07-16 10:36:50 STDIO [INFO] U9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9
2014-07-16 10:36:50 STDIO [INFO] OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm1pub:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: 10.62.81.165:8092, Authorization: Basic Q09NTU9OX0RJTUVOU0lPTjo=, Host: storm2pub:8092]


2014-07-16 10:36:50 STDIO [INFO] ViewOperationImpl handleResponse() HttpResponse: HTTP/1.1 400 Bad Request [Server: MochiWeb/1.0 (Any of you quaids got a smint?), Date: Wed, 16 Jul 2014 05:06:50 GMT, Content-Length: 0] json: errorcode:400
Comment by Michael Nitschinger [ 16/Jul/14 ]
Hi,

thanks for reporting this. I'm not exactly sure whats going on here, could it be that all those authorization headers are part of _one request_?

Can you clarify if you are using the default bucket or something named differently?
I think this is a bug, but I'm not entirely sure if this is the same as the others reported here, let me get a fix up for this since this seems to be trivial.
Comment by RICHARDS PETER [ 16/Jul/14 ]
We have created a new bucket - COMMON_DIMENSION and this bucket has a view also with the same name. The view definition and code to retrieve the data is already shared by meet_ravip.
Comment by Michael Nitschinger [ 16/Jul/14 ]
Thanks, the good news is that I have a trivial bugfix for your case I think. Currently we retry a request if it fails for some reason against a node, but the code just adds the headers again and I guess the server just rejects it at some point. So I'll add checks around the add to make sure they are only added once.
Comment by Michael Nitschinger [ 16/Jul/14 ]
See: http://review.couchbase.org/#/c/39429/

I think this will get into 1.4.4, but you can patch it on your own if you can't wait for it. If you do, let me know if it works
Comment by RICHARDS PETER [ 16/Jul/14 ]
Thanks Michael,
That is really a good news. Please share the details of the fix when it is available. We will test the fix in our setup and update you about our findings.
Comment by Michael Nitschinger [ 16/Jul/14 ]
1.4.4 is scheduled for first week of august, but I'll merge it into master much sooner (this week I hope) once it goes through review. If you know how to build it from github (if not I can help you with) you can play around with it sooner.
Comment by RICHARDS PETER [ 16/Jul/14 ]
Thanks Michael.
I am about to test the aforementioned patch on my setup. I have applied the changes to the 1.4 source code which we downloaded a few days back. I will update you about our observation. Since this issue cannot be reproduced very quickly, my update on this issue may be slightly delayed (24 hours minimum).
Comment by RICHARDS PETER [ 17/Jul/14 ]
Hi Michael,
I would like to clarify one more doubt related to this comment:
"Currently we retry a request if it fails for some reason against a node, but the code just adds the headers again and I guess the server just rejects it at some point. "
We would like to minimize such failures. Is it possible to find out the reason why the request failed against a node? Is it possible to capture such cases in our log by modifying any class? Could you please guide us on the same?
Comment by Michael Nitschinger [ 17/Jul/14 ]
Hi,

normally you don't need to be concerned about that. There are various situations as part of the regular operations (like adding a node) where there is a limited timespan where the node is added, but does not contain any partitions yet. We get a response from the server and retry behind the scenes. Also with failovers where the config is not updated the node returns an error and we retry.

If we don't get to a good result we will try until the op times out with the given timeout, so you should be good on that front. Of course, the behavior reported here was because of a bug.
Comment by RICHARDS PETER [ 17/Jul/14 ]
Thanks Michael,
We normally get this error after some restart of our application. Till now we were not able to reproduce this issue. However we got a new error after the latest restart of our application. The stacktrace is as follows:

Caused by: com.couchbase.client.vbucket.ConfigurationException: Could not fetch a valid Bucket configuration.
at com.couchbase.client.vbucket.provider.BucketConfigurationProvider.bootstrap(BucketConfigurationProvider.java:123)
at com.couchbase.client.vbucket.provider.BucketConfigurationProvider.getConfig(BucketConfigurationProvider.java:373)
at com.couchbase.client.CouchbaseConnectionFactory.getVBucketConfig(CouchbaseConnectionFactory.java:317)
at com.couchbase.client.CouchbaseClient.(CouchbaseClient.java:258)

All that we did was an application restart. There were no other changes in the environment. We could see all the buckets on the couchbase nodes when this error was produced. The buckets also had the same items as before. During the restart there will be around 200 threads accessing the same data. I would like to know the possible scenarios which can lead to this error. Would you like me to open a new issue for this?
Comment by Michael Nitschinger [ 17/Jul/14 ]
Yes please, and if you could provide debug level logging (or finest even) would be good, thanks!




[JCBC-503] Client shutdown doesn't release observers and leak memory. Created: 29/Jul/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.2
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Major
Reporter: Benoit Beaudet Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
If a client is using VBucket the shutdown method call BucketConfigurationProvider.shutdown().

{code}
try {
      CouchbaseConnectionFactory cf = (CouchbaseConnectionFactory) connFactory;
      cf.getConfigurationProvider().shutdown();
      if(vconn != null) {
        vconn.shutdown();
      }
{code}

The observers array reference closed CoucheBaseClient.
To fix the issue, I need to call directly unsubscribe method.
connectionFactory.getConfigurationProvider().unsubscribe((Reconfigurable) client);
 

 Comments   
Comment by Michael Nitschinger [ 04/Aug/14 ]
Does that look valid?
http://review.couchbase.org/#/c/40263/
Clearing them on shutdown seems valuable
Comment by Benoit Beaudet [ 04/Aug/14 ]
Hi!

I think that the observers list is global. A shutdown of 1 client will clears all active clients observers?.
This is why I used unsubscribe.

 @Override
  public void unsubscribe(Reconfigurable rec) {
    observers.remove(rec);
  }
Comment by Michael Nitschinger [ 05/Aug/14 ]
Hey,

the thing is if you shut down the config manager noone else can use it anyway because its down, so we can just unsubscribe all of them right in the shutdown method. There is no need to keep more of them around, especially because this is intended to be internal interface.
Comment by Benoit Beaudet [ 05/Aug/14 ]
In my scenario. I use a pool to create more connections under load and release them after an idle period.
I shutdown clients not only at web app shutdown, but also at runtime.

The factory is a singleton in my project. For each CouchbaseClientConnection created with the factory, an observer is added.
cf.getConfigurationProvider().subscribe(this);

CouchbaseClient.shutdown method call indirectly the factory getConfigurationProvider().
   CouchbaseConnectionFactory cf = (CouchbaseConnectionFactory) connFactory;
      cf.getConfigurationProvider().shutdown();

This is why is suggested that observers array size should match created connections. Using internal subscribe/unsubscribe to match the pattern.



Comment by Michael Nitschinger [ 05/Aug/14 ]
Every time you create a new CouchbaseClient object, all of those objects are created again.

You should never ever shut down or listen on the configuration provider on your own, that is internal behavior. You should 1) always only use the CouchbaseClient interface and 2) reuse them as long as possible.

Why do you shut them down at runtime?
Comment by Benoit Beaudet [ 05/Aug/14 ]
I use CouchbaseClient.shutdown() to evict clients and use new CouchbaseClient(using the singleton CouchbaseConnectionFactory as parameter) to recreate them. I shouldn't use the Configuration provider directly.
But the internal configuration provider is keeping references on all connections created, even if the pool closed them. I create more than 1 client at bootup and close it at shutdown.
The pool manage the growth and idling clients. It's the reason to shutdown a client if idling for a long time.




[JCBC-483] Add INFO level log of client version to startup Created: 27/Jun/14  Updated: 05/Aug/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: 1.4.2
Fix Version/s: 1.4.5
Security Level: Public

Type: Improvement Priority: Major
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
In CouchbaseClient <init> we log details from the CouchbaseConnectionFactory. It would also be useful to INFO level log the client version and the spymemcached version.

 Comments   
Comment by Matt Ingenthron [ 27/Jun/14 ]
Since it's such a minor change, setting it to 1.4.3. I'll defer to you on whether it should be pushed back or not.
Comment by Michael Nitschinger [ 30/Jun/14 ]
It's a little harder to do because there is the BuildInfo, but we need to make sure its properly in path and distributed with our jars. I'm not sure this is done right now but I could be wrong.

If we could defer it to 1.4.4 then it would be better to release 1.4.3 tomorrow.




[JCBC-464] Client never recovers when all nodes are shut down Created: 03/Jun/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File thread-dump.txt    
Issue Links:
Relates to

 Description   
When running this demo, if I disconnect the cluster entirely and restart it, the client reconnects but never recovers:
https://github.com/couchbaselabs/CBLiveDemo

 Comments   
Comment by Michael Nitschinger [ 14/Jul/14 ]
http://review.couchbase.org/#/c/39346/




[JCBC-490] Log JSON body when failed to parse valid body Created: 14/Jul/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.4
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
This change does not fix an error itself, but makes it much easier to discover what valid JSON gets sent over where we are failing to handle it.




[JCBC-505] Concurrency issue in the View Query builder with regexes Created: 30/Jul/14  Updated: 05/Aug/14  Resolved: 05/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified





[JCBC-495] 1.4x doesn't build if source dir contains whitespace Created: 16/Jul/14  Updated: 04/Aug/14  Resolved: 04/Aug/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.4.3
Fix Version/s: 1.4.4
Security Level: Public

Type: Bug Priority: Minor
Reporter: David Haikney Assignee: David Haikney
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
After running a git clone; git checkout of 1.4.3 I was unable to build on my local machine. "ant package" yielded the following:

 [exec] /Users/dhaikney/Documents/Google Drive/dev/couchbase-java-client/src/scripts/write-version-info.sh: line 47: ${changesfile}: ambiguous redirect

Some of the variables in the write-version-info.sh script need surrounding quotes to allow for the whitespace.

 Comments   
Comment by David Haikney [ 16/Jul/14 ]
http://review.couchbase.org/39443




[JCBC-437] implement JSR-107 Java Temporary Caching API Created: 25/Mar/14  Updated: 31/Jul/14

Status: Open
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: Task Priority: Critical
Reporter: Matt Ingenthron Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
As previously discussed, we should look to implement JSR-107 to meet the standards.




[JCBC-501] Implement Support for DCP Created: 29/Jul/14  Updated: 29/Jul/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: .future
Security Level: Public

Type: New Feature Priority: Major
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Implement support for 3.0+ DCP as a replacement for the TAP protocol and TAP Client.






[JCBC-496] Add unit test for JCBC-488 Created: 22/Jul/14  Updated: 28/Jul/14

Status: Open
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: .next
Security Level: Public

Type: Improvement Priority: Major
Reporter: Michael Nitschinger Assignee: Wei-Li Liu
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
We need to add a unit test for the fix in http://www.couchbase.com/issues/browse/JCBC-488 just to make sure it works as expected. The appropriate change ishttp://review.couchbase.org/#/c/39429/ and has been merged already.

I'd propose a unit test which mocks out the underlying stuff and just pushes a HttpOperation a few times through the ViewConnection#addOp which should add the headers only 1 times instead of N.

 Comments   
Comment by Wei-Li Liu [ 25/Jul/14 ]
I think we have something similar to JCBC-488 test case in the scenario test framework (Service Restart)

I pull in your fix http://review.couchbase.org/#/c/39429/ and reproduce the issue by switching thread # to 200 instead of 10 (default)
The report shows that if thread is 10 works for both mc (get,set) and ht, cb (view, query). If jumps thread # to 200, view and query workload would fail
https://docs.google.com/spreadsheets/d/1sjDpJL_xuVv3kDhbnJKn5qNLmvzCi80gbdq5FkOObzQ/edit#gid=1476387871

I can make a separate unit test to make sure when there are multiple http operations, there can only be 1 header.






[JCBC-151] Client does timeout on connect in specific java environments (was believed to be java7 related). Created: 21/Nov/12  Updated: 21/Jul/14  Resolved: 31/Dec/13

Status: Resolved
Project: Couchbase Java Client
Component/s: Core
Affects Version/s: 1.1-dp4
Fix Version/s: 1.3.0
Security Level: Public

Type: Bug Priority: Critical
Reporter: Michael Nitschinger Assignee: Michael Nitschinger
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to JCBC-388 Refactor View Connection Management Resolved

 Description   
People report that the Client does not work with 1.7. Here is a sample stack trace:

2012-11-20 00:29:11.228 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-20 00:29:11.240 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@3f757322
2012-11-20 00:29:11.502 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-20 00:29:11.505 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-20 00:29:11.647 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-20 00:29:12.051 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-20 00:29:12.060 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
2012-11-20 00:29:17.111 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [localhost/127.0.0.1:8092(closed)]
java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:68)
at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:428)
at Example1.main(Example1.java:43)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:80)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:65)
... 2 more
2012-11-20 00:30:12.168 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-20 00:30:12.177 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Disconnected from the target VM, address: '127.0.0.1:65280', transport: 'socket'

Process finished with exit code 0


Also See http://www.couchbase.com/forums/thread/java-asyncconnectionmanager-timed-out-waiting-operation-please-help-console-log-included
And http://stackoverflow.com/questions/13466010/using-java-api-in-scala-to-query-views-in-couchbase-throws-timeout-exception?utm_source=twitterfeed&utm_medium=twitter

 Comments   
Comment by Quinn Slack [ 27/Nov/12 ]
This doesn't appear to be strictly related to Java7. I was able to get similar code to work on Java7. I even simulated loading a Play2 environment with other JARs that could potentially cause conflicts. It's possible that my simulation of the Play2 environment was insufficient and that Play2 does other stuff...

Code at https://github.com/sqs/couchbase-scala-example. Run with:

CBURL=http://localhost:8091/pools CBPASSWORD=mypassword sbt -Dconfig.file=conf/application.conf '~run'

Here is the output on my system that shows it's on Java7 (openjdk) and that shows the expected output from the views:

on java version 1.7.0_09
[info] play - Application started (Prod)
2012-11-27 01:34:27.115 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/127.0.0.1:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2012-11-27 01:34:27.120 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@6267e5a2
2012-11-27 01:34:27.147 INFO com.couchbase.client.ViewConnection: Added localhost/127.0.0.1:8092 to connect queue
2012-11-27 01:34:27.149 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2012-11-27 01:34:27.176 INFO net.spy.memcached.auth.AuthThread: Authenticated to localhost/127.0.0.1:11210
2012-11-27 01:34:27.282 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2012-11-27 01:34:27.288 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: localhost/127.0.0.1:8092 - Session request successful
Res = List(com.couchbase.client.protocol.views.ViewRowNoDocs@4667820f, com.couchbase.client.protocol.views.ViewRowNoDocs@358bcae5, com.couchbase.client.protocol.views.ViewRowNoDocs@6cb59bd9, com.couchbase.client.protocol.views.ViewRowNoDocs@70afb51, com.couchbase.client.protocol.views.ViewRowNoDocs@61f98673)
2012-11-27 01:34:27.396 INFO com.couchbase.client.CouchbaseConnection: Shut down Couchbase client
2012-11-27 01:34:27.403 INFO com.couchbase.client.ViewNode: Couchbase I/O reactor terminated
Comment by Michael Nitschinger [ 27/Nov/12 ]
Are you running on jdk7 on mac? Also, when you use play 2.1 it should work. Play2-related issues more were because of netty version incompatibilities.
Comment by Quinn Slack [ 27/Nov/12 ]
This is jdk7 on Arch Linux and Play 2.1-RC1.

I tried using a different netty, but no luck. What worked for me was moving calls to "new CouchbaseClient" out of static "object" initializers, since they appeared to be getting called in the Netty I/O thread (I got this exception in new CouchbaseClient: java.lang.IllegalStateException: await*() in I/O thread causes a dead lock or sudden performance drop. Use addListener() instead or call await*() from a different thread).
Comment by Michael Nitschinger [ 28/Nov/12 ]
Hi Quinn, this issue is a different one! The issue described here is that people seem to find connection timeouts with java 7.
Comment by Quinn Slack [ 28/Nov/12 ]
They may be related--or may just have similar symptoms. I was not getting timeout exceptions, but I was seeing view requests hang indefinitely until I made the fix described above.
Comment by dragos [ 13/Mar/13 ]
Hi Michael, can you help with this error ? I am trying to find a workaround or something to work. This is the only thing which is keeping us from using Couchbase. I also posted here : http://www.couchbase.com/forums/thread/couchbase-connectivity-problem-aws-vpc . As you can see in the times the timeout is received really fast.

2013-03-13 11:40:47.702 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.X.XXX:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 11:40:52.712 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@5cbfe9d
2013-03-13 11:40:52.840 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.X.XXX/10.0.X.XXX:11210
2013-03-13 11:40:57.860 INFO com.couchbase.client.ViewConnection: Added 10.0.X.XXX to connect queue
2013-03-13 11:40:57.863 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 11:40:58.070 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 11:40:58.077 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.X.XXX:8092 - Session request successful
2013-03-13 11:41:03.113 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.X.XXX/10.0.X.XXX:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:63)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi dragos,

can you please post the full code where you connect to couchbase? (including factories and such)

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Mike, this is the POC client for Couchbase:
package couchbase.training.view.poc;

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URI;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.TimeUnit;

import org.apache.commons.io.IOUtils;
import org.json.JSONException;
import org.json.XML;

import com.couchbase.client.CouchbaseClient;
import com.couchbase.client.CouchbaseConnectionFactory;
import com.couchbase.client.CouchbaseConnectionFactoryBuilder;
import com.couchbase.client.protocol.views.ComplexKey;
import com.couchbase.client.protocol.views.Query;
import com.couchbase.client.protocol.views.Stale;
import com.couchbase.client.protocol.views.View;
import com.couchbase.client.protocol.views.ViewResponse;
import com.couchbase.client.protocol.views.ViewRow;

public class CouchbasePOC
{
public static void main(String[] args) throws FileNotFoundException, JSONException, IOException
{
System.out.println("Hello from CouchBase");

// Set the URIs and get a client
final List<URI> uris = new LinkedList<URI>();

// Connect to localhost or to the appropriate URI(s)
uris.add(URI.create("http://10.0.7.164:8091/pools"));

final String mappedName="problems";
long start = System.currentTimeMillis();

CouchbaseConnectionFactoryBuilder m_couchbaseConnectionFactoryBuilder = new CouchbaseConnectionFactoryBuilder();
m_couchbaseConnectionFactoryBuilder.setMaxReconnectDelay(10000);
m_couchbaseConnectionFactoryBuilder.setOpQueueMaxBlockTime(100);
m_couchbaseConnectionFactoryBuilder.setOpTimeout(20000);
m_couchbaseConnectionFactoryBuilder.setShouldOptimize(true);
m_couchbaseConnectionFactoryBuilder.setViewTimeout(300000);

CouchbaseConnectionFactory connFactory = m_couchbaseConnectionFactoryBuilder.buildCouchbaseConnection(uris, mappedName, "");

CouchbaseClient client = null;
try
{
client = new CouchbaseClient(connFactory);
}
catch (IOException e)
{
System.err.println("IOException connecting to Couchbase: " + e.getMessage());
System.exit(1);
}

final View view = client.getView(mappedName, mappedName);
final Query query = new Query();

final ComplexKey key = ComplexKey.of("problem",null,null,null,null);
query.setKey(key);
query.setStale( Stale.UPDATE_AFTER );
int counter=0;
try
{
final ViewResponse result = client.query(view, query);
for(ViewRow row : result)
{
counter++;
System.out.println(row.getId());
}
}
catch(java.lang.VerifyError e)
{
System.out.println("Error: " + e.getMessage());
}

System.out.println("time: " + (System.currentTimeMillis()-start) );
System.out.println("documents: " + counter );

client.shutdown(3, TimeUnit.SECONDS);
System.exit(0);
}
}
Comment by dragos [ 13/Mar/13 ]
I hope you would not mind that i called you Mike and not Michael.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hehe no worries.

Can you do me a favor and remove the factory for a second and just use CouchbaseClient() without anything else? Let me know if the problem still persists or not! Thanks
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
First i want to thank you for the quick feed-back. Your owesome !!!

Before using the factory i used the CouchbaseClient(uris, bucketName, "") which is also timing out. I added the factory to be able increase the timeout times for the connection but still the same problem.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hmm okay, so its not the factory (we had some issues with default values in the past, thats why I'm asking).

Can you pin me down your operating system and exact java version you're using?Thanks for your help!
Comment by Michael Nitschinger [ 13/Mar/13 ]
Also, since you're so responsive (I hope we can pin this down finally!).. Can you run the same code with DEBUG log enabled (see https://code.google.com/p/spymemcached/wiki/Logging).. just use Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST); instead of Logger.getLogger("net.spy.memcached").setLevel(Level.FINEST);

Thanks!
Comment by dragos [ 13/Mar/13 ]
This is my test environment for the client which will be similar for production.
OS Ubuntu Server 11.10 x64
java -version result:
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

My network infrastructure is like this:
In an AWS virtual private center (VPC) i have a subnet and in this subnet there are 2 machines. The client machine and the Couchbase server. The machines have identical OS and java version as mentioned above. I have open the ports and also checked the network ACL. The traffic is free to go,no restriction. If i am using a rest query with curl for the view i receive the result, the problem is only in the java SDK.
If needed more information please ask. I will gladly offer it.

Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

yes, if you can please rerun the script with DEBUG logging enabled (see last comment).. Thanks very much in advance!
Comment by dragos [ 13/Mar/13 ]
I've setted the logger to finest and this is the error log obtained. Also i tested again the connection with curl and no problemes.

Mar 13, 2013 1:30:14 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
2013-03-13 13:30:14.679 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-03-13 13:30:19.689 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@4e99353f
2013-03-13 13:30:19.810 INFO net.spy.memcached.auth.AuthThread: Authenticated to 10.0.7.164/10.0.7.164:11210
2013-03-13 13:30:24.828 INFO com.couchbase.client.ViewConnection: Added 10.0.7.164 to connect queue
2013-03-13 13:30:24.830 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-03-13 13:30:25.025 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-03-13 13:30:25.033 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /10.0.7.164:8092 - Session request successful
2013-03-13 13:30:30.083 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
        at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:475)
        at couchbase.training.view.poc.CouchbasePOC.main(CouchbasePOC.java:68)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
        at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
        at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)
        ... 2 more

curl result

curl http://10.0.7.164:8092/problems/_design/dev_problems/_view/problems?stale=false\&connection_timeout=60000\&limit=10\&skip=0
{"total_rows":0,"rows":[
]
}
[4]+ Done
Comment by dragos [ 13/Mar/13 ]
I am available for next steps in debuging. With what can i help next ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, hmm no there should be more output (DEBUG), not just info... you need to copy this whole snippet before your code actually executes:

        // Tell spy to use the SunLogger
        Properties systemProperties = System.getProperties();
        systemProperties.put("net.spy.log.LoggerImpl", "net.spy.memcached.compat.log.SunLogger");
        System.setProperties(systemProperties);

        Logger.getLogger("com.couchbase.client").setLevel(Level.FINEST);

        //get the top Logger
        Logger topLogger = java.util.logging.Logger.getLogger("");

        // Handler for console (reuse it if it already exists)
        Handler consoleHandler = null;
        //see if there is already a console handler
        for (Handler handler : topLogger.getHandlers()) {
            if (handler instanceof ConsoleHandler) {
                //found the console handler
                consoleHandler = handler;
                break;
            }
        }

        if (consoleHandler == null) {
            //there was no console handler found, create a new one
            consoleHandler = new ConsoleHandler();
            topLogger.addHandler(consoleHandler);
        }

        //set the console handler to fine:
        consoleHandler.setLevel(java.util.logging.Level.FINEST);


I guess you're running it from your IDE?
Comment by dragos [ 13/Mar/13 ]
I am building the client on local IDE then uploading it to EC2 test machine. The snipped of code word and now i have a full log.

Mar 13, 2013 1:53:10 PM com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default?uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP readToString
FINE: Attempting to read configuration from URI: http://10.0.7.164:8091/pools/default/buckets?v=120523822&uuid=44b1e62dd7e65cd38ae7faaabe5ebb64
Mar 13, 2013 1:53:11 PM net.spy.memcached.MemcachedConnection createConnections
INFO: Added {QA sa=/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Mar 13, 2013 1:53:11 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Updating nodesMap in VBucketNodeLocator.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Adding node with address 10.0.7.164:11210.
Mar 13, 2013 1:53:16 PM com.couchbase.client.vbucket.VBucketNodeLocator fillNodesEntries
FINE: Node added is {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8}.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=false, c=true, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=8})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
INFO: Connection state changed for sun.nio.ch.SelectionKeyImpl@2c76e369
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added Cmd: 10 Opaque: 1 to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=0, #iq=1, topRop=null, topWop=null, toWrite=0, interested=8}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 24 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: Cmd: 10 Opaque: 1 and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: No selectors ready, interrupted: false
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleInputQueue
FINE: Handling queue
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection insertOperation
FINE: Added SASL auth operation to {QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4}
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=false, w=true, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=0, #Wops=1, #iq=0, topRop=null, topWop=SASL auth operation, toWrite=0, interested=4})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selected 1, selected 1 keys
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Handling IO for: sun.nio.ch.SelectionKeyImpl@2c76e369 (r=true, w=false, c=false, op={QA sa=10.0.7.164/10.0.7.164:11210, #Rops=1, #Wops=0, #iq=0, topRop=SASL auth operation, topWop=null, toWrite=0, interested=1})
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Read 37 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.auth.AuthThread$1 receivedStatus
INFO: Authenticated to 10.0.7.164/10.0.7.164:11210
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleReads
FINE: Completed read op: SASL auth operation and giving the next 0 bytes
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Done dealing with queue.
Mar 13, 2013 1:53:16 PM net.spy.memcached.MemcachedConnection handleIO
FINE: Selecting with delay of 0ms
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewConnection createConnections
INFO: Added 10.0.7.164 to connect queue
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseClient <init>
INFO: viewmode property isn't defined. Setting viewmode to production mode
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ConfigurationProviderHTTP subscribe
FINE: Subscribing an object for reconfiguration updates com.couchbase.client.CouchbaseClient
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686] OPEN


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is true and Channel State is OPEN.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] BOUND: /10.0.7.213:55729


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.213:55729 and Channel State is BOUND.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINEST: Channel state changed: [id: 0x2e273686, /10.0.7.213:55729 => /10.0.7.164:8091] CONNECTED: /10.0.7.164:8091


Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler handleUpstream
FINER: Channel state change is not a disconnect. Event value is /10.0.7.164:8091 and Channel State is CONNECTED.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: STATUS: 200 OK
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: VERSION: HTTP/1.1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Cache-Control = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Content-Type = application/json; charset=utf-8
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Date = Wed, 13 Mar 2013 13:55:42 GMT
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Pragma = no-cache
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Server = Couchbase Server 2.0.0-1976-rel-enterprise
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: HEADER: Transfer-Encoding = chunked
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER:

Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: CHUNKED CONTENT {
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: {"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3066
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 2048
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: ],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 3072
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: 1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketUpdateResponseHandler finerLog
FINER: Chunk length is: 361
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.BucketMonitor logFiner
FINER: Getting server list returns this last chunked response:
{"name":"problems","bucketType":"membase","authType":"sasl","saslPassword":"","proxyPort":0,"replicaIndex":false,"uri":"/pools/default/buckets/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","streamingUri":"/pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824","localRandomKeyUri":"/pools/default/buckets/problems/localRandomKey","controllers":{"compactAll":"/pools/default/buckets/problems/controller/compactBucket","compactDB":"/pools/default/buckets/problems/controller/compactDatabases"},"nodes":[{"couchApiBase":"http://10.0.7.164:8092/problems","replication":0.0,"clusterMembership":"active","status":"healthy","thisNode":true,"hostname":"10.0.7.164:8091","clusterCompatibility":131072,"version":"2.0.0-1976-rel-enterprise","os":"x86_64-unknown-linux-gnu","ports":{"proxy":11211,"direct":11210}}],"stats":{"uri":"/pools/default/buckets/problems/stats","directoryURI":"/pools/default/buckets/problems/statsDirectory","nodeStatsListURI":"/pools/default/buckets/problems/nodes"},"ddocs":{"uri":"/pools/default/buckets/problems/ddocs"},"nodeLocator":"vbucket","autoCompactionSettings":false,"fastWarmupSettings":false,"uuid":"7675b7791efe17220792c2b41ff6c824","vBucketServerMap":{"hashAlgorithm":"CRC","numReplicas":1,"serverList":["10.0.7.164:11210"],"vBucketMap":[[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1],[0,-1]]},"bucketCapabilitiesVer":"","bucketCapabilities":["touch","couchapi"]}
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@4cf3fdc4
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: Received an update, notifying reconfigurables about a com.couchbase.client.vbucket.config.Bucketcom.couchbase.client.vbucket.config.Bucket@fadb83cf
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.ReconfigurableObserver update
FINEST: It says it is problems and it's talking to /pools/default/bucketsStreaming/problems?bucket_uuid=7675b7791efe17220792c2b41ff6c824
Mar 13, 2013 1:53:21 PM com.couchbase.client.CouchbaseConnection reconfigure
FINE: Node 10.0.7.164/10.0.7.164:11210 will stay in cluster config after reconfiguration.
Mar 13, 2013 1:53:21 PM com.couchbase.client.vbucket.VBucketNodeLocator updateLocator
FINE: Received updated configuration with insignificant changes.
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager processConnectionRequests
INFO: Opening new Couchbase HTTP connection
Mar 13, 2013 1:53:21 PM com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback completed
INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi, thanks for the log..

interestingly, there is nothing unusual in this.. I'll investigate further and come back to you if I need more info..

thanks so far!
Comment by dragos [ 13/Mar/13 ]
Thank you for spending time on this. Looking forward for a fix :) .
Comment by Michael Nitschinger [ 13/Mar/13 ]
interestingly, the connection is open and then times out...

INFO: /10.0.7.164:8092 - Session request successful
Mar 13, 2013 1:53:21 PM com.couchbase.client.ViewNode$EventLogger connectionOpen
FINE: Connection open: [/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionTimeout
SEVERE: Connection timed out: [10.0.7.164/10.0.7.164:8092]
Mar 13, 2013 1:53:26 PM com.couchbase.client.ViewNode$EventLogger connectionClosed
FINE: Connection closed: [10.0.7.164/10.0.7.164:8092(closed)]

this sounds highly suspicious to me, maybe this is a bug in apache httpcore-nio...
Comment by dragos [ 13/Mar/13 ]
The time out is displayed after 5 seconds. As you suggested i removed the factory and the detailed logged above i obtained with using CouchbaseClient(uris, mappedName, ""). Do you have some way of testing the httpCore-nio ? Should i try to look for latest version of httpCore-nio.. ?
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi,

yes you could try that.. I ran the test suite with 4.2.3 which is the latest one and there were no errors, so our code should work with it. You need to exchange the httpcore and httpcore-nio with the latest 4.2.3 jars

I don't think this is 100% the issue, but it helps us get rid of one more factor!

Thanks!
Michael
Comment by Michael Nitschinger [ 13/Mar/13 ]
Okay, the 5 second period can come from this:

 HttpParams params = new SyncBasicHttpParams();
      params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 5000)
          .setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 5000)
....


SO_TIMEOUT is: Defines the socket timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for data or, put differently, a maximum period inactivity between two consecutive data packets).

and CONNECTION_TIMEOUT is: Determines the timeout in milliseconds until a connection is established.


Dragos, can it be the case that it takes more than 5 seconds to return a result when using curl or so? That would explain the "fail fast" timeout here.. not saying that these times are okay this way, just to find the root cause..

Thanks!
Comment by dragos [ 13/Mar/13 ]
Hi Michael,
I've made a test with httpcore-nio-4.2.jar and with httpcore-4.2.1.jar and your intuition is good. The httpcore is not the problem because the timeout is still there. With curl the results is back fast, in less then half a second. So the problem must be somewhere in the java SDK in the way the view is retrieved.
Comment by dragos [ 13/Mar/13 ]
I've moved the Couchbase server to the same machine as the client to have all in same environment for testing. Now all is working fine but it is not possible for our product to stay on the same machine as Couchbase server for obvious reasons. In production we will have more then one app machine which we will try to access the Couchbase cluster.
Comment by Michael Nitschinger [ 13/Mar/13 ]
Hi Dragos,

okay so its definitely something with those timeouts.. The thing is we can increase them for sure, but the question is if it would work for your application given the long network latency - and if your application stays performant that way.

When you're running curl on a request, how long does it take?
Comment by dragos [ 14/Mar/13 ]
The curl request is very fast (a few milliseconds, and less then 0.5 seconds i am sure) and the network latency it is not a problem. Some how the couchbase client is taking to much time connecting. When it need to connect to a network machine in the same lan it takes more then 10 seconds. I experimented this in EC2 environment and on my local machine with a VM. If couchbase is local with the test app, the couchbase client is connecting fast.
Do you have any points on how a connection should be setup for couchbase client ? I am trying to use the factory builder and the connection factory from java sdk but still with no success.
Comment by Tim Smith (Inactive) [ 14/Mar/13 ]
Dragos,

I can think of two separate things to try. One is to increase the SO_TIMEOUT to 8000, CONNECTION_TIMEOUT to 12000. Identify what's being hit.

Probably more effective would be to use Wireshark to capture all traffic on the client machine, and see what is really happening. Start the capture (with tshark command-line tool, for example), then run the test client, let it time out, then run the curl command, let it succeed, then stop the capture. Gzip the capture data and attach it to this issue.

Tim
Comment by dragos [ 14/Mar/13 ]
Hi Tim,
I've started some load tests to be able to evaluate Couchbase, so this means i moved Couchbase server on the same machine as the load test client. I tried to use a pool of couchbase clients always connected to the server, and after a while the test failed because couldn't create more connections. From my initial research i seen that couchbase client is managing his own connection can you tell me how this is done or what are the policies of shutting down connections ? Can you point me to some best practices for couchbase client connectivity ?
Comment by Tim Smith (Inactive) [ 14/Mar/13 ]
I will handle that question about connection pooling, etc., separately, since it is unrelated to this bug about timeouts.

On this specific bug report, the current state as I understand it is that: curl returns very quickly with the correct results. The Java client times out after 5 seconds without returning the results.

My suggestion is to set up a very simple Java client that performs a single view query, and to get a packet capture of both curl and the Java client, to identify what the two are doing differently and why the curl succeeds but the Java client fails.
Comment by Jahangir Zinedine [ 27/Mar/13 ]
Hi Michael,

Any update on this issue?
I faced the same issue and here is my environment:
JRuby 1.7.3
Oracle JDK 1.7(with 1.6 the same thing happened)
Couchbase 2.0.1
Java SDK 1.1.3(with 1.1.4 the same thing happened)
And latest netty and http-core and other jar files.

Appreciate your help.

Thanks,
Jani
Comment by dragos [ 01/Apr/13 ]
Hi All,
Finally after a long time i found the workaround needed for the java client to not timeout. As has been suggested by the Couchbase team to use a host name for the Couchabase server i applied the same fix but to the client machine. Meaning on the client machine were the java Couchbase client was timing out ( in Amazon VPC ) we added a hostname for the Couchbase server. Our test URI from http://10.0.7.164:8091/pools has become http://couchbase:8091/pools. This workaround is also fixing a 10 seconds connectivity for the client.

On Ubuntu the hostname file is at: /etc/hosts
On Windows 7 the hostname file is at: C:\Windows\system32\drivers\etc\hosts

Example of host name entry in client machine:
10.0.7.146 couchbase
Comment by Michael Nitschinger [ 02/Apr/13 ]
Thanks very much dragos for tracking this down!

Does this mean that in your environment after the change, you're not seeing any performance impact anymore? Everything works as expected and performant?

Thanks!
Comment by dragos [ 02/Apr/13 ]
Hi Michael,
In this moment the client does not time out and the time taken by java couchbase client to connect takes around 1.5 seconds from 10 seconds which we can consider a normal behavior. As for performance with this workaround we can continue the POC and evaluate the performance. As a first impression i can say that we see a good speed but i will keep my reservation until final POC is tested and we can achieve a full load test.

What i want to emphasize that this is a welcomed workaround but it is not a solution on a longer term for our cloud production site. In a scenario where we need to create new machines and maybe new couchbase clusters it involves manual or specific automatic changes to the hosts file and of course extra managing of this. Bottom line is that we are waiting the fix on the couchbase java sdk 1.1.5 which will make this workaround obsolete.

Also i feel the need to mention that this workaround was found in collaboration with the Couchbase team which provided the basic information for the workaround to be found, so thank you guys.
Comment by Michael Nitschinger [ 02/Apr/13 ]
Hi Dragos,

I'm not sure if there is a thing that we can fix inside the client when this is a DNS/hostname resolution issue with the OS and amazon?

We would definitely need to investigate further, but all that the client does is try to use the hostname/IP provided during bootstrap. If nothing comes back or the host can't be found, we can't do much about it. Anyway, I think we need to do some more investigation here to really see whats the problem.
Comment by aboyev [ 09/Sep/13 ]
Hello, Michael!
I have the same problem. Here is my setup:
- Couchbase 2.1.1
- Java SDK 1.1.9
- Java 1.6.0_27 (and also tried 1.7)
[ERROR] getView | Exception while doing getView: Timed out waiting for operation
[ERROR] Exception while doing getView: I/O reactor has been shut down
It surely has something to do with the client, because when I launch second client it works properly for a while.
Also, it seems to happen under high loads more frequently
Comment by gaurang jhawar [ 24/Oct/13 ]
How did u change the "Meaning on the client machine were the java Couchbase client was timing out ( in Amazon VPC ) we added a hostname for the Couchbase server. Our test URI from http://10.0.7.164:8091/pools has become http://couchbase:8091/pools. This workaround is also fixing a 10 seconds connectivity for the client." in AWS .. I have EC2 instance for which i want to set the hostname and connect through it like you did... Can you tell me the exact steps for that
Comment by Michael Nitschinger [ 25/Oct/13 ]
Hi Gaurang,

are you also exhibiting a delay on connect? Or during operations? I think these two may be related but should be looked at separately and defined. What environment, client, workload and so on are you running?
Comment by gaurang jhawar [ 25/Oct/13 ]
My env is JDK 1.7.0_09
Client is my local machine ...

Couchbase server installed on aws ..

Reconnecting due to failure to connect to {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
java.net.ConnectException: Connection timed out: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:423)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:261)
at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:288)
2013-10-24 22:14:01.893 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}, attempt 1.
2013-10-24 22:14:05.903 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
2013-10-24 22:14:09.632 INFO com.couchbase.client.ViewConnection: Added 172.31.9.124 to connect queue
Comment by Michael Nitschinger [ 25/Oct/13 ]
One thing to note here is that this is not a good idea. You have "the internet" between your computer (client) and the server which adds latency. Can you try running the application also in an AWS instance and see if the error goes away?
Comment by gaurang jhawar [ 25/Oct/13 ]
I gave the elastic amazon IP in the url and also tried what dragos tried but that also didn`t work for me .. I made ALL TCP ports on firewall ON ... And I am not sure why it reconnects to the IP or fails to connect and load data in the bucket.

But its the same case as dragos ... for production that`s not viable .. since not everything can be on AWS.

Also I can connect to the same instance via telnet and http(browser) .. I don`t know why I am facing this issue with Java 1.7.

"2013-10-24 22:14:01.893 WARN com.couchbase.client.CouchbaseConnection: Closing, and reopening {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}, attempt 1.
2013-10-24 22:14:05.903 INFO com.couchbase.client.CouchbaseConnection: Reconnecting {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
2013-10-24 22:14:09.632 INFO com.couchbase.client.ViewConnection: Added 172.31.9.124 to connect queue
2013-10-24 22:14:09.632 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
in INIT **************.
Here is the entity set : users
2013-10-24 22:14:09.868 WARN com.couchbase.client.CouchbaseConnection: Node expected to receive data is inactive. This could be due to a failure within the cluster. Will check for updated configuration. Key without a configured node is: 0.
30 Seconds: Load is in progress
2013-10-24 22:14:09.993 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-24 22:14:12.037 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@6dae04e2
2013-10-24 22:14:12.037 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to failure to connect to {QA sa=172.31.9.124/172.31.9.124:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
java.net.ConnectException: Connection timed out: no further information"

NOTE here it tried to reconnect to the client and says its inactive and I am not sure if this is a firewall issue or something else.
Comment by Michael Nitschinger [ 25/Oct/13 ]
Can you provide debug log information for the connect process? Maybe this helps pinning down where the issue is. (you can read here how to do it if you don't know already) :http://nitschinger.at/Logging-with-the-Couchbase-Java-Client

thanks
Comment by gaurang jhawar [ 25/Oct/13 ]
http://s000.tinyupload.com/index.php?file_id=46832814372422198765
Comment by Michael Nitschinger [ 25/Oct/13 ]
Thanks, can you also share the code you use?

In the code it looks like you are creating the CouchbaseClient object twice.
Please note that the bootstrapping succeeds (!) it is different from the ticket reported here.

Are you sure port 11210 is open from client to server? It looks like something is blocking it, http seems to work fine.
Comment by gaurang jhawar [ 25/Oct/13 ]
http://s000.tinyupload.com/index.php?file_id=34200463437173886967



ya the port is opened.

http://s000.tinyupload.com/index.php?file_id=06508245784961590671

Both inbound / outbound on aws as well as my local machine firewall ports are open
Comment by gaurang jhawar [ 26/Oct/13 ]
Any updateS?
Comment by couchbase-user [ 27/Oct/13 ]
This is a nightmare, can't believe no solution has been found yet for over a year now.....
Comment by Michael Nitschinger [ 28/Oct/13 ]
Hi couchbase-user, are you also running app/db over the internet or on a specific ec2 environment?
Comment by couchbase-user [ 28/Oct/13 ]
hey, actually I'm running them all on a local small network.
I am using ubuntu 12.4 on all machines and running java application on headnode to get view data and store them locally.
Comment by Michael Nitschinger [ 28/Oct/13 ]
Hm, I wonder why you are hitting this, I'm not sure its related.. do you get the same exception as the top of the ticket here? Can you provide debug logs so we can see at which point it is happening?

Maybe we need to tweak socket timeouts.
Comment by couchbase-user [ 28/Oct/13 ]
hey, here is my execution log:

28-Oct-2013 09:29:09 com.couchbase.client.CouchbaseProperties setPropertyFile
INFO: Could not load properties file "cbclient.properties" because: File not found with system classloader.
2013-10-28 09:29:09.618 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.100:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:09.620 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.101:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:09.620 INFO com.couchbase.client.CouchbaseConnection: Added {QA sa=/192.168.0.102:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-10-28 09:29:14.635 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@7f09fd93
2013-10-28 09:29:14.635 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@79a5f739
2013-10-28 09:29:14.636 INFO com.couchbase.client.CouchbaseConnection: Connection state changed for sun.nio.ch.SelectionKeyImpl@68e6ff0d
2013-10-28 09:29:14.656 INFO com.couchbase.client.ViewConnection: Added Cloud-assistant.local to connect queue
2013-10-28 09:29:19.663 INFO com.couchbase.client.ViewConnection: Added 192.168.0.102 to connect queue
2013-10-28 09:29:19.666 INFO com.couchbase.client.ViewConnection: Added 192.168.0.100 to connect queue
2013-10-28 09:29:19.667 INFO com.couchbase.client.CouchbaseClient: viewmode property isn't defined. Setting viewmode to production mode
2013-10-28 09:29:19.788 INFO com.couchbase.client.http.AsyncConnectionManager: Opening new Couchbase HTTP connection
2013-10-28 09:29:19.794 INFO com.couchbase.client.http.AsyncConnectionManager$ConnRequestCallback: /192.168.0.102:8092 - Session request successful
2013-10-28 09:29:24.815 ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out: [192.168.0.102/192.168.0.102:8092]
Exception in thread "main" java.lang.RuntimeException: Timed out waiting for operation
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:67)
    at com.couchbase.client.CouchbaseClient.getView(CouchbaseClient.java:483)
    at HelloWorld.Export_Couchbase_View(HelloWorld.java:85)
    at HelloWorld.main(HelloWorld.java:287)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
    at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:85)
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:74)
    at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:64)

I thought this is exactly the same as the error at the top of this thread....
Comment by Michael Nitschinger [ 28/Oct/13 ]
In your logs I can see that 192.168.0.101 is changed to Cloud-assistant.local. Could it be that this is a DNS issue in your environment?
Also please make sure that ports 8091, 8092 (!) and 11210 are reachable on each node.

Both the socket and connection timeout for views are set to 5 second which is already "quite high" if you want to run a real workload over it.
Comment by couchbase-user [ 28/Oct/13 ]
I'm not sure what the DNS has to do with it or how to check it, something to note here is that sometimes it works fine (very rare).
I have also just tried to change the switch but still the same problem.
I'm working on investigating the DNS and firewall issue. thanks
Comment by Michael Nitschinger [ 28/Oct/13 ]
Okay, if you go directly to the view url, how long does it take you to? like with curl... so we can see how far you are over the 5 seconds and if you are under it then there's something else going on.
Comment by couchbase-user [ 28/Oct/13 ]
I have added different servers names to hosts files in all servers and disabled the firewall on all of them and now it seems to work fine.
Thanks a lot for your help Michael.
Comment by Michael Nitschinger [ 28/Oct/13 ]
perfect, good to know it works now.
Comment by gaurang jhawar [ 28/Oct/13 ]
Any updates on my issue Michael Nitschinger. Thanks
Comment by Michael Nitschinger [ 29/Oct/13 ]
Since we fixed the suspected issue with "couchbase-user" in kind of the same way,
ca you first verify that

- All ports are reachable from all the boxes (11210, 8091, 8092)
- DNS settings are configured properly, this is especially important in EC2

If you look through the thread here, some of the people reporting this thing fixed once they made sure their environmental settings were okay. Can you double check?
Comment by gaurang jhawar [ 31/Oct/13 ]
Couchbase-user how did u configure hostname . can u tell me a step by step approach .. dont give me links to couchbase documentation that didn`t help ..

Michael .. the ports are open I am able to ping .. it connects via java and everything but the timeout is quite frequent ..



C:\Users\gaurang\Downloads>tcping 54.219.141.78 8091

Probing 54.219.141.78:8091/tcp - Port is open - time=21.692ms
Probing 54.219.141.78:8091/tcp - Port is open - time=19.357ms
Probing 54.219.141.78:8091/tcp - Port is open - time=15.414ms
Probing 54.219.141.78:8091/tcp - Port is open - time=16.225ms

Ping statistics for 54.219.141.78:8091
     4 probes sent.
     4 successful, 0 failed.
Approximate trip times in milli-seconds:
     Minimum = 15.414ms, Maximum = 21.692ms, Average = 18.172ms

C:\Users\gaurang\Downloads>tcping 54.219.141.78 11210\

Probing 54.219.141.78:11210/tcp - Port is open - time=18.236ms
Probing 54.219.141.78:11210/tcp - Port is open - time=23.513ms
Probing 54.219.141.78:11210/tcp - Port is open - time=1218.881ms
Probing 54.219.141.78:11210/tcp - Port is open - time=14.712ms

Ping statistics for 54.219.141.78:11210
     4 probes sent.
     4 successful, 0 failed.
Approximate trip times in milli-seconds:
     Minimum = 14.712ms, Maximum = 1218.881ms, Average = 318.836ms

Comment by CathodTech [ 13/Dec/13 ]
We have been facing the same issue for couples of months - with Couchbase on RHEL, and we have just found out the solution that works for us.

We have 2 Tomcat nodes running; 1 with Java 1.7 and another with 1.6. Couchbase was deployed on another 4 servers in the same network. We simply defined all Couchbase nodes in /etc/hosts file in both Tomcat nodes as you normally would (with x.x.x.x hostname), then it magically works for us - without having to restart Tomcat, nor Couchbase nodes.
Comment by Michael Nitschinger [ 13/Dec/13 ]
Thanks for pointing this out, I'll see if I can get it into the official docs to aid other people. To me this really looks like java-related and network-related. Interestingly only a few people are hitting it.
Comment by CathodTech [ 14/Dec/13 ]
For the issue I faced, it looks more like Java API related. Perhaps, something to do with DNS (e.g., something like gethostbyname() / gethostbyaddr()), and how the API handles the case when IP has no hostname. Note that we only use IP address to add nodes into Couchbase cluster and Java on other servers also connect to Couchbase with IP address, not hostname.

From the trace, we found that java code has no problem connecting (via connection pool) with Couchbase nodes that has IP/hostname pair in /etc/hosts. But for Couchbase nodes that were not defined in /etc/hosts, it attempted to connect, then the connection dropped, and returned timeout issue.

My team also found that later when he restarted java to re-deploy, Couchbase connection attempts at start-up were much faster (on both Java 1.6 and 1.7).

For this issue, we didn't face it when deploying Java and Couchbase on the same server as, perhaps, it is common to have "127.0.0.1 localhost" defined in /etc/hosts.

We hope that the solution we found would help you and others as well. Not sure if others are facing the same scenario, though.
Comment by David Borsos [ 18/Dec/13 ]
I've ran into a very similar/the same situation with the latest couchbase (2.2.0) server and java client (1.2.3), I thought I share my experiences.

The problem I faced was when trying to call CouchbaseClient.getView(). This apparently goes to one node in the couchbase cluster and queries some view metadata over HTTP (roughly). However I got "ERROR com.couchbase.client.ViewNode$EventLogger: Connection timed out" and that pretty much killed my client there. Seeing the solutions here with the hosts file workaround, I tried that and it indeed worked, but it's not really a suitable solution for us, so I decided to see into this a little bit more. Here is what I found.

(Disclaimer: this worked for me, and although I found other ways around this they may not work for you, and I did these attempting to solve this particular issue and didn't really care about anything else these changes may break, so be careful!)

(0) Baseline, the environment specific thing that actually causes the issue is that call java api call InetAddress.getHostFromNameService(InetAddress addr, boolean check) returns very slowly if there's no hostname to be found for a given IP. I have no idea why is that in my environment, I suppose could be because of a bunch of different reasons. Slow here means that it takes 4-5 secs for me for this call to return with an unknown hostname

This comes into the picture when the couchbase client tries to get a view. It uses Apache Http client to do this, and the following things happen:
(1) When the request for the view gets created, eventually we get to BaseIOReactor.execute which instantiates SessionHandle, and registers the time when this happened (this will cause the timeout)
(2) Somewhere in the request processing, we actually DO hit getHostFromNameService. More specifically this happens because a RequestTargetHost interceptor runs and this injects the "Host" header into the HTTP request.
(3) some time after (2) we hit a timeout check which compares the timestamp recorded in (1) and the current time, obviously resulting in a timeout and the error in the client if (2) takes a long time as indicated

The ideal solution would obviously be to fix whatever is wrong with my DNS configs, but I'm not (yet) sure what's wrong (running Ubuntu 13.10 btw). And also make sure that our live environments won't have the same issue...

One workaround I found that might be generally usable is to override the JVM's DNS resolver, by starting the client with: -Dsun.net.spi.nameservice.provider.1=dns,sun This worked for me, but I don't know what side-effects it may have (generally this may not be a great idea). Additionally this resulted in a significant increase of startup speed as the couchbase client seems to do a lot of ip->hostname lookups when it's starting (are these really necessary?)

Also I'd like to ask from the Couchbase folks if it's really necessary to do this reverse DNS lookup for every HTTP request. I cloned your client from git, and removed the RequestTargetHost interceptor from these calls; that also solved the timeout problem, and all seems to work fine. I didn't do exhaustive testing on it, and this, too, can be environment specific - running my couchbase cluster in local VMs at the moment.

Thanks.
Comment by Michael Nitschinger [ 19/Dec/13 ]
Hi David,

thanks for the heads-up, that gives me a better idea of whats going on. I basically revamped the whole view connection management for the next 1.3 release which hopefully should address this to. Would you want to try it out based on a preliminary build?

The thing is that you can't remove the RequestTargetHost, but it only does the DNS lookup if HTTP.TARGET_HOST is not set, but we can infer that from our configuration. So the new code basically sets it before the request is put in the queue, once we've determined which node to ask, then it should be skipped.

I already have that locally, so if you want to try it out we can quickly see if it fixes the issue.
Comment by Michael Nitschinger [ 19/Dec/13 ]
The issue will be fixed with the overall changeset of jcbc-388, which will appear in - as it stands now - 1.3, slated for jan 2014.
Comment by David Borsos [ 19/Dec/13 ]
Michael,

That sounds great. I'd be happy to help you verify your new code; send it over or let me know how can I get it, I'll switch to it and see if it solves the problem. If we can get the new version in early January (I saw it's scheduled for the 7th) that's cool too, we won't be doing much during the holidays anyway :)

Thanks
Comment by Sergii [ 19/Dec/13 ]
Hi Michael,

i have run into the same issue with couchbase 2.2 on mac - Java 1.7

Can i have the changed jar to try the fix?

Thanks
Comment by Michael Nitschinger [ 31/Dec/13 ]
A fix for this has been merged into master and will be available in the next (1.3.0) release.
Comment by Michael Nitschinger [ 31/Dec/13 ]
The new code minimizes dns lookups to workaround the issue, but the environment still needs to be fixed properly :)
Comment by Oded Hassidi [ 21/Jul/14 ]
Was this fixed, since we are having issues with timeot on Java SDK v1.4.x, and we are using JDK 7?
This is not happen all the time, if I run a thread that runs many request it occur every 2-4 min
Comment by Michael Nitschinger [ 21/Jul/14 ]
This issue happened during startup, are you experiencing issues during startup? Where do you get the timeouts?
Comment by Oded Hassidi [ 21/Jul/14 ]
Thanks for the quick response.
The timeout are not at startup it happens during runtime. every 2-3 minutes I get the followed:

SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
java.lang.RuntimeException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:75)
        at com.couchbase.client.CouchbaseClient.query(CouchbaseClient.java:778)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for operation
at com.couchbase.client.internal.HttpFuture.waitForAndCheckOperation(HttpFuture.java:93)
at com.couchbase.client.internal.ViewFuture.get(ViewFuture.java:65)
at com.couchbase.client.internal.ViewFuture.get(ViewFuture.java:49)
at com.couchbase.client.internal.HttpFuture.get(HttpFuture.java:72)
... 45 more




[JCBC-487] com.couchbase.client.protocol.views.Qery key construct Created: 07/Jul/14  Updated: 15/Jul/14  Resolved: 15/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: Views
Affects Version/s: 1.4.1, 1.4.3
Fix Version/s: None
Security Level: Public

Type: Bug Priority: Major
Reporter: reinerRubin Assignee: Michael Nitschinger
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   
Hello.
There is a little bug in query.setKey. When

String login = "0987654321";
Query query = new Query();
query.setKey(login);

query will be with ?key=987654321 not 0987654321.

It's happen in query.quote()
(https://github.com/couchbase/couchbase-java-client/blob/1.4.3/src/main/java/com/couchbase/client/protocol/views/Query.java#L292)


Also in version 1.3.0 in similar situation (key = "0987654321") key will be key=0987654321 and in response
java.util.concurrent.ExecutionException: OperationException: SERVER: bad_request Reason: invalid UTF-8 JSON: {{error,garbage_after_value},\"0987654321\"}

// Excuse my bad English

 Comments   
Comment by Michael Nitschinger [ 07/Jul/14 ]
You should use ComplexKey instead of the string directly.

Try:

query.setKey(ComplexKey.of("0987654321");
Comment by reinerRubin [ 07/Jul/14 ]
>You should use ComplexKey instead of the string directly.
Thanks. It's work.
But I don't sure about setKey(String) behavior.
Comment by Michael Nitschinger [ 15/Jul/14 ]
Yeah, with the string its not so straightforward, thats why we added the complex key. for now please stick to it, it will be easier with 2.0
Comment by reinerRubin [ 15/Jul/14 ]
Ok. Thanks!




[JCBC-440] Inconsistent result of paginated query. Created: 26/Mar/14  Updated: 09/Jul/14

Status: Open
Project: Couchbase Java Client
Component/s: Core, Views
Affects Version/s: 1.3.2
Fix Version/s: .future
Security Level: Public

Type: Bug Priority: Major
Reporter: Vladislav Koroteev Assignee: Michael Nitschinger
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: Red Hat Enterprise Linux Server release 6.3 (Santiago)
Java 7
Couchbase Server Community Edition 2.2.0


 Description   
I get a strange behavior with paginated query. While I paginate on the result set, I get duplicates of some documents. I'm doing the following in my code:

// force start indexing of view
query.setStale(Stale.FALSE);
query.setReduce(true);
ViewResponse respFromReduce = client.query(view, query);
String expectedCountFromScroll = null;
for (ViewRow viewRow : respFromReduce) {
    expectedCountFromScroll = viewRow.getValue();
}
query.setReduce(false);
query.setIncludeDocs(false);
//disable indexing of view from sdk
query.setStale(Stale.OK);
Paginator scroll = client.paginatedQuery(view, query, SCROLL_SIZE);
int docsFromScrollCnt = 0;
while (scroll.hasNext()) {
    ViewResponse response = scroll.next();
    docsFromScrollCnt += response.size();
}

And I see, that docsFromScrollCnt > expectedCountFromScroll.
I think, that while query Couchbase auto-indexing view, so I decide to disable auto-indexing by set viewUpdateDeamon's parameters. I set updateInterval to value, that greater then querying time, updateMinChanges and replicaUpdateMinChanges to zero.

It's not helped, but I noticed following interesting feature. Count of duplicate documents always equals n*SCROLL_SIZE.

How can I avoid duplicate of document from paginated query?

- See more at: http://www.couchbase.com/communities/q-and-a/strange-behavior-paginated-query#sthash.MLFf15Gx.dpuf

 Comments   
Comment by Michael Nitschinger [ 02/Jun/14 ]
Hi,

I need further clarification on the use case to triage this.

- Are you running workload while doing the request? That is, are you updating documents? Keep in mind that even with stale=false, the indexer on the server updates the view in the background and the results can change during the pagination.

- What reduce function do you use?

I'm pretty sure this is because the data is updated in between the calls, remember that your reduce call is more or less "atomic", but if you paginate on results there is no cursor, you are getting the batches with potential indexer updates in between.
Comment by Vladislav Koroteev [ 08/Jul/14 ]
- Are you running workload while doing the request?
Yes. When I get document from result set I change field on which I make view. For example, I have map-function {emit(doc.a, null)} and I changed field "a" of document. But I disable auto-indexing by set viewUpdateDeamon's parameters with REST-API. So I think, that indexing don't started. Also I use stale=OK while paginating result set of view. So, I don't understand why I have such result of view.

- What reduce function do you use?
I use built-in _count function.

-I'm pretty sure this is because the data is updated in between the calls, remember that your reduce call is more or less "atomic"
Before my test, database is not workload. I see, that process of auto-indexing is finished, because I call from web-interface reduce function with stale=FALSE. After that I disable auto-indexing, that's why result of view confused me.
Comment by Michael Nitschinger [ 09/Jul/14 ]
Normally there is an indexer running in the background, how did you disable auto-indexing? Also, if you want stale data you need to use stale=OK (since stale=false will update the index). If you on the other hand need 100% accurate results you need to do stale=false AND PersistTo.MASTER on the write side.
Comment by Vladislav Koroteev [ 09/Jul/14 ]
-how did you disable auto-indexing?

From documentation http://docs.couchbase.com/couchbase-manual-2.2/#automated-index-updates.
I make such http-request "POST http://nodename:8091/settings/viewUpdateDaemon updateInterval=10000000&updateMinChanges=0"

I request view reduce function with stale=FALSE. So I make index is fresh. Then I request view with stale=OK in order to avoid indexing of view, but still get duplicates documents in result set. Another strange feature, that count of duplicates always equals N*SCROLL_SIZE. N is variable in different requests.




[JCBC-473] Flush support Created: 10/Jun/14  Updated: 07/Jul/14  Resolved: 07/Jul/14

Status: Resolved
Project: Couchbase Java Client
Component/s: None
Affects Version/s: None
Fix Version/s: 2.0-dp2
Security Level: Public

Type: New Feature Priority: Critical
Reporter: Michael Nitschinger Assignee: