Java Client, v2.3.1, random KeepAliveRequest aborted?

siriousje · July 22, 2016, 12:45am

Hiya,

I’m using the Java client, version 2.3.1, for a webserver that uses couchbase to return rendered json documents. It works like a charm, but every now and then I the below stacktrace in my log. I know it’s INFO, but I wonder why the stacktrace and if there’s anything I can do to prevent it or if I’m doing anything wrong. The environment is made using a 30 seconds keepAliveInterval:

CouchbaseEnvironment environment = DefaultCouchbaseEnvironment.builder()
                .keepAliveInterval(30000)
                .build();

Btw, the versions of rxjava and netty are 1.1.7 and 3.10.5.Final respectively (due to other dependencies)

2016-07-22 00:21:24,600 INFO cb-io-1-3 com.couchbase.client.core.endpoint.AbstractGenericHandler - [/127.0.0.1:8092][ViewEndpoint]: Connection reset by peer: An established connection was aborted by the software in your host machine
2016-07-22 00:21:24,602 INFO cb-io-1-3 com.couchbase.client.core.endpoint.AbstractGenericHandler - Exception thrown while cancelling outstanding operation: KeepAliveRequest{observable=rx.subjects.AsyncSubject@628fc2f3, bucket='null'}
rx.exceptions.OnErrorNotImplementedException: Request cancelled in-flight.
        at rx.internal.util.InternalObservableUtils$ErrorNotImplementedAction.call(InternalObservableUtils.java:386)
        at rx.internal.util.InternalObservableUtils$ErrorNotImplementedAction.call(InternalObservableUtils.java:383)
        at rx.internal.util.ActionSubscriber.onError(ActionSubscriber.java:44)
        at rx.observers.SafeSubscriber._onError(SafeSubscriber.java:152)
        at rx.observers.SafeSubscriber.onError(SafeSubscriber.java:115)
        at rx.subjects.SubjectSubscriptionManager$SubjectObserver.onError(SubjectSubscriptionManager.java:229)
        at rx.subjects.AsyncSubject.onError(AsyncSubject.java:119)
        at com.couchbase.client.core.endpoint.AbstractGenericHandler.handleOutstandingOperations(AbstractGenericHandler.java:486)
        at com.couchbase.client.core.endpoint.AbstractGenericHandler.exceptionCaught(AbstractGenericHandler.java:449)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:230)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:209)
        at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:416)
        at com.couchbase.client.deps.io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:79)
        at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:144)
        at com.couchbase.client.deps.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
        at com.couchbase.client.deps.io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:223)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:230)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:209)
        at com.couchbase.client.deps.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:230)
        at com.couchbase.client.deps.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:209)
        at com.couchbase.client.deps.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:950)
        at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:87)
        at com.couchbase.client.deps.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:485)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:399)
        at com.couchbase.client.deps.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:371)
        at com.couchbase.client.deps.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
        at com.couchbase.client.deps.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.couchbase.client.core.RequestCancelledException: Request cancelled in-flight.
        ... 24 more

siriousje · July 22, 2016, 2:37am

so far it looks like the issue goes away if I downgrade the client to 2.2.7, haven’t seen this exception for a while now using 2.2.7 while it is back in seconds if I use 2.3.0 or 2.3.1

simonbasle · July 22, 2016, 8:14am

@siriousje there is a bug in 2.2.6 and 2.2.7 where keep alive requests are not sent… so that would explain why you don’t see the exception anymore, but it’s not a fix

siriousje · July 22, 2016, 9:57am

ah yes, well, I didn’t consider going back to an older version a fix more that there’s something wrong with the current one. I’m not familiar with the 2.x client much (since this week actually) but the message “bucket=‘null’” seems weird to me, as I definitely connect to a specific bucket (and everything works, it’s just this INFO level log every few seconds)