Couchbase.IO.SendTimeoutExpiredException: The operation has timed out

@xiduan -

The PoolConfiguration.SendTimeout property controls the time the client will wait for a response from the server before raising SendTimeoutExpiredException. There is an example of the configuration on this page..

The real problem is that something between the client and the server is closing the connections. If you always need to timeout and then recreate the connections, you’ll never get the performance expected. I highly suggest you look deeper into what is causing the connections to close in your environment.

-Jeff

1 Like

Hi @jmorris

Thank you very much for the explanation - we will check in depth in between the client and the server.

As you mentioned, if we have constant requests, there shouldn’t be any problem. I believe so as well, because the issue is caused by disconnected connections.

If the SDK is going to re-establish the connection. I assume it will take very short time to complete the new connection for local nodes, am I right?
Is there any estimated practice time for such rebuild connection process?

Regards,
Xi

Looking at your log files it is exactly 5 minutes between when the connection is started until it is disconnected.
(i.e. connection 58e60e3a-4c0d-4114-b8af-ea95e5c89519)

[DEBUG][2018-03-17 14:45:19,933] Couchbase.Authentication.SASL.ScramShaMechanism [8]
Client First Message 10.38.110.11:11210 - 58e60e3a-4c0d-4114-b8af-ea95e5c89519: n,n=cachedata,r=CfPttq8ZOx5epGtpC1E18o8nucvR [U:cachedata|P:

[WARN][2018-03-17 14:50:19,938] Couchbase.IO.ConnectionBase [49]
Handling disconnect for connection 58e60e3a-4c0d-4114-b8af-ea95e5c89519: Couchbase.IO.RemoteHostClosedException: The remote host (10.38.110.11:11210) has gracefully closed this connection.

If you have a multi-node Couchbase cluster I would suggest looking at the log files on those for cluster comms, you might find that node-to-node connections are also being dropped.

As a work to keep the connections active you could set the TCP keep alive to 60 seconds.

@jmorris @clinton1ql

Current we are using single node in the development environment.
We found the cause of the cut-off of the connections that should keep-alive - the apache default setting on keep-alive timeout.
Changing the apache keep-alive timeout setting seems to solve the problem.

Thank you very much for the help!

Regards,
Xi

1 Like