We have made cluster in which 3 nodes set up and give 18GB memory [6GB per node] in our Live environment with following configuration:
- Couchbase Server Community Edition 7.0.2 build 6703
- OS: CentOS Stream release 8 (kernel version: 4.18.0-529.el8.x86_64)
- Couchbase Java SDK version 3.3.4
- Type of bucket ephemeral
We are getting below error on the time of putting the data in bucket of Couchbase.
2025-02-13 15:21:52,707 ERROR [com.test.common.util.MembaseUtil] (default task-421) Exception in makeMembaseCacheEntrySynch(key: YnbMy6iWahDYwfBYTtW6QWm1XA66a56QopK5xFAffBM=, expiryTime:600000, bucketname:myBucket):: com.couchbase.client.core.error.AmbiguousTimeoutException: UpsertRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0xd26c099e00000001","idempotent":false,"lastChannelId":"D26C099E00000001/000000009E496673","lastDispatchedFrom":"10.10.0.4:57046","lastDispatchedTo":"11.11.11.90:11210","reason":"TIMEOUT","requestId":212599234,"requestType":"UpsertRequest","retried":19,"retryReasons":["KV_TEMPORARY_FAILURE"],"service":{"bucket":"myBucket","collection":"_default","documentId":"YnbMy6iWahDYwfBYTtW6QWm1XA66a56QopK5xFAffBM=","errorCode":{"description":"Temporary failure. Try again","name":"ETMPFAIL"},"opaque":"0xcac031c","scope":"_default","type":"kv","vbucket":74},"timeoutMs":5000,"timings":{"dispatchMicros":100,"encodingMicros":38,"totalDispatchMicros":2149,"totalMicros":5007504}}
at com.couchbase.client.java.AsyncUtils.block(AsyncUtils.java:51)
at com.couchbase.client.java.Collection.upsert(Collection.java:431)
at com.test.common.util.MembaseUtil.makeMembaseCacheEntrySynch(MembaseUtil.java:332)
at com.test.newApp.security.util.AuthorizationUtil.putUSPInMembase(AuthorizationUtil.java:301)
at com.test.newApp.security.filter.SecurityFilter.processRequest(SecurityFilter.java:143)
at com.test.newApp.security.filter.SecurityFilter.processFilter(SecurityFilter.java:78)
at com.test.newApp.security.filter.BaseFilter.doFilter(BaseFilter.java:119)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:60)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:132)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:85)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:72)
at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.Exception: The above exception was originally thrown by another thread at the following location.
at com.couchbase.client.core.msg.BaseRequest.cancel(BaseRequest.java:184)
at com.couchbase.client.core.msg.Request.cancel(Request.java:70)
at com.couchbase.client.core.Timer.lambda$register$2(Timer.java:157)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715)
at com.couchbase.client.core.deps.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790)
at com.couchbase.client.core.deps.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503)
at com.couchbase.client.core.deps.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
... 1 more
We have checked in web console at the time of above ERROR, but we did not get any alert or issue, and all bucket status are running. We have checked Couchbase logs but did not find any root cause regarding above error. For logs Couchbase logs
Why are we getting this type of ERROR even though Couchbase server was up?
Let us know any other details require or need to check any configuration to troubleshoot the issue.
Thanks in advance.