Hi @prasanpd it looks like the WaitUntilReady command is timing out which means that the SDK cannot connect to the cluster for some reason. If you enable logging (https://docs.couchbase.com/go-sdk/current/howtos/collecting-information-and-logging.html - gocb.VerboseStdioLogger() will give you the most information) then it should give you more information about what’s happening and why the SDK can’t connect.
WaitUntilReady was introduced in version 2.1.0 so is applicable to 2.1.6. It’s an operation which you are calling in your code for the above error to contain “OperationID”:“WaitUntilReady”.
Hi @prasanpd could you share a code snippet of how you setup the connection with the SDK and enabling logging? The logging output should appear in your stdout, how do you run your application?
@chvck I’m using go-sdk@2.4.1 with couchbase enterprise@7.0.3. I got the same error at the moment. I’m creating 3 nodes cluster on GCP those nodes reside in the same region and zone.
here is our connection string = couchbase://xx.xx.xx.70,xx.xx.xx.34.xx.xx.xx.253
here is our exception
unambiguous timeout | {“InnerError”:{“InnerError”:{“InnerError”:{},“Message”:“unambiguous timeout”}},“OperationID”:“WaitUntilReady”,“Opaque”:"",“TimeObserved”:30004468042,“RetryReasons”:[“NOT_READY”,“CONNECTION_ERROR”],“RetryAttempts”:243,“LastDispatchedTo”:"",“LastDispatchedFrom”:"",“LastConnectionID”:""}
Hi @Nuttipong_Taechasang it looks like the SDK is totally unable to connect to your cluster, and looking at the logs it looks like it’s trying to connect to port 18091 via KV. Port 18091 is typically the secure management port, which is HTTP so that won’t work.
The logs show the addresses that the SDK is attempting to connect to as MemdAddrs:[34.143.203.70:18091] which is where the 18091 is coming from.
When you create your connection string are you including port 18091 as a part of it?
thanks for the fast response. Exactly SDK cannot connect to the Cluster. To be honest - where is 18091 port come from, and I didn’t prefer to use SSL at this time but somehow as you can see from the log. I didn’t set up anything though.
The way that I created my cluster and connect to the cluster is very simple as following below
Add node 2 and 3 with same port and difference ip
./couchbase-cli server-add -c ${public_ip}:8091
–username xxx
–password xxx
–server-add ${private_ip}
–server-add-username xxx
–server-add-password xxx
–group-name “${group_name}”
–services data,index,query
use Go-SDK to connect my cluster then get the error above
Hi @Nuttipong_Taechasang your connection string looks like it has a typo "couchbase://xx.xx.xx.70,xx.xx.xx.34.xx.xx.xx.253” should probably be "couchbase://xx.xx.xx.70,xx.xx.xx.34,xx.xx.xx.253”. Although how that’s resulting in only xx.xx.xx.70:18091 getting passed through I’m currently not sure about.
@Nuttipong_Taechasang - port 18091 is the SSL/TLS port for connecting to Couchbase and using TLS is a requirement for connecting to Capella whereas it can be disabled for connecting directly to a Couchbase Server.
In addition to @chvck mention of a typo, you can also use couchbases:// (note the s at the end as in https) to denote a TLS connection.
If you’re still seeing a timeout issue connecting to Capella then I would encourage you to double-check your “allowed IP” list. You can use 0.0.0.0/0 to allow any IP to connect but this is definitely insecure and not recommended for a production environment.
Finally, you could also use our SDK Doctor as a way of validating connectivity from wherever you are running your application.
Hi Perry, thanks for your suggestion. Sorry for my mistake about typo but my actual code is valid this is just a copy mistake on the reply.
My intention is connecting with insecure mode, so how can I disable TLS one?
Here is my use case,
I’m currently not using Capella one. I’ve been created a Couchbase cluster with bare metal on my own over the GCP by 3 nodes cluster in the same region and zone.
Note: Diagnostics can only provide accurate results when your cluster
is in a stable state. Active rebalancing and other cluster configuration
changes can cause the output of the doctor to be inconsistent or in the
worst cases, completely incorrect.
19:56:22.033 INFO Parsing connection string couchbase://34.143.137.223,34.143.221.13,34.143.150.2/default
19:56:22.034 INFO Connection string identifies the following CCCP endpoints:
19:56:22.034 INFO 1. 34.143.137.223:11210
19:56:22.034 INFO 2. 34.143.221.13:11210
19:56:22.034 INFO 3. 34.143.150.2:11210
19:56:22.034 INFO Connection string identifies the following HTTP endpoints:
19:56:22.034 INFO 1. 34.143.137.223:8091
19:56:22.034 INFO 2. 34.143.221.13:8091
19:56:22.034 INFO 3. 34.143.150.2:8091
19:56:22.034 INFO Connection string specifies bucket default
19:56:22.034 INFO Performing DNS lookup for host 34.143.137.223
19:56:22.034 INFO Performing DNS lookup for host 34.143.221.13
19:56:22.034 INFO Performing DNS lookup for host 34.143.150.2
19:56:22.034 INFO Attempting to connect to cluster via CCCP
19:56:22.034 INFO Attempting to fetch config via cccp from 34.143.137.223:11210
19:56:22.179 ERRO Failed to fetch configuration via cccp from 34.143.137.223:11210 (error: EOF)
19:56:22.179 INFO Attempting to fetch config via cccp from 34.143.221.13:11210
19:56:22.326 ERRO Failed to fetch configuration via cccp from 34.143.221.13:11210 (error: EOF)
19:56:22.326 INFO Attempting to fetch config via cccp from 34.143.150.2:11210
19:56:22.463 ERRO Failed to fetch configuration via cccp from 34.143.150.2:11210 (error: EOF)
19:56:22.463 INFO Attempting to connect to cluster via HTTP (Terse)
19:56:22.463 INFO Attempting to fetch terse config via http from 34.143.137.223:8091
19:56:22.540 ERRO Failed to fetch terse configuration via http from 34.143.137.223:8091 (error: http error (status code: 404))
19:56:22.540 INFO Attempting to fetch terse config via http from 34.143.221.13:8091
19:56:22.613 ERRO Failed to fetch terse configuration via http from 34.143.221.13:8091 (error: http error (status code: 404))
19:56:22.613 INFO Attempting to fetch terse config via http from 34.143.150.2:8091
19:56:22.690 ERRO Failed to fetch terse configuration via http from 34.143.150.2:8091 (error: http error (status code: 404))
19:56:22.690 INFO Attempting to connect to cluster via HTTP (Full)
19:56:22.690 INFO Failed to connect via HTTP (Full), as it is not yet supported by the doctor
19:56:22.690 INFO Selected the following network type:
19:56:22.690 ERRO All endpoints specified by your connection string were unreachable, further cluster diagnostics are not possible
19:56:22.690 INFO Diagnostics completed
Summary:
[ERRO] Failed to fetch configuration via cccp from 34.143.137.223:11210 (error: EOF)
[ERRO] Failed to fetch configuration via cccp from 34.143.221.13:11210 (error: EOF)
[ERRO] Failed to fetch configuration via cccp from 34.143.150.2:11210 (error: EOF)
[ERRO] Failed to fetch terse configuration via http from 34.143.137.223:8091 (error: http error (status code: 404))
[ERRO] Failed to fetch terse configuration via http from 34.143.221.13:8091 (error: http error (status code: 404))
[ERRO] Failed to fetch terse configuration via http from 34.143.150.2:8091 (error: http error (status code: 404))
[ERRO] All endpoints specified by your connection string were unreachable, further cluster diagnostics are not possible
Just to confirm - you are able to successfully connect when using just xx.xx.xx.70 in your connection string but not when you have the others added?
Another common situation we see is when trying to connect to a cloud provider from outside their network. The problem is that cloud instances often have different IP addresses inside than they do outside, and so when a Couchbase SDK tries to connect it gets the inside addresses and is unable to reach them. This can be worked around by using our alternate address feature, but you might also want to try connecting from within the same virtual network as your Couchbase cluster is running (which is what you’d want to do for the best performance anyway)
From the logs you just posted, it looks like there is some connection issue between wherever you are running the sdk doctor and the Couchbase nodes. Can you confirm that the necessary firewall and ports are open?
Here is the SDK docter log from what I tried with a single node in this case I’m able to connect the cluster properly.
Summary:
[WARN] Your connection string specifies only a single host. You should consider adding additional static nodes from your cluster to this list to improve your applications fault-tolerance
[ERRO] Failed to fetch configuration via cccp from 34.143.229.224:11210 (error: EOF)
[ERRO] Failed to fetch terse configuration via http from 34.143.229.224:8091 (error: http error (status code: 404))
[ERRO] All endpoints specified by your connection string were unreachable, further cluster diagnostics are not possible
Ahh ok, so that confirms what I was referring to with the alternate address:
When a Couchbase SDK instance connects to that cluster, it is connecting over a public IP address but is getting told about the nodes of the cluster via their private addresses (the nodes themselves don’t know their public address). When the SDK gets told about those nodes, it is then unable to connect to them because it can’t reach the internal network from outside.
So - you either need to map the public address onto each node (via the alternate address feature) OR you need to connect to the cluster from within the Cloud provider. I would strongly recommend this one as the performance will be much better.
Parsing connection string