I am trying docker for mac (e.g. native docker) and I have one issue with running local code against my couchbase node… We have a cluster of 1 node container which is registered using its internal private IP - this is fine for all our containers as they can connect to this private ip.
But if I need to run some code from my developer machine, like in my IDE, even if I specify as a bootstrap node to connect to 127.0.0.1:8091, then it fails because it tries to connect to private_ip:8091, private_ip:8092, private_ip:8093, private_ip:11210, …
I understand it is because the cluster then make the client aware of its configuration and the client start using this. Is there a way to tel the Java SDK to ignore this information and just use the ip provided in its environment setup, so that it will work?
No, the client will always treat the cluster configuration as the source of truth, the stuff you can override is just during bootstrap. So you need to configure the cluster to expose the ports properly and/or make sure docker exposes them the right way.
I think @arungupta can help you get set up with docker for sure.
@lbertrand here is a sample github repo that shows to start a Couchbase Docker container and run a simple Java application that stores a JSON document to it.
The containers are running in a private network, using docker compose so each container has its own IP.
From my local laptop I can connect to the couchbase cluster by using localhost:8091, but then, the client get the node hostname which is the private IP and will fail to connect to the other port.
During bootstrap, a cluster node will give you the map to all of the other cluster nodes. It’ll need to be able to resolve these hostnames or IP addresses. To do that with Docker, I believe you’ll need to configure with multi-host networking.
As long as the place where your client library is executing has IP or hostname access to how the Couchbase server nodes know themselves and the ports are correctly available from the docker hosts, the client will be able to bootstrap across the cluster.
The main reason we have this ‘cluster map’ is that in Couchbase, while running the client knows exactly which node to go to for certain operations (to get the best performance) and the client will automatically adjust where it sends requests when the topology changes. Since the cluster tells the client which hosts have which services dynamically, you will need the client and cluster nodes to have a consistent understanding of hostnames or IPs.
Hope that helps! I think @mnunberg may have a config that may help with multiple nodes, but I’m not sure of that.
All make sense and I understand how this is supposed to work…
We run our code in containers too so, yes, the other containers have access nicely to couchbase node in this case.
It is just that if I need to run some java code from my IDE for example, in this case I can’t as the IP is private from the macBook point of view… I can make it work by running on a docker-machine as I can route all my private IP range to the public IP of the docker-machine… This is how I was running locally so far…
But we are trying with docker native for MacOS and this is where it fails because I do not have a public IP to route traffic to… So can’t therefore connect to the private IP of the couchbase docker container… This is what lead to my question about possibility to turn the bootstrapping off so that the client just use whatever I gave it in the list of nodes - this is really specific for just local testing.
There is more involved in bootstrapping than the list of nodes. The cluster map tells the client what services are on which nodes and for the KV service, which nodes are responsible for vbuckets. With some code modifications it’d be possible to fake this, but it’d be unusual to do so. The client and cluster nodes have regular communication about health and topology changes. That would need to be accounted for as well.
One question: if this is for local testing, why not just test with a single node? A single node is functionally the same thing as a multi-node cluster and I would recommend any situational testing such as failed nodes to be done with a network configuration similar to how you’d run production.
It is a single node - but the server is node by its private IP in the docker network… (see original post explaining this)
All is fine when running in containers, as I said, this is just a downside of using docker native for docker in this case as it is not exposing a bridge i could route traffic to from my local macbook.
Do not read to much into this, I was just looking for a quick easy way to run local code against the private container if possible by forcing the couchbase client to ignore IP given by node but keep using the one given when creating it, in this case 127.0.0.1
Ah, I see @lbertrand. I jumped to a conclusion there without reading earlier in the thread. I assumed you were asking for something more complicated since earlier replies didn’t address it and you said “containers”.
What you are looking to do is pretty simple. You can tell the docker container to take a particular hostname, then configure the Couchbase cluster-of-one with that hostname and finally configure your local OS to be able to resolve that hostname. This is sometimes referred to as split horizon name resolution*.
Steps:
On your Mac, start your docker image mostly as normal, but pass a param for the hostname that includes a “.” since Couchbase requires it. For instance: docker run -d --name couchbase -h "cbtest.local" -p 8091-8094:8091-8094 -p 11210-11211:11210-11211 couchbase. That will start a container named “couchbase” which will let processes in the image resolve “cbtest.local” to itself.
Configure the cluster-of-one from your Mac’s browser. Connect to http://localhost:8091 and on the screen there, make sure you configure the hostname as cbtest.local:
Set up Mac OS to resolve cbtest.local to 127.0.0.1. For instance, sudo vi /etc/hosts and add the line: 127.0.0.1 cbtest.local
Test that this resolves from Mac OS. The official way to do this is with dscacheutil: $ dscacheutil -q host -a name cbtest.local name: cbtest.local ip_address: 127.0.0.1
But you can also just ping the container: ping cbtest.local.
After that, any program you run in your IDE will just need to be pointed to the hostname that both MacOS and Docker will resolve to the same place. I ran this in IntelliJ:
package com.couchbase.client.example;
import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.Cluster;
import com.couchbase.client.java.CouchbaseCluster;
public class Main {
public static void main (String args[]) {
Cluster cluster = CouchbaseCluster.create("cbtest.local");
Bucket bucket = cluster.openBucket("travel-sample");
System.out.println(bucket.get("airline_5209"));
}
}
No hacking of the Java client needed. This is all just “normal” and all services should resolve just fine!
* Technically, split-horizion DNS back in the olden days. It’s the same concept, just done with local resolution instead of DNS in this case.
Unfortunately, I am not sure I can do this directly. Our docker image deployment is automated (like it will be for our normal deployment) and looking at the API http://localhost:8091/pools/default, I can see a hostname setup, like hostname: “private_ip:8091”.
Is this the same as the hostname that will be set during cluster setup or is this something else? If it is different, is there a way to call an API to set it up?
