I want to run multiple instances of Couchbase on the same machine, for automated test purposes, and I am creating a framework to start new instances using Docker. This requires that I forward ports (in docker) to avoid conflicts. Each Couchbase instance is using the default ports (8091,…) locally but the exposed ports on the host (where the client is running) has a different range (20xxx).
This works fine, and I am able to initialize the cluster, and it all works fine via the management web page.
With the .NET SDK I am running into some problems. I am successful in configuring some custom ports, like:
ClientConfiguration.MgmtPort
ClientConfiguration.CapiPort
ClientConfiguration.MemcachedPort
and I am also configuring BucketConfiguration.Port to the same as the MemcachedPort.
I can get the bucket and add documents. But I cannot run queries.
If I run the exact same code, with docker, but forwarding the internal ports to the same range, e.g. 8091:8091 it can all work.
I believe this is because I cannot configure the Query Port from the .NET SDK.
Is there any way to do this or any workaround known?
To my knowledge, this cannot be done. The ports in use are reported to the SDK by the server during the bootstrap phase. Since the server doesn’t know about the port mappings you have in place, it’s reporting its internal port number rather than the external port number. There may be some way to configure the server to work around this, but if it exists I don’t know about it.
My recommendation for running automated tests (and the way we do it at CenterEdge) is to run them entirely within Docker. Not just Couchbase Server, but the tests themselves as well. Then port mapping becomes a non-issue as each container gets an IP and can communicate using their internal port numbers.
As I wanted to run tests that interact with multiple couchbase instances, e.g. tests running on one machine but simulating two back-end systems, I couldn’t find a good solution to run the tests inside the docker container.
I guess it would be possible to spin-up full stacks within two separate docker images and run testing against them, but then I would lose some flexibility. I do like the idea of a middle-ground between E2E tests and integration tests where I can interact directly with the code-under-test and even replace specific modules with fakes or monitor the behavior in ways that are more complex (and flaky) with E2E tests, but providing suitable test coverage.
I found a workaround by re-configuring the internal ports of the couchbase services inside the docker container, so that the external and internal (thus also the bootstrapped ports reported to the SDK) are the same.
E.g. I reconfigured the rest api port from 8091 to X and I forward port X:X for the docker container
(and repeat the same for query, memcached ports etc.)
Hey @Jon1 don’t want to muddy the waters but I’ve heard of some developers using KIND (Kubernetes In Docker) and then using something like Autonomous Operator to spring up and down Couchbase Clusters on the fly (as long as the container has enough resources of course). This can get pretty complex really quickly, but it has a ton of flexibility once you have K8S.