It’s pretty straight-forward to run an instance of Couchbase Server in a Docker container. It takes a bit more work to set up and fully configure a cluster. In this post, I’m going to walk through a shell script I wrote to automate the process.

Background

For Couchbase Connect 2017, we built an application that shows off everything from the Couchbase Analytics Service through to real-time mobile data synchronization using Couchbase Mobile. We use an NFC temperature sensing patch, generate alerts through the Vue.js-based web client, alter schemas on the fly, send push notifications, fail over clusters, and more. Check out this video of it in action.

In the demo, Couchbase Server, Sync Gateway, and the web backend are all running in the cloud. We’re working on releasing the code for the entire project. As part of that, I want to be able to run a trimmed-down version on a single machine.

That means, ideally, running two Couchbase Server clusters, Sync Gateway, and the Node.js backend application, all simultaneously.

Docker makes running separate instances of Couchbase pretty easy. Completely configuring a whole cluster still takes some work, though. That’s where the script I wrote comes in.

A Flexible Cluster Creation Script

The script is written using Bash. I had a few goals in mind while developing it:

  1. Make something general purpose and easy to use.
  2. Allow for flexible configuration, while falling back to sensible defaults.
  3. Minimize dependencies to Bash, common standard utilities (some things are just worth doing in awk), and the Docker cli.

I’ve posted the code and other related pieces on Github in this gist. Here’s the cluster formation script.

https://gist.github.com/HodGreeley/fa88c74baf55115ef83135d4d069e796#file-server

Outline

Roughly speaking, in order, the script takes care of the following:

  • Configuring parameters
  • Starting the requested number of Couchbase Server instances, one per Docker container, using the latest production image
  • Mapping the necessary ports (offsetting each instance to avoid collisions)
  • Setting the administrative account and password
  • Selecting the services available and setting the memory allocations for them
  • Creating a bucket
  • Granting rights to a client account using RBAC
  • Combining the nodes into a cluster
  • Rebalancing the final cluster

I won’t go through the script in detail. There are comments that tell which section corresponds to the outline. Feel free to leave a comment here or on Github if you have questions.

Usage

I wrote this to set up clusters on my Mac. I expect it will work equally well for any machine that can run Bash and Docker. The script doesn’t have any options. Everything is controlled by supplying parameters as key/value pairs. They’re supplied four ways. In order of priority, from lowest to highest,

  • Defaults (written into the script itself)
  • Existing environment variables
  • Lines fed to the standard input
  • Supplied as command line arguments

In the last two instances, parameters are supplied just the way you would define an environment variable. E.g. to request 3 nodes, add COUCHBASE_NODE_COUNT=3, either on the command line or redirected from a file. Look at where the defaults are set to see what you can control.

Port mapping takes a little explanation. Couchbase uses several port ranges. To create a cluster, a number of ports have to both be exposed by Docker and mapped to open ports on the host machine. To do this, specify blocks of ranges and mappings, separated by double colons (::). For example, setting COUCHBASE_SERVER_PORTS="9091-9094:8091-8094::12210:11210 maps the standard Couchbase ports 8091-8094 and 11210 to the host machine ports 9091-9094 and 12210, respectively.

Example

In a typical scenario, you might have the cluster administrator account information assigned in environment variables. I.e.

You might then keep some other general configuration information in a file. The name doesn’t matter. Here’s one I use I call london-cluster


Finally, to start the cluster, you would invoke something like the following on the command line.

In the gist you can also find a similar script for setting up Sync Gateway. That script is a little more built out in terms of having commands to create a container, start it, stop it, and remove it. Because Sync Gateway requires a configuration file, the script has to work a little harder to do the parameter substitutions.  Otherwise it’s similar to but simpler than the server script.

There’s also a script that sets up the entire demo, including using the previous two scripts. As I mentioned earlier, we’re working on releasing the code for the whole application end-to-end. The setup script gives an idea of what’s involved.

Postscript

Couchbase is open source and free to try out.
Get started with sample code, example queries, tutorials, and more.
Find more resources on our developer portal.
Follow us on Twitter @CouchbaseDev.
You can post questions on our forums.
We actively participate on Stack Overflow.
Hit me up on Twitter with any questions, comments, topics you’d like to see, etc. @HodGreeley

Author

Posted by Hod Greeley, Developer Advocate, Couchbase

Hod Greeley is a Developer Advocate for Couchbase, living in Silicon Valley. He has over two decades of experience as a software engineer and engineering manager. He has worked in a variety of software fields, including computational physics and chemistry, computer and network security, finance, and mobile. Prior to joining Couchbase in 2016, Hod led developer relations for mobile at Samsung. Hod holds a Ph.D. in chemical physics from Columbia University.

Leave a reply