Every software application has secrets. Password, API Key, secure Tokens all fall into category of secrets.  There are dire consequences if your production secret keys would get into the wrong hands. You’ll want to tightly control how and when your secret keys are accessible.

Couchbase has added more services to its infrastructure and these services have internal and external credentials, and storing credentials of these services is a challenge.  Another challenge is rotation of secrets for all internal and external services.

Couchbase 4.6 introduces management of secrets, where all secrets shared are encrypted when stored and passed correctly to nodes and services, along with ease of rotation of secrets.  There is not going to impact on any SDK client or UI and performance.

Couchbase maintains 2 levels of key hierarchy to allow easier in rotating master password without re-encoding data, supporting multiple master passwords and also will be easier to integrate with KMIP server.At startup of node, master password is created or is supplied by the user,  from which a master key is created using a strong Key Derivation Function. Couchbase uses PBKDF2 for generation key.

A random data_key is also created on server startup which is then encrypted with master key.  All secrets are encrypted using data_key on disk.  Couchbase uses an AES 256-bit algorithm in GCM mode to encrypt secrets.

To bootstrap the system, the master key is used to open the encrypted data key. The decrypted data key is then used to open the encrypted secrets, and the secrets are used to start Couchbase Server.  Couchbase recommends using a strong master password.

With Secret Management in 4.6, you can rotate your secrets at different levels of the key hierarchy periodically or in the event of a breach.

Master password rotation/first level of rotation and reset of password can be done using REST API or cli. Couchbase allows flexibility of setting one master password per node.  In case if the master password is lost and server is stopped, the node is lost. Data from node can be recovered using other tools with server.

Second level of rotation can be done by changing the data key using the REST API or cli.  

All rotation and setting of master password is audited by application.

An example of setting up server for master password using cli on ubuntu 14.

  • Install and configure couchbase server.
  • Setup master password using cli, execute the command below and pass password on prompt.

/opt/couchbase/bin/couchbase-cli master-password -c -u Administrator -p password –new-password

  • Stop the server – /etc/init.d/couchbase-server stop
  • Configure an environment variable

export CB_MASTER_PASSWORD=<password>

  • Start the server – /etc/init.d/couchbase-server start

Note if you are using sudo to start the server use -E option to sudo to start the server.

  • Rotate the data key using cli, execute the command below:

/opt/couchbase/bin/couchbase-cli master-password -c -u Administrator -p password –rotate-data-key

  • For changing the master password, execute the command below. Pass password on the prompt

/opt/couchbase/bin/couchbase-cli master-password -c -u Administrator -p password –new-password


Logging Details:


Babysitter log on password change [ns_server:info,2017-01-20T13:12:30.079Z,babysitter_of_ns_1@<0.65.0>:encryption_service:call_gosecrets_and_store_data_key:227]Master password change succeded
Babysitter log on incorrect master password during server start or env. variable is set incorrect [ns_server:error,2017-01-20T13:13:07.066Z,babysitter_of_ns_1@<0.65.0>:encryption_service:init:174]Incorrect master password. Error: {error,”cipher: message authentication failed”}
Babysitter log when master password is set correct for CB Server =========================PROGRESS REPORT=========================

supervisor: {local,ns_babysitter_sup}

started: [{pid,<0.65.0>},






[ns_server:debug,2017-01-22T12:08:46.432Z,babysitter_of_ns_1@<0.70.0>:supervisor_cushion:init:39]starting ns_port_server with delay of 5000


Posted by Ritam Sharma, Senior Manager Quality Engineering, Couchbase

Ritam Sharma is a Director of Quality Engineering at Couchbase. Prior to joining to Couchbase, he was co-founder and CTO at kOnnect DIGITAL.

Leave a reply