Unlocking Data Security: Customer-Managed Encryption Keys in Capella

Capella understands how important data security is to your business, especially when using cloud services. That’s why we’re excited to announce a new feature that lets you take control over your data protection: Customer-Managed Encryption Keys (CMEK).

What is CMEK?

CMEK is a well-known cloud security practice that allows you to use self-managed encryption keys to encrypt and decrypt data at rest. In this practice, the encryption key is created and resides in the customer-owned environment and is used by the third-party vendor to encrypt/decrypt customer data that resides with the vendor. The main goal of this practice is to allow customers to fully manage security aspects, like the encryption algorithm and key rotation policies.

Who should use CMEK?

A customer-managed encryption key system is ideal for businesses that:

• • Handle highly sensitive data subject to strict compliance.
  • Need to meet specific data security regulations.

Getting started with CMEK in Capella

The ability to associate Customer-Managed Encryption Keys is supported via the Capella Management API.  Today, this feature is available for all AWS and GCP clusters in Capella, where customers can associate the CMEK with a new or an existing cluster.

Under the hood, Capella has no knowledge of the content of the key and uses the key to simply encrypt and decrypt data at rest.

When a CMEK is associated with an existing Capella cluster, the cluster is redeployed, and the persistent volumes are encrypted with this key. This operation also causes an online swap rebalance of the nodes to allow Capella to encrypt the data in a reliable manner.

This blog is a tutorial where we will create a new Customer-Managed Encryption Key and associate it to a Capella cluster. Along the way, we will use the V4 Management APIs to create, associate, and rotate the key.

Prerequisites

Creating a key in cloud-native Key Management Service (KMS)

First, we will create a new key in our cloud-native KMS. To do this, ensure you have the right permissions to access the KMS in AWS or GCP programmatically or via the UI console.

Once in the cloud KMS console, while configuring the key, ensure that the key is of type Symmetric. This will create a single key that can be used for encryption and decryption.

The second important step is to define the Key Usage to allow Encrypt and Decrypt operations. This will ensure that the key can be used specifically to encrypt and decrypt data at rest.

AWS:

GCP:

Regionality of the Key

When configuring the key in AWS or GCP, ensure it resides in the same region as the Capella cluster. Both cloud providers allow us to select the regionality of the key, which can be either Single or Multi-Regional.

In AWS, if the key is multi-region, it is important to have at least one key replica in the same region as the Capella cluster. We must then associate this replica key’s ARN (Amazon Resource Name) with the Capella cluster.

In GCP, a Global Key Ring will ensure the key is available in any GCP location. Do check GCP’s supported locations for Cloud KMS and ensure that the Capella cluster’s location matches the supported locations for KMS.

Capella V4 Management API Setup

For the next steps in this tutorial, we will need access to execute V4 Management APIs in Capella. Follow this blog to quickly get started with the V4 Management APIs.

Step 1: Making the Key Accessible to Capella

Now that we have a CMEK successfully created in our self-managed cloud account, we need to ensure that Capella is able to use this key to encrypt/decrypt data at rest.

To provide this access, we must first capture Capella’s corresponding cloud account ID, which is unique to each organization deployed in Capella.

Execute this V4 API to get the information:

A sample response will look something like this –

Copy the corresponding cloud account ID. Ex: If your CMEK is located in AWS, copy the Capella AWS account ID. This also means you need to create a CMEK in the same cloud provider as your Capella cluster’s cloud provider.

Updating the Key Access Policy

In AWS, add access to Capella by updating the CMEK’s access policy as follows:

Replace <capella-aws-account-id> placeholder with the value for aws-capella-account from the API response.

For GCP, simply grant Cloud KMS CryptoKey Encrypter/Decrypter permissions to Capella’s Service account: rc-cluster-admin@<capella-gcp-project-id>.iam.gserviceaccount.com.

Step 2: Informing Capella about the Key

In Step 1, we ensured that Capella was able to use the key to encrypt/decrypt data at rest. In this step, we need to inform Capella that such a CMEK exists and that it can be used by clusters.

We will now add the CMEK metadata to our Capella organization:

Remember, here, the key config ARN is the ARN of the key, as seen in the customer-owned AWS account:

For GCP, the API payload will accept the resourceName of the KMS key.

This API will respond with a CMEK ID. Please note this ID as it will be used in subsequent API calls.

Once the key is added to Capella, we can easily perform list, read, and delete key operations using the V4 APIs on this key. See this API specification for more details.

Do note that Capella will only allow the deletion of the key if no cluster is actively associated with the key.

Step 3: Associating the Encryption Key with a Cluster

Next, we want to use this CMEK to encrypt/decrypt the data in one of our Capella clusters. To do this, note down the project ID and cluster ID of the particular cluster from the Capella UI.

Use this API to associate the CMEK with the said cluster. The cmekId is the ID received in Step 2 when the CMEK metadata was added to Capella:

Once this API is invoked, the cluster will be redeployed while Capella moves all the data to new persistent volumes. These volumes are newly created with the provided CMEK. This operation will result in a swap rebalance across all nodes of the cluster, without any downtime. The activity typically takes ~5-10 mins, depending on the data and cluster size.

Finally, we will see the cluster return to a healthy status and the CMEK associated with the cluster. We can find this information by making a GET cluster details API call.

To unassociate the key from the cluster, simply execute this API:

This will redeploy the cluster, remove the key, and use a new encryption key fully managed by Capella to encrypt the data at rest. This activity, too, results in a swap rebalance and takes a few minutes.

Associating the Key with a New Cluster

The key can be associated with a new cluster by executing the create cluster API and passing CMEK ID in the request payload as follows:

Step 4: Rotating the Encryption Key

An important aspect of enhanced data security is to rotate the encryption key on a schedule. Capella allows you to inform about key rotations but cannot rotate the key itself. The rotation period can be decided as per your security governance policies.

To do this, create a new CMEK in your cloud-native KMS account. Invoke the following API to inform Capella to update the key ARN or key resource name for the same CMEK ID that is associated with the Capella cluster(s).

While AWS and GCP allow us to provide a rotation policy for the same key resource, due to restricted access, Capella cannot detect if the key was rotated automatically in your cloud account(s). Hence, the above key rotation API will only accept a key resource name different from the original key’s resource name.

Once this API is invoked, Capella will automatically detect all clusters using the key with the said CMEK ID and perform a re-deployment to rotate the associated CMEK. Capella will remove the older key resource and associate the new key resource with the cluster’s persistent volumes. This operation will also result in a swap-rebalance of the data across all nodes of the cluster(s), again without any downtime.

Finally, you will see that the clusters are back to a healthy state, and the new key resource is associated with the said CMEK ID.

Conclusion

This is how you can take control of your data security by using Customer-Managed Encryption Keys for all your Couchbase clusters in Capella.

Resources and Next Steps

Check out these links on the V4 Management API reference and the detailed documentation for using Customer Managed Encryption Keys:

• • Capella Management API Reference
  • Customer Managed Encryption Keys (CMEK) in Capella

If you have questions or feedback, please leave a comment below. The Couchbase Forums or Couchbase Discord channels are another good place to reach out with questions.