Couchbase 7.1 comes with more flexibility and choice for data backups. We introduce direct backup to Google Cloud Platform (GCP) cloud storage. We also have a new ‐‐resume option for the cbbackupmgr restore command that handles restores, enhanced resilience and provides more reliable conformance to Recovery Time Objectives (RTO). These new features build on top of the improvements made to our backup and restore technology for increased performance in previous versions. 

New data protection features in Couchbase 7.1

To summarize, the Couchbase 7.1 GA features are:

• • Direct backup to GCP Cloud Storage
  • Resume option for cbbackupmgr restores

In addition to the new GA features, the following features will be in the Developer Preview:

• • Direct backup to Azure blob store
  • Encrypted backups

Direct backup to GCP cloud storage

We introduced native cloud storage integration as an Enterprise Edition feature with direct backup to S3-API storage in Couchbase 6.6. The archive location could be specified using the prefix s3:// and an S3-API bucket name and path. In Couchbase 7.1, direct backup to Google Cloud Storage is introduced. GCP storage is specified with the prefix gs:// and the GCP bucket name and path for the archive location. Direct backup to Google Cloud Storage uses native Google Cloud Platform (GCP) client libraries to access the Google Cloud Storage bucket.

Like the existing S3-API cloud backups, the Google Cloud Storage backups use a file system staging directory (specified by ‐‐obj-staging-directory) to temporarily store metadata for fast, low-latency interaction. The memory buffers the data and streams it directly to the cloud storage. Both the cbbackupmgr CLI and the Backup Service now have support for direct backup to Google Cloud Storage.

There are a couple of new cloud backup options available to support using different GCP credential types:

• • ‐‐obj-auth-file accepts a path to a service account authentication file
  • ‐‐obj-refresh-token is needed to refresh OAuth2 tokens when using client-id credentials specified by ‐‐obj-access-key-id and ‐‐obj-secret-access-key

Specifying ‐‐obj-auth-by-instance-metadata is optional when running the cbbackupmgr CLI on a GCP VM. Unless other credentialing options are explicitly used, the ‐‐obj-auth-by-instance-metadata is used by default and the VM instance credentials are used for the Google Cloud Storage access. The example below uses cbbackupmgr CLI from outside GCP to backup to a Google Cloud Storage bucket called my-gcp-bucket.

Example using a GCP service account authentication file and native direct backup to GCP in Couchbase 7.1: 

Resume option for cbbackupmgr restores

While direct backup to cloud storage is very popular, there can be a couple of drawbacks during restores. For one, the cost of retrieving data – getting data out of cloud storage tends to be much more costly than putting data in. Also, with cloud backups in general, increased environmental factors may cause temporary failures, such as timeouts due to temporary network issues. While needing to restart a restore from the beginning is painful no matter the circumstances, it can be much more painful when using cloud storage.

While the cbbackupmgr backup command has supported a ‐‐resume option since Couchbase 4.5, the cbbackupmgr restore command did not have a ‐‐resume option. With Couchbase 7.1, you can now resume the restore when you encounter a failure due to a temporary environmental factor, avoiding having to restart from the beginning. With the ‐‐resume option, the previous work done by the cbbackupmgr restore is “remembered,” and the restore continues from where it left off. The ‐‐resume option for cbbackupmgr restore should increase conformance to the Recovery Time Objective (RTO).

Developer Preview features

Direct backup to Azure blob containers

We introduce cbbackupmgr CLI direct backup to Azure blob containers in Developer Preview. The Backup Service will not be part of the Developer Preview for this feature. The archive location for the Azure blob container can be specified using the prefix az:// with the container name and path. When running on an Azure VM, the ‐‐obj-auth-by-instance-metadata setting is implied and does not have to be explicitly set. The Azure storage account name and key are expected when using ‐‐obj-access-key-id and ‐‐obj-secret-access-key options.

Encrypted backups

Encrypted backups are now possible with the Developer Preview when using either the cbbackupmgr CLI or the Backup Service for enhanced backup security. When using this feature with cbbackupmgr CLI, an encrypted connection to the cluster should be used for end-to-end security since the backup client encrypts the data before the data is sent to the archive storage.

Encrypted backups use a customer-provided external Key Management Service (KMS). The KMS key encrypts (envelopes) the backup repository key used to encrypt the data. When using the KMS, there will only be one KMS API call per backup, restore, or other subcommands that require the customer-provided external key. The ‐‐passphrase option can be used only for development instead of a KMS key.

The supported KMS are:

• • Amazon Web Services Key Management Service (AWS KMS)
  • Google Cloud Key Management Service (GCP KMS)
  • Azure Key Vault
  • HashiCorp Vault Transit Secrets Engine

The supported encryption algorithms include AES256GCM (the default) and AES256CBC.

To start using the encrypted backups feature, you must first create an encrypted backup repository by specifying the ‐‐encrypted option. An external passphrase or key is also needed, either with the ‐‐passphrase option (for development use only) or by providing the access information for an external KMS. The passphrase or KMS key unlocks the backup repository encryption key. Losing the passphrase or KMS key will make the entire repository inaccessible.

For the actual data encryption, cbbackupmgr creates derived keys per backup, bucket, and vBucket to help avoid issues with repeated nonces. While using AES256GCM, cbbackupmgr can safely encrypt up to 4.3 trillion documents per bucket in each backup. After that, the chances of repeated nonces increase, weakening the encryption.

These cbbackupmgr subcommands will require the passphrase or the KMS key: config, backup, restore, merge, examine. The cbbackupmgr info subcommand does not require the passphrase or the KMS key since it does not access user data.

cbbackupmgr available for Community Edition

Both Community Edition (CE) and Enterprise Edition (EE) users can use the cbbackupmgr CLI. The cbbackup and cbrestore utilities have been deprecated for CE since 7.0 and will be removed in the future. For the EE, cbbackup and cbrestore have been deprecated since 5.0.

Further resources

Try out the latest Couchbase release and follow up with the backup and restore documentation at these links:

• • Download Couchbase 7.1
  • What’s New in Couchbase 7.1
  • cbbackupmgr Cloud Backup
  • cbbackupmgr CLI Reference
  • Backup Service Overview
  • Manage Backup and Restore
  • Backup Service REST API