It is a best practice to backup and restore all nodes together to minimize any inconsistencies in data. Membase is always per-item consistent, but does not guarantee total cluster consistency or in-order persistence.
There are two options for restoring a Membase Server cluster in versions 1.7.1 and higher:
Assumptions:
The cluster must contain the same number of nodes and those nodes must have the same IP addresses (or hostnames if you followed the instructions listed here)
You must restore all of the config.dat files as well as all of the database files to their original locations
Stop the Membase Server service on all nodes
Replace the database and configuration files
Restart the service.
This must be performed on each node within the entire cluster.
You can use this feature to migrate an entire cluster into new set of machines. This is particularly useful when:
In cloud environments, where the IP addresses of nodes will have changed
To create dev/test clusters with the same data as the production cluster
Assumptions:
The two clusters must have the same topology (number of nodes)
The original cluster must be in a healthy state. This means that all nodes should be up and running and no rebalance or failover operation should be running.
It is a best practice for both clusters to be of the same OS and memory configuration
Instructions:
Take a backup of the data files of all nodes using the above procedure. Alternately, shut down the membase-server on all nodes and copy the DB files.
Install Membase Server (of at least version 1.7.1) on new nodes and cluster together. If using the web console to setup your cluster, a 'default' bucket will be created. Please delete this bucket before proceeding.
Place the copies of the original files into the data directory on all the new nodes.
You do not have to "match" up the nodes one-for-one. However, ensure that each set of original data files gets placed onto one and only one node of the new cluster
Please ensure that you retain file ownership properties for those files which you placed on the destination node.
Start membase-server on the new nodes
Create a bucket with the same name and SASL configuration on the new nodes.
After the bucket creation, each node will start loading items from the data files into memory.
The cluster will be in a balanced state after warm up.
Do not start a rebalance process while nodes are still warming up.
If any nodes go down during the warmup, it is a best practice to restart all nodes together.
There are a number of bugs in older versions of the mbrestore script. Anyone using mbrestore should make sure to get the latest script to ensure proper functionality. You can download the latest from here. The latest version of the script WILL work with any previous versions of Membase.
If you are unable to meet the above requirements for restoring a cluster, you can use the mbrestore tool (included in versions 1.6.4 and higher). This script will read the key-value pairs from a set of database files and send them as client commands into a bucket of your choosing.
This is useful if:
You want to restore data into a cluster of a different size
You want to transfer/restore data into a different bucket
You have a broken or corrupted database file (usually from running out of space on a disk drive)
The mbrestore tool provides the following options:
%mbrestore opts db_files (use -h for detailed help) -a --add Use add instead of set to avoid overwriting existing items -H --host Hostname of moxi server to connect to (default is 127.0.0.1) -p --port Port of moxi server to connect to (default is 11211) -t --threads Number of worker threads -u --username Username to authenticate with (this is the name of the bucket you are sending data into) -P --password Password to authenticate with (this is the password of the bucket you are sending data into)
Depending on the amount of data, this script can take an extended amount of time to run. It is a best practice to make sure that your connection to the server running the script is not broken, or that you are using something to let the script run in the background (i.e. screen) Linux:
/opt/membase/bin/mbrestore -a default default-0.mb default-1.mb default-2.mb default-3.mbIn order to correctly restore you must put all of the database backup file names as command arguments Windows:
shell> cd "C:/Program Files/Membase/Server/bin/" shell> mbrestore -a "C:/backup/2010-12-22/default" "C:/backup/2010-12-22/default-0.mb" \ "C:/backup/2010-12-22/default-1.mb" "C:/backup/2010-12-22/default-2.mb" "C:/backup/2010-12-22/default-3.mb"
In order to correctly restore you must put all of the database backup file names as command arguments.