You use this tool to transfer data and design documents between two clusters or from a file to a cluster. With this tool you can also create a copy of data from a node that no longer running. This tool is the underlying, generic data transfer tool that cbbackup and cbrestore are built upon. It is a lightweight extract-transform-load (ETL) tool that can move data from a source to a destination. The source and destination parameters are similar to URLs or file paths.
| Linux | /opt/couchbase/bin/ |
| Windows | C:\Program Files\Couchbase\Server\bin\ |
| Mac OS X | /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/bin/ |
The following is the syntax and examplesfor this command:
> ./cbtransfer [options] source destination Examples: cbtransfer http://SOURCE:8091 /backups/backup-42 cbtransfer /backups/backup-42 http://DEST:8091 cbtransfer /backups/backup-42 couchbase://DEST:8091 cbtransfer http://SOURCE:8091 http://DEST:8091 cbtransfer 1.8_COUCHBASE_BUCKET_MASTER_DB_SQLITE_FILE http://DEST:8091
The following are the standard command options which you can also view with cbtransfer -h:
| -h, --help | Command help |
| -b BUCKET_SOURCE | Single named bucket from source cluster to transfer |
| -B BUCKET_DESTINATION, --bucket-destination=BUCKET_DESTINATION | Single named bucket on destination cluster which receives transfer. This allows you to transfer to a bucket with a different name as your source bucket. If you do not provide defaults to the same name as the bucket-source |
| -i ID, --id=ID | Transfer only items that match a vbucketID |
| -k KEY, --key=KEY | Transfer only items with keys that match a regexp |
| -n, --dry-run | No actual transfer; just validate parameters, files, connectivity and configurations |
| -u USERNAME, --username=USERNAME | REST username for source cluster or server node |
| -p PASSWORD, --password=PASSWORD | REST password for cluster or server node |
| -t THREADS, --threads=THREADS | Number of concurrent workers threads performing the transfer. Defaults to 4. |
| -v, --verbose | Verbose logging; provide more verbosity |
| -x EXTRA, --extra=EXTRA | Provide extra, uncommon config parameters |
| --single-node | Transfer from a single server node in a source cluster. This single server node is a source node URL |
| --source-vbucket-state=SOURCE_VBUCKET_STATE | Only transfer from source vbuckets in this state, such as 'active' (default) or 'replica'. Must be used with Couchbase cluster as source. |
| --destination-vbucket-state=DESTINATION_VBUCKET_STATE | Only transfer to destination vbuckets in this state, such as 'active' (default) or 'replica'. Must be used with Couchbase cluster as destination. |
| --destination-operation=DESTINATION_OPERATION | Perform this operation on transfer. "set" will override an existing document, 'add' will not override, 'get' will load all keys transferred from a source cluster into the caching layer at the destination. |
The following are extra, specialized command options you use in this form cbtransfer -x [EXTRA OPTIONS]:
| batch_max_bytes=400000 | Transfer this # of bytes per batch. |
| batch_max_size=1000 | Transfer this # of documents per batch |
| cbb_max_mb=100000 | Split backup file on destination cluster if it exceeds MB |
| max_retry=10 | Max number of sequential retries if transfer fails |
| nmv_retry=1 | 0 or 1, where 1 retries transfer after a NOT_MY_VBUCKET message. Default of 1. |
| recv_min_bytes=4096 | Amount of bytes for every TCP/IP batch transferred |
| report=5 | Number batches transferred before updating progress bar in console |
| report_full=2000 | Number batches transferred before emitting progress information in console |
| try_xwm=1 | As of 2.0.2, transfer documents with metadata. 1 is default. 0 should only be used if you transfer from 1.8.x to 1.8.x. |
The most important way you can use this tool is to transfer data from a Couchbase node that is no longer running to a cluster that is running:
./cbtransfer \ couchstore-files://COUCHSTORE_BUCKET_DIR \ couchbase://HOST:PORT \ --bucket-destination=DESTINATION_BUCKET ./cbtransfer \ couchstore-files:///opt/couchbase/var/lib/couchbase/data/default \ couchbase://10.5.3.121:8091 \ --bucket-destination=foo
This next examples shows how you can send all the data from a node to standard output:
shell> ./cbtransfer http://10.5.2.37:8091/ stdout:Will produce a output as follows:
set pymc40 0 0 10 0000000000 set pymc16 0 0 10 0000000000 set pymc9 0 0 10 0000000000 set pymc53 0 0 10 0000000000 set pymc34 0 0 10 0000000000
Note Couchbase Server will store all data from a bucket, node or cluster, but not the associated design documents. To to so, you should explicitly use cbbackup to store the information and cbrestore to read it back into memory.
Tranfering Design Documents Only
As of Couchbase Server 2.0.2 you can transfer design documents from one cluster to another one with the option, design_doc_only=1:
> ./cbtransfer http://10.5.2.30:8091 http://10.3.1.10:8091 -x design_doc_only=1 -b bucket_one -B bucket_two transfer design doc only. bucket msgs will be skipped. done
This will transfer all design documents associated with
bucket_one to bucket_two on
the cluster with node http://10.3.1.10:8091. In
Couchbase Web Console you can see this updated design documents
when you click on the View tab and select
bucket_two in the drop-down.