Cbtransfer stops midway for a huge volume of documents

Hello All,

I am currently facing a problem while trying to to do cbtransfer from a node to the filesystem (this is to backup the existing data). The total number of documents in the bucket is 4387914. Out of these the cbtransfer process always stops after writing 3289800 documents to the file. The file size stands at 1.5 GB and the progress of the cbtransfer is always shown as 75%.

I have already tried to use the cbb_max_mb and batch_max_size by setting them to lower values 10 and 10 respectively but the freeze happens at the same point.

The couchbase version is 3.0.1 community edition.

Has someone run into this error before? If yes, how do I fix it?

Thanks in advance.

Hi, never ran in that error before, @mikew should be able to help you.

hi @mikew

Would you be able to help me with this please?

I haven’t seen this issue so I will need to try to reproduce it. Can you provide me the commands you used so I can see exactly what you are doing?

Also, cbtransfer really shouldn’t be used for backing up and restoring data. We have the cbbackup command and cbbackupwrapper which I highly recommend that you use instead of cbtransfer. The only things that we usually recommend using cbtransfer for is recovering data from couchstore files if you cannot start the server on a failed node and loading csv files.