Rebalance failed. See logs for detailed reason. You can try again

Couchbase Server
1

Hello,

I’m setting up a cluster with two nodes. When i try to rebalance i receive the follow error:

Service ‘cbas’ exited with status 2. Restarting. Messages:
(0xc4201383c0, 0xc4201383d8, 0xc4203a71c0, 0x20, 0x0, 0x0, 0x0, 0xc4203a7240, 0x19, 0xc4205fe0e0, …)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:771 +0x2e5
main.(*Mgr).handleFailedCcRemoval.func1(0xc4201383d8)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:738 +0x73
main.(*Mgr).updateStateLOCKED(0xc4201383c0, 0xc420325ca8)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:1041 +0x3a
main.(*Mgr).updateState(0xc4201383c0, 0xc420325ca8)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:1037 +0x73
main.(*Mgr).handleFailedCcRemoval(0xc4201383c0, 0xc4203a71c0, 0x20, 0x0, 0x0, 0x0, 0xc4203a7240, 0x19, 0xc4205fe0e0, 0x2, …)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:741 +0x15a
main.(*Mgr).startAndMonitorRebalance(0xc4201383c0, 0xc4203a71c0, 0x20, 0x0, 0x0, 0x0, 0xc4203a7240, 0x19, 0xc4205fe0e0, 0x2, …)
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:481 +0xcc6
created by main.(*Mgr).StartTopologyChange
goproj/src/github.com/couchbaselabs/cbas/cbas/manager.go:424 +0xa7b
[goport(/opt/couchbase/bin/cbas)] 2020/02/25 08:42:44 child process exited with status 2

When i check the logs in indexer.log i can see the following error:

[Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer private_ip:41502. Error = EOF. Kill Pipe.

Rebalance is failing when is on 20%.

Currently i’m run out of ideas.

Thank you in advance,
Antonia

2

Hi,

What version of Couchbase Server is this? From the snippet of the error, it looks like it may be an issue which was fixed in 6.0.3, MB-35688.

If you are on a version newer than that, we will probably need to get our hands on cbcollect info logs from the nodes in order to help us figure out what’s going on…

Thanks!

-MDB