Couchbase Server

Rebalance exited with reason unexpected_exit ( rebalancing cluster with xdcr and views )

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Duplicate
  • Affects Version/s: 2.0
  • Fix Version/s: 2.0
  • Component/s: ns_server
  • Security Level: Public
  • Labels:
    None
  • Environment:
    EC2 High-Memory Double Extra Large Instance, Ubuntu 12.04
    build 1971

Description

Test case:
xdcr unidir (east->west), 1 bucket per cluster, 10M x 2KB items per cluster, 3 ddocs (8 views), mixed workload with queries.
rebalance from 3<->3 to 4<->3

From logs:

Rebalance exited with reason {unexpected_exit,
{'EXIT',<0.12156.402>,
{{badmatch,
[{'EXIT',
{{badmatch,{error,nxdomain}},
{gen_server,call,
[<18120.6681.401>,had_backfill,
30000]}}}]},
[{ns_single_vbucket_mover,
'-wait_backfill_determination/1-fun-1-',
1}]}}}


<0.11203.402> exited with {unexpected_exit,
{'EXIT',<0.12156.402>,
{{badmatch,
[{'EXIT',
{{badmatch,{error,nxdomain}},
{gen_server,call,
[<18120.6681.401>,had_backfill,30000]}}}]},
[{ns_single_vbucket_mover,
'-wait_backfill_determination/1-fun-1-',1}]}}}

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Pavel Paulau added a comment -
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-107-20-6-150.compute-1.amazonaws.com-1252012-1449-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-23-20-61-95.compute-1.amazonaws.com-1252012-1452-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-23-22-246-165.compute-1.amazonaws.com-1252012-1454-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-54-242-190-218.compute-1.amazonaws.com-1252012-1451-diag.zip

https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-79-224.us-west-2.compute.amazonaws.com-1252012-152-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-221-13.us-west-2.compute.amazonaws.com-1252012-1455-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-208-246.us-west-2.compute.amazonaws.com-1252012-1459-diag.zip
Show
Pavel Paulau added a comment - https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-107-20-6-150.compute-1.amazonaws.com-1252012-1449-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-23-20-61-95.compute-1.amazonaws.com-1252012-1452-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-23-22-246-165.compute-1.amazonaws.com-1252012-1454-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-54-242-190-218.compute-1.amazonaws.com-1252012-1451-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-79-224.us-west-2.compute.amazonaws.com-1252012-152-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-221-13.us-west-2.compute.amazonaws.com-1252012-1455-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7364/315c08c6/ec2-50-112-208-246.us-west-2.compute.amazonaws.com-1252012-1459-diag.zip
Hide
Permalink
Farshid Ghods added a comment -
did it succeed second time rebalancing ?
Show
Farshid Ghods added a comment - did it succeed second time rebalancing ?
Hide
Permalink
Pavel Paulau added a comment -
Second attempt is still in progress. It may take many hours...
Show
Pavel Paulau added a comment - Second attempt is still in progress. It may take many hours...
Hide
Permalink
Aleksey Kondratenko added a comment -
We've seen erlang resolver failures even for "127.0.0.1" under timeouts. nxdomain is reason here too.
Show
Aleksey Kondratenko added a comment - We've seen erlang resolver failures even for "127.0.0.1" under timeouts. nxdomain is reason here too.

People

  • Assignee:
    Aleksey Kondratenko
    Reporter:
    Pavel Paulau
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: