Details
Description
Setup:
1.Setup a 18 node cluster. Enable Auto-failover
2.Load data on all 3 buckets [around 50M, 22M, 500k] items.
3. Continue loading data..
4. Remove orchestrator node [105] Add new node [126]
5. Issue rebalance on this cluster. Rebalance failed with "replicator died" - filed bug 5343
6. Re-issue rebalance . Rebalance fails with " change_filter_failed"
Output:
Reissue rebalance fails with "change_filter_failed"
Node 105 is auto-failed over
Rebalance exited with reason {{change_filter_failed,
{'EXIT',
{{badmatch,
{failed,
{error,
{badmatch,
{memcached_error,auth_error,
<<"Auth failure">>}},
[{ebucketmigrator_srv,connect,4},
{ebucketmigrator_srv,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}}},
[{ns_vbm_sup,
'-local_change_vbucket_filter/4-fun-2-',6},
{misc,'-executing_on_new_process/1-fun-0-',
3}]}}},
[{ns_vbm_sup,change_vbucket_filter,4},
{ns_vbm_sup,'-set_replicas/3-fun-2-',5},
{lists,foldl,3},
{ns_vbm_sup,set_replicas,3},
{ns_vbm_sup,'-set_replicas_on_nodes/3-fun-1-',
3},
{lists,foreach,2},
{ns_vbm_sup,apply_changes,2},
{ns_vbucket_mover,sync_replicas,0}]}
Logs at - https://s3.amazonaws.com/bugdb/jira/bug4-rebalance-181/bug5.tar
1.Setup a 18 node cluster. Enable Auto-failover
2.Load data on all 3 buckets [around 50M, 22M, 500k] items.
3. Continue loading data..
4. Remove orchestrator node [105] Add new node [126]
5. Issue rebalance on this cluster. Rebalance failed with "replicator died" - filed bug 5343
6. Re-issue rebalance . Rebalance fails with " change_filter_failed"
Output:
Reissue rebalance fails with "change_filter_failed"
Node 105 is auto-failed over
Rebalance exited with reason {{change_filter_failed,
{'EXIT',
{{badmatch,
{failed,
{error,
{badmatch,
{memcached_error,auth_error,
<<"Auth failure">>}},
[{ebucketmigrator_srv,connect,4},
{ebucketmigrator_srv,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}}},
[{ns_vbm_sup,
'-local_change_vbucket_filter/4-fun-2-',6},
{misc,'-executing_on_new_process/1-fun-0-',
3}]}}},
[{ns_vbm_sup,change_vbucket_filter,4},
{ns_vbm_sup,'-set_replicas/3-fun-2-',5},
{lists,foldl,3},
{ns_vbm_sup,set_replicas,3},
{ns_vbm_sup,'-set_replicas_on_nodes/3-fun-1-',
3},
{lists,foreach,2},
{ns_vbm_sup,apply_changes,2},
{ns_vbucket_mover,sync_replicas,0}]}
Logs at - https://s3.amazonaws.com/bugdb/jira/bug4-rebalance-181/bug5.tar
Activity
Farshid Ghods
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Fix Version/s | 2.0-beta [ 10113 ] | |
| Fix Version/s | 2.0-developer-preview-5 [ 10290 ] |
Peter Wansch
made changes -
| Status | Open [ 1 ] | Resolved [ 5 ] |
| Resolution | Fixed [ 1 ] |
Peter Wansch
made changes -
| Sprint Status | Current Sprint | |
| Sprint Priority | 4 |
Farshid Ghods
made changes -
| Summary | Rebalance fails with " change_filter_failed" if user changes bucket password before rebalance process | Rebalance can fail with various error if user changes bucket password before rebalance process |
Peter Wansch
made changes -
| Resolution | Fixed [ 1 ] | |
| Status | Resolved [ 5 ] | Reopened [ 4 ] |
| Assignee | Aliaksey Artamonau [ Aliaksey Artamonau ] | Aleksey Kondratenko [ alkondratenko ] |
Aleksey Kondratenko
made changes -
| Status | Reopened [ 4 ] | Resolved [ 5 ] |
| Resolution | Fixed [ 1 ] |
Farshid Ghods
made changes -
| Status | Resolved [ 5 ] | Closed [ 6 ] |