Couchbase Server

autofailover should not auto failover anybody when rebalance is marked as in-progress. Or at least be more careful there

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.8.0
  • Fix Version/s: 1.8.1
  • Component/s: ns_server
  • Security Level: Public

Description

In one of customers:

* rebalance was running
* node orchestrating rebalance failed
* node orchestrating rebalance was automatically failed over by new master
* node orchestrating rebalance re-joined (still thinking it was rebalancing see MB-5049)
* we had config conflict and bad things as a result

Looks like during rebalance we should just disable autofailover because during rebalance humans can take care of correct actions

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Dipti Borkar added a comment -
Looks like this is a major code change, we should move to 1.8.2
Show
Dipti Borkar added a comment - Looks like this is a major code change, we should move to 1.8.2
Hide
Permalink
Aleksey Kondratenko added a comment -
hm. It's not that major. And I've just uploaded fix into gerrit. http://review.couchbase.org/14789
Show
Aleksey Kondratenko added a comment - hm. It's not that major. And I've just uploaded fix into gerrit. http://review.couchbase.org/14789
Aliaksey Artamonau made changes -
Field Original Value New Value
Summary autofailover should be auto failover anybody when rebalance is marked as in-progress. Or at least be more careful there autofailover should not auto failover anybody when rebalance is marked as in-progress. Or at least be more careful there
Hide
Permalink
Aliaksey Artamonau added a comment -
Fix merged.
Show
Aliaksey Artamonau added a comment - Fix merged.
Aliaksey Artamonau made changes -
Status Open [ 1 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-ns-server-2-0 #333 (See [http://qa.hq.northscale.net/job/github-ns-server-2-0/333/])
    disallow automatic failover during rebalance.MB-5050 (Revision e7db1454427a6e9cfe0bb2730ba27f025c1fafab)

     Result = SUCCESS
Aliaksey Artamonau :
Files :
* src/auto_failover_logic.erl
* src/auto_failover.erl
Show
Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #333 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/333/ ])     disallow automatic failover during rebalance. MB-5050 (Revision e7db1454427a6e9cfe0bb2730ba27f025c1fafab)      Result = SUCCESS Aliaksey Artamonau : Files : * src/auto_failover_logic.erl * src/auto_failover.erl
Farshid Ghods made changes -
Labels 1.8.1-release-notes

People

  • Assignee:
    Aleksey Kondratenko
    Reporter:
    Aleksey Kondratenko
Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: