Couchbase Server

Rebalance failure due to replicator_died with "memcached_error,auth_error,<<"Auth failure">>" because node was marked as ready before isasl.pw was read

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 1.8.1-release-candidate
  • Fix Version/s: 1.8.1
  • Component/s: ns_server
  • Security Level: Public
  • Labels:
    None
  • Environment:
    Ubuntu 64 bit
    181-901-rel

Description

Decided to file a new bug, but the old one is here:-
http://www.couchbase.com/issues/browse/MB-5259
http://www.couchbase.com/issues/browse/MB-5291


<0.4181.3> exited with {exited,
{'EXIT',<0.4182.3>,
{replicator_died,
{'EXIT',<15697.23073.1>,
{badmatch,
{memcached_error,auth_error,
<<"Auth failure">>}}}}}}


Failing testcase:-
swaprebalance.SwapRebalanceFailedTests.test_failover_swap_rebalance,replica=1,num-buckets=4,num-swap=1,swap-orchestrator=True
1) Failover a node orchestrator
2) Swap rebalance it.

  1. GZip Archive
    10.1.3.152-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    1.76 MB
    Karan Kumar
  2. GZip Archive
    10.1.3.153-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    2.03 MB
    Karan Kumar
  3. GZip Archive
    10.1.3.155-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    1.25 MB
    Karan Kumar
  4. GZip Archive
    10.1.3.156-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    1.87 MB
    Karan Kumar
  5. GZip Archive
    10.1.3.157-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    1.94 MB
    Karan Kumar
  6. GZip Archive
    10.1.3.158-8091-diag.txt.gz
    11/Jun/12 6:11 PM
    1.81 MB
    Karan Kumar

Activity

Descending order - Click to sort in ascending order
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-ns-server-2-0 #380 (See [http://qa.hq.northscale.net/job/github-ns-server-2-0/380/])
    MB-5513: don't allow ns_memcached to think it started twice (Revision e42fa1fcd59b54800da73a2702458adb2eae069d)

     Result = SUCCESS
Aliaksey Kandratsenka :
Files :
* src/ns_memcached.erl
Show
Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #380 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/380/ ])      MB-5513 : don't allow ns_memcached to think it started twice (Revision e42fa1fcd59b54800da73a2702458adb2eae069d)      Result = SUCCESS Aliaksey Kandratsenka : Files : * src/ns_memcached.erl
Hide
Permalink
Thuan Nguyen added a comment -
Integrated in github-ns-server-2-0 #371 (See [http://qa.hq.northscale.net/job/github-ns-server-2-0/371/])
    MB-5513: wait isasl.pw refresh before flagging warmup completion (Revision 9f7487ec66bc5c6dc8dde0eb4347869cd96d9a79)

     Result = SUCCESS
Aliaksey Artamonau :
Files :
* src/ns_config_isasl_sync.erl
* src/ns_memcached.erl
Show
Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #371 (See [ http://qa.hq.northscale.net/job/github-ns-server-2-0/371/ ])      MB-5513 : wait isasl.pw refresh before flagging warmup completion (Revision 9f7487ec66bc5c6dc8dde0eb4347869cd96d9a79)      Result = SUCCESS Aliaksey Artamonau : Files : * src/ns_config_isasl_sync.erl * src/ns_memcached.erl
Hide
Permalink
Aleksey Kondratenko added a comment -
Done
Show
Aleksey Kondratenko added a comment - Done
Hide
Permalink
Karan Kumar added a comment -
=========================CRASH REPORT=========================
  crasher:
    initial call: ebucketmigrator_srv:init/1
    pid: <0.6065.2>
    registered_name: []
    exception error: no match of right hand side value
                     {memcached_error,auth_error,<<"Auth failure">>}
      in function ebucketmigrator_srv:connect/4
      in call from ebucketmigrator_srv:init/1
    ancestors: ['ns_vbm_sup-bucket-3','single_bucket_sup-bucket-3',
                  <0.3856.2>]
    messages: []
    links: [#Port<0.23530>,<0.5813.2>,#Port<0.23527>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 377
    stack_size: 24
    reductions: 1406
  neighbours:
Show
Karan Kumar added a comment - =========================CRASH REPORT=========================   crasher:     initial call: ebucketmigrator_srv:init/1     pid: <0.6065.2>     registered_name: []     exception error: no match of right hand side value                      {memcached_error,auth_error,<<"Auth failure">>}       in function ebucketmigrator_srv:connect/4       in call from ebucketmigrator_srv:init/1     ancestors: ['ns_vbm_sup-bucket-3','single_bucket_sup-bucket-3',                   <0.3856.2>]     messages: []     links: [#Port<0.23530>,<0.5813.2>,#Port<0.23527>]     dictionary: []     trap_exit: false     status: running     heap_size: 377     stack_size: 24     reductions: 1406   neighbours:

People

  • Assignee:
    Aleksey Kondratenko
    Reporter:
    Karan Kumar
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: