Couchbase Server

Rebalance exited with reason bad_replicas_due_to_bad_results

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Cannot Reproduce
  • Affects Version/s: 2.0
  • Fix Version/s: 2.0.1
  • Component/s: ns_server
  • Security Level: Public
  • Labels:
    None
  • Environment:
    Physical servers, CentOS 5.8, 32GB, 4 cores, 2 x SSD
    Build 1950

Description

Traditional rebalance-in case, 3->4 nodes, 10M x 2K items, views (1 ddoc, 8 views).

Starting with usual "Control connection to memcached on 'ns_1@10.2.1.66' disconnected: {badmatch, {error, timeout}}".

Then "Rebalance is done, but failed to verify replications on following nodes:['ns_1@10.2.1.66']"

Maybe related to MB-6218.

Issue Links

depends on

Bug - A problem which impairs or prevents the functions of the product. MB-6595 [RN 2.0.1]][longevity] something unknown is causing severe timeouts in ns_server. Particularly under views building and/or compaction. Which causes rebalance to fail and other types of badness.

  • Blocker - Blocks development and/or testing work, production could not run.
  • Closed - The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.

Technical task - A technical task. MB-7261 [system test] Rebalance stopped by janitor /Resetting rebalance status since it's not really running (couchdb:error Uncaught error in HTTP request: {exit,{open_db_failed,{not_found,no_db_file}}})

  • Critical - Crashes, loss of data, severe memory leak.
  • Resolved - A resolution has been taken, and it is awaiting verification by reporter. From here issues are either reopened, or are closed.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Pavel Paulau added a comment -
Diags:
https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.65-11212012-437-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.66-11212012-440-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.67-11212012-439-diag.zip
https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.68-11212012-441-diag.zip
Show
Pavel Paulau added a comment - Diags: https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.65-11212012-437-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.66-11212012-440-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.67-11212012-439-diag.zip https://s3.amazonaws.com/bugdb/jira/MB-7234/6ef4e18d/10.2.1.68-11212012-441-diag.zip
Hide
Permalink
Dipti Borkar added a comment -
Can you add more information about which test you are running?

views / xdcr / KV ?
Show
Dipti Borkar added a comment - Can you add more information about which test you are running? views / xdcr / KV ?
Hide
Permalink
Farshid Ghods added a comment -
assigning to Pavel per Dipti comments.

please assign this ticket back to alk afterwards
Show
Farshid Ghods added a comment - assigning to Pavel per Dipti comments. please assign this ticket back to alk afterwards
Hide
Permalink
Pavel Paulau added a comment -
Test described in issue description.
Show
Pavel Paulau added a comment - Test described in issue description.
Hide
Permalink
Steve Yen added a comment -
from bug-scrub...

Alk, any triage thoughts on this?
Show
Steve Yen added a comment - from bug-scrub... Alk, any triage thoughts on this?
Hide
Permalink
Aleksey Kondratenko added a comment - - edited
Thoughts-wide (have _not_ yet looked inside) looks like ordinary "lack-of-async-io-threads-related" timeout bug.
Show
Aleksey Kondratenko added a comment - - edited Thoughts-wide (have _not_ yet looked inside) looks like ordinary "lack-of-async-io-threads-related" timeout bug.
Hide
Permalink
Steve Yen added a comment -
thanks alk. to 2.0.1 per bug-scrub.
Show
Steve Yen added a comment - thanks alk. to 2.0.1 per bug-scrub.
Hide
Permalink
Farshid Ghods added a comment -
please rerun the test with the latest 2.0.1 builds
Show
Farshid Ghods added a comment - please rerun the test with the latest 2.0.1 builds
Hide
Permalink
Pavel Paulau added a comment -
not reproduced on 2.0.1 so far.
Show
Pavel Paulau added a comment - not reproduced on 2.0.1 so far.

People

  • Assignee:
    Pavel Paulau
    Reporter:
    Pavel Paulau
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: