Couchbase Server

System Test : Higher ep_commit_time, beam size on source nodes during xdcr.

Details

  • Type: Bug Bug
  • Status: Open Open
  • Priority: Critical Critical
  • Resolution: Unresolved
  • Affects Version/s: 2.0.1
  • Fix Version/s: None
  • Component/s: cross-datacenter-replication
  • Security Level: Public
  • Labels:
    None
  • Environment:
    2.0.1-144-rel

    EC2 env.
    2 buckets,
    1 bidirectional replication on default bucket
    1 unifdirectional replication on sasl bucket.

Description

Seeing unusual stats for

( during xdcr_start phase)
1.ep_commit_time
2. rsize_beam : > 2.8G on some points
3. Uneven compaction growth
4. Uneven CPU across the cluster.

The source node cluster stats are here

https://github.com/couchbaselabs/couchbase-qe-docs/tree/master/system-tests/plum-cluster/01_30

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Jin Lim added a comment -
Per bug scrub, engineering must at least identify and understand the root cause of this behavior. Assuming a fix may require cross-components change, the actual delivery of the fix may schedule into the next release after 2.0.1.
Show
Jin Lim added a comment - Per bug scrub, engineering must at least identify and understand the root cause of this behavior. Assuming a fix may require cross-components change, the actual delivery of the fix may schedule into the next release after 2.0.1.
Hide
Permalink
Jin Lim added a comment -
Ketaki, per bug scrub, please collect atop from the 2nd running of the test. If and only if possible though understanding that it is over ec2 environment and longevity test. Thanks.
Show
Jin Lim added a comment - Ketaki, per bug scrub, please collect atop from the 2nd running of the test. If and only if possible though understanding that it is over ec2 environment and longevity test. Thanks.
Hide
Permalink
Ketaki Gangal added a comment -
Setup the xdcr_kv ec2 cluster for system testing.

Source : http://ec2-107-22-40-124.compute-1.amazonaws.com:8091
Destination : http://ec2-54-242-239-237.compute-1.amazonaws.com:8091

Stats Collection
Source : http://10.3.121.45:3133/fast
Destination: http://10.6.2.58:3133/fast

To add : Stats after a run time of 10 hours.

-Ketaki
Show
Ketaki Gangal added a comment - Setup the xdcr_kv ec2 cluster for system testing. Source : http://ec2-107-22-40-124.compute-1.amazonaws.com:8091 Destination : http://ec2-54-242-239-237.compute-1.amazonaws.com:8091 Stats Collection Source : http://10.3.121.45:3133/fast Destination: http://10.6.2.58:3133/fast To add : Stats after a run time of 10 hours. -Ketaki
Hide
Permalink
Jin Lim added a comment -
more informations are needed for describing this undesired behavior from QE.
Show
Jin Lim added a comment - more informations are needed for describing this undesired behavior from QE.

People

  • Assignee:
    Ketaki Gangal
    Reporter:
    Ketaki Gangal
Vote (0)
Watch (6)

Dates

  • Created:
    Updated: