Couchbase Server

rebalancing can get stuck due to a bug detecting the backfill completion during vbucket takeover

Details

Description

Install couchbase server 1.8.0 release with hotfix mb-4738 on 4 nodes (12 GB RAM each) cluster in ec2.
Load 38 million items to cluster.
Resident ratio: 41%
Data size on disk: 78GB
Remove a node.
Rebalance OK.
Add another node in (not the removed node).
Rebalance hang at around 80+%
  1. Text File
    107.22.11.161_tap.txt
    08/Mar/12 5:30 PM
    4 kB
    Thuan Nguyen
  2. Text File
    107.22.70.136_tap.txt
    08/Mar/12 5:30 PM
    20 kB
    Thuan Nguyen
  3. Text File
    107.22.84.123_stat.txt
    02/Mar/12 4:55 PM
    8 kB
    Thuan Nguyen
  4. Text File
    107.22.84.123_tap.txt
    08/Mar/12 5:30 PM
    22 kB
    Thuan Nguyen
  5. Text File
    107.22.84.123_tap.txt
    02/Mar/12 4:55 PM
    18 kB
    Thuan Nguyen
  7. Text File
    23.20.45.23_stat.txt
    02/Mar/12 4:55 PM
    8 kB
    Thuan Nguyen
  8. Text File
    23.20.45.23_tap.txt
    08/Mar/12 5:30 PM
    17 kB
    Thuan Nguyen
  9. Text File
    23.20.45.23_tap.txt
    02/Mar/12 4:55 PM
    16 kB
    Thuan Nguyen
  11. Text File
    23.20.50.242_stat.txt
    02/Mar/12 4:55 PM
    8 kB
    Thuan Nguyen
  12. Text File
    23.20.50.242_tap.txt
    08/Mar/12 5:30 PM
    24 kB
    Thuan Nguyen
  13. Text File
    23.20.50.242_tap.txt
    02/Mar/12 4:55 PM
    20 kB
    Thuan Nguyen
  15. Text File
    50.17.157.98_stat.txt
    02/Mar/12 4:55 PM
    7 kB
    Thuan Nguyen
  16. Text File
    50.17.157.98_tap.txt
    02/Mar/12 4:55 PM
    15 kB
    Thuan Nguyen

Activity

Ascending order - Click to sort in descending order
Thuan Nguyen made changes -
Field Original Value New Value
Attachment 23.20.45.23_tap.txt [ 12465 ]
Attachment 23.20.50.242_tap.txt [ 12466 ]
Attachment 107.22.11.161_tap.txt [ 12467 ]
Attachment 107.22.70.136_tap.txt [ 12468 ]
Attachment 107.22.84.123_tap.txt [ 12469 ]
Thuan Nguyen made changes -
Assignee Chiyoung Seo [ chiyoung ]
Chiyoung Seo made changes -
Status Open [ 1 ] Resolved [ 5 ]
Resolution Fixed [ 1 ]
Dipti Borkar made changes -
Fix Version/s 1.8.1 [ 10295 ]
Farshid Ghods made changes -
Component/s couchbase-bucket [ 10173 ]
Component/s ep_engine [ 10013 ]
Farshid Ghods made changes -
Labels 1.8.0-release-notes
Farshid Ghods made changes -
Summary rebalance in hang in longevity test cluster rebalancing can get stuck due to a bug detecting the backfill completion during vbucket takeover
Farshid Ghods made changes -
Labels 1.8.0-release-notes 1.0-release-notes 1.8.0-release-notes
Farshid Ghods made changes -
Labels 1.0-release-notes 1.8.0-release-notes 1.8.0-release-notes 1.8.1-release-notes
Farshid Ghods made changes -
Status Resolved [ 5 ] Closed [ 6 ]

People

  • Assignee:
    Chiyoung Seo
    Reporter:
    Thuan Nguyen
Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: