Details
-
Type:
Bug
-
Status:
Closed
-
Priority:
Blocker
-
Resolution: Cannot Reproduce
-
Affects Version/s: 2.0-beta
-
Fix Version/s: 2.0-beta-2
-
Component/s: couchbase-bucket
-
Security Level: Public
-
Labels:
-
Environment:centos 6.2 64 bit build 2.0.0-1697
Description
Cluster information:
- 11 centos 6.2 64bit server with 4 cores CPU
- Each server has 10 GB RAM and 150 GB disk.
- 8 GB RAM for couchbase server at each node (80% total system memmories)
- Disk format ext3 on both data and root
- Each server has its own drive, no disk sharing with other server.
- Load 9 million items to both buckets
- Initial indexing, so cpu a little heavy load
- Cluster has 2 buckets, default (3GB) and saslbucket (3GB)
- Each bucket has one doc and 2 views for each doc (default d1 and saslbucket d11)
* Create cluster with 10 nodes installed couchbase server 2.0.0-1697
10.3.121.13
10.3.121.14
10.3.121.15
10.3.121.16
10.3.121.17
10.3.121.20
10.3.121.22
10.3.121.24
10.3.121.25
10.3.121.23
* Data path /data
* View path /data
* Do swap rebalance. Add node 26 and remove node 25
* Rebalance failed and saw a lot of error message memcached exited with status 71 in log page.
Link to diags of all nodes https://s3.amazonaws.com/packages.couchbase/diag-logs/orange/201209/11nodes-1697-memcached-exit-71-20120910.tgz
Link to atop node 13 https://s3.amazonaws.com/packages.couchbase/atop-files/orange/201209/atop-node13
Due to large size of atop file, all other atop files are in /tmp directory of each node
- 11 centos 6.2 64bit server with 4 cores CPU
- Each server has 10 GB RAM and 150 GB disk.
- 8 GB RAM for couchbase server at each node (80% total system memmories)
- Disk format ext3 on both data and root
- Each server has its own drive, no disk sharing with other server.
- Load 9 million items to both buckets
- Initial indexing, so cpu a little heavy load
- Cluster has 2 buckets, default (3GB) and saslbucket (3GB)
- Each bucket has one doc and 2 views for each doc (default d1 and saslbucket d11)
* Create cluster with 10 nodes installed couchbase server 2.0.0-1697
10.3.121.13
10.3.121.14
10.3.121.15
10.3.121.16
10.3.121.17
10.3.121.20
10.3.121.22
10.3.121.24
10.3.121.25
10.3.121.23
* Data path /data
* View path /data
* Do swap rebalance. Add node 26 and remove node 25
* Rebalance failed and saw a lot of error message memcached exited with status 71 in log page.
Link to diags of all nodes https://s3.amazonaws.com/packages.couchbase/diag-logs/orange/201209/11nodes-1697-memcached-exit-71-20120910.tgz
Link to atop node 13 https://s3.amazonaws.com/packages.couchbase/atop-files/orange/201209/atop-node13
Due to large size of atop file, all other atop files are in /tmp directory of each node
Activity
- All
- Comments
- Work Log
- History
- Activity
- Gerrit Reviews
Karan Kumar
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Attachment | memcached_logfile [ 14919 ] |
Karan Kumar
made changes -
| Summary | [longevity] memcached constantly exited with status 71 on node 13 | [longevity] memcached constantly exited with status 71 and unable to restart |
Karan Kumar
made changes -
| Assignee | Karan Kumar [ karan ] | Trond Norbye [ trond ] |
Thuan Nguyen
made changes -
| Priority | Major [ 3 ] | Blocker [ 1 ] |
Karan Kumar
made changes -
| Assignee | Trond Norbye [ trond ] | Chiyoung Seo [ chiyoung ] |
Chiyoung Seo
made changes -
| Assignee | Chiyoung Seo [ chiyoung ] | Mike Wiederhold [ mikew ] |
Chiyoung Seo
made changes -
| Sprint Status | Current Sprint |
Mike Wiederhold
made changes -
| Component/s | couchbase-bucket [ 10173 ] | |
| Component/s | bucket-engine [ 10010 ] |
Mike Wiederhold
made changes -
| Assignee | Mike Wiederhold [ mikew ] | Trond Norbye [ trond ] |
Farshid Ghods
made changes -
| Fix Version/s | 2.0-beta-refresh [ 10385 ] | |
| Fix Version/s | 2.0-beta [ 10113 ] |
Chiyoung Seo
made changes -
| Assignee | Trond Norbye [ trond ] | Chiyoung Seo [ chiyoung ] |
Karan Kumar
made changes -
| Labels | system-test |
Farshid Ghods
made changes -
| Summary | [longevity] memcached constantly exited with status 71 and unable to restart | [longevity] memcached hangs when aborting during swap rebalance operation and fails to restart |
Farshid Ghods
made changes -
| Summary | [longevity] memcached hangs when aborting during swap rebalance operation and fails to restart | [longevity] memcached hangs when aborting during swap rebalance operation and fails to restart ( exit 71 ) |
Farshid Ghods
made changes -
| Labels | system-test | 2.0-beta-release-notes system-test |
Chiyoung Seo
made changes -
| Assignee | Chiyoung Seo [ chiyoung ] | Thuan Nguyen [ thuan ] |
Chiyoung Seo
made changes -
| Status | Open [ 1 ] | Resolved [ 5 ] |
| Resolution | Cannot Reproduce [ 5 ] |
Chiyoung Seo
made changes -
| Sprint Status | Current Sprint |
Farshid Ghods
made changes -
| Status | Resolved [ 5 ] | Closed [ 6 ] |