[MB-7626] Couchbase continually crashes and restart when Couch bucket is created in OpenVZ environment Created: 29/Jan/13  Updated: 23/Jun/14  Resolved: 23/Jun/14

Status: Resolved
Project: Couchbase Server
Component/s: installer
Affects Version/s: 2.0, 2.0.1, 2.1.0
Fix Version/s: 3.0
Security Level: Public

Type: Bug Priority: Critical
Reporter: creator11 Assignee: Anil Kumar
Resolution: Fixed Votes: 1
Labels: scrubbed
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment: OpenVZ VPS container running Ubuntu 12.04 with a CentOS 6 OS as the host on 2.6.32 kernel.

Issue Links:
Duplicate
is duplicated by MB-4376 setup get HDD and memory totale and f... Closed
Triage: Untriaged
Flagged:
Release Note

 Description   
I have done some QA to help narrow down an issue I have discovered.

1. Setup Couchbase from 64bit Ubunut 12 package.
2. Next select a CouchDB bucket.
3. Next complete the setup.

Expected: Couchbase server and bucket functioning.

Actual: The Couchbase server appears to just keep crashing / i.e. restarting every few seconds.

If I remove the couchDB bucket no crashes / restart occur.
If I create a memcached bucket only the server is functioning fine and the bucket functions as expected.

This is on a OpenVZ VPS container running Ubuntu 12.04 with a CentOS 6 OS as the host on 2.6.32 kernel.

I went ahead and setup couchbase on another Ubuntu 12.04 dedicated server (exact replica of the above server without the virtualization) and no issues.

What debug output information from the logs would you like to help identify this issue?


 Comments   
Comment by creator11 [ 29/Jan/13 ]
Ok I found the fix to this issue.

memsup as a previous bug I found was causing a segmentation fault. Upgraded to latest erlang with erlang-os-mon 2.2.10 and issue has been resolved on an OpenVZ container. I just symlinked to the new version for couchbase to use.
Comment by creator11 [ 29/Jan/13 ]
Might be handy to add this to a knowledge base for others, since I have found 2 other people that had similar issues on the web in forums.
Comment by Chiyoung Seo [ 29/Jan/13 ]
Farshid,

Please close the bug if it's not required for the QE verification.
Comment by Mike Wiederhold [ 25/Feb/13 ]
This might be good to add to our documentation.
Comment by kzeller [ 04/Mar/13 ]
Added to 2.0.1 RN:

In OpenVZ Linux containers, the server had crashed and restarted when you
created a Couchbase bucket. This was due to an issue in the memsup process from Erlang.
To fix this issue, you should upgrade to the latest version of Erlang, and have Couchbase Server use this version.
Comment by kzeller [ 04/Mar/13 ]
Added to 2.0.1 RN:

In OpenVZ Linux containers, the server had crashed and restarted when you
created a Couchbase bucket. This was due to an issue in the memsup process from Erlang.
To fix this issue, you should upgrade to the latest version of Erlang, and have Couchbase Server use this version.
Comment by kzeller [ 04/Mar/13 ]
See RN 2.0.1 addition. If you reopen, please assign to engineer responsible for fixing.
Comment by Thuan Nguyen [ 07/Mar/13 ]
Integrated in ui-testing #11 (See [http://qa.hq.northscale.net/job/ui-testing/11/])
    MB-7626: tune rebalance options during cluster setup (Revision 7c828a6a1de4f7c8fb76265766d3eb53f183fe3e)

     Result = SUCCESS
pavelpaulau :
Files :
* pytests/performance/perf.py
Comment by Perry Krug [ 20/Mar/13 ]
Karen, can you provide instructions on which version of Erlang to use here and instructions on how to apply the workaround mentioned?
Comment by kzeller [ 20/Mar/13 ]
Hi,

Can you provide the version of Erlang I Ishould document, and is the workaround just to upgrade to this version? Or are there other steps someone must do in order to upgrade to Erlang using CB?


Thanks,

Karen
Comment by Joseph H. [ 20/Mar/13 ]
I've actually applied the the Workaround for MB-4376 successfully (at least for a development environment) to CB 2.0.1 as I didn't succeed in getting CB to use a entirely different version of Erlang.
Comment by Perry Krug [ 21/Mar/13 ]
Note that we have confirmed through another customer that the workaround in mb-4376 is successful.

Can this get rolled into 2.0.2?
Comment by kzeller [ 21/Mar/13 ]
Hi Perry, confirm this, ignore formatting (will go in release notes, where workarounds, known issues, etc. go):

=========

last email from prospect:
I’ve solved the problem by compiling memsup with the patch mentioned in http://www.couchbase.com/issues/browse/MB-4376 applied. This patch is included in the R15B01 release of Erlang. Perhaps Issue 4376 shouldn’t be marked fixed until Couchbase is using Erlang R15B01 or later?

==========

Existing content:

In OpenVZ Linux containers, the server had crashed and restarted when you created a Couchbase bucket. This was due to an issue in the memsup process from Erlang. To fix this issue, you should upgrade to the latest version of Erlang, and have Couchbase Server use this version.

Proposed addition:

1) You should stop Couchbase server
2) Make a copy of your original memsup then apply the patch available on GitHub: https://github.com/vorobev/otp/compare/maint-memsup.patch

You can also download the following two files and compile yourself:

-https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.c

-https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.h

gcc memsup.c -o memsup

Place the compiled code in /opt/membase/lib/erlang/lib/os_mon-2.2.5/priv/bin/

3) Restart Couchbase Server
Comment by Perry Krug [ 21/Mar/13 ]
That's fine from a workaround perspective I guess...but my request was more about actually fixing the bug for 2.0.2.

I'll add the ns_server component.

Thanks
Comment by kzeller [ 28/Mar/13 ]
Added content:

https://github.com/couchbase/docs/commit/b8094b5c61802fb4b76f0ae6c9f4979c5f1aa6f4
Comment by kzeller [ 28/Mar/13 ]
Added content:

https://github.com/couchbase/docs/commit/b8094b5c61802fb4b76f0ae6c9f4979c5f1aa6f4
Comment by Perry Krug [ 29/Mar/13 ]
Karen, this isn't just about documentation, this is a bug that needs to be fixed in the product.
Comment by Maria McDuff (Inactive) [ 16/Apr/13 ]
deferring out of 2.0.2
Comment by Aleksey Kondratenko [ 12/Aug/13 ]
We're simply waiting until we upgrade to R16-something. When that happens this issue will automatically be fixed.
Comment by Wayne Siu [ 22/Aug/13 ]
Removing the release notes description in the summary, and assigning it to PM to prioritize
Comment by Drew Schatt [ 07/Apr/14 ]
This is sort of a big issue for us - we were considering deploying couchbase on some rather beefy virtual servers here. In our case, we're using proxmox, and couchbase 2.2.0_community and 2.5.1 both have the issue... which makes them unusable.
Comment by Anil Kumar [ 23/Jun/14 ]
As per Alk with R16 this should be fixed.
Generated at Fri Sep 19 23:21:51 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.