[MB-8456] [Doc'd] Update Documentation with the recommendation that THP should be disabled on Couchbase nodes. Created: 12/Jun/13  Updated: 04/Apr/14

Status: Reopened
Project: Couchbase Server
Component/s: documentation
Affects Version/s: 2.0.1
Fix Version/s: 2.1.0
Security Level: Public

Type: Bug Priority: Minor
Reporter: Saran Kumar Assignee: Ruth Harris
Resolution: Unresolved Votes: 0
Labels: info-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates to
relates to MB-9193 Installer should throw a warning if i... Resolved

 Description   
Nodes on which THP(Transparent Huge Pages) are enabled, we are seeing high swap usage and severe timeouts in ns_server. This may get much worse under memory pressure. So ns_server had to restart whole bunch of services because of random timeouts and in the end it reached max restart intensity and therefore terminated.

So by default THP must be disabled and this information needs to be documented.

 Comments   
Comment by kzeller [ 18/Jun/13 ]
Hi Alk,

This request to document is coming outside of Couchbase. It is not clear to me whether this is something we should recommend or not (disable THP)

Let me know if this is something we recommend or not. If we do want to recommend it, has this option been tested?


Regards,

Karen

Comment by Aleksey Kondratenko [ 18/Jun/13 ]
It's not quite outside. I have no idea if we tested this much, but we have seen at least one customer who had issues with transparent hugepages and issues stopped without.

This matches some of our developer testing.

So yes I want us to strongly recommend disabling transparent hugepages.

Text can be something like that:

RHEL6 and other newer distributions running on physical hardware are known to have transparent hugepages feature enabled. Which is in general a measurable performance boost. However under some conditions which Couchbase Server is known to trigger it may cause severe delays in page allocations. Therefore we strongly recommend everyone with that feature to disable it.

Comment by kzeller [ 20/Jun/13 ]
Added to RN 2.1.0:

<rnentry type="knownissue">

<version ver="2.1.0a"/>

<class id="perf"/>

<issue type="cb" ref="MB-8456"/>


<rntext>

<para>
RHEL6 and other newer Linux distributions running on physical hardware are known to have transparent
hugepages feature enabled. In general this can provide a measurable performance boost.
However under some conditions that Couchbase Server is known to trigger, this
it may cause severe delays in page allocations. Therefore we strongly recommend you disable this feature with Couchbase Server.

</para>


</rntext>

</rnentry>
Comment by Dave Rigby [ 12/Feb/14 ]
I think we should make this more prominent. More specifically:

1. This is tagged as "performance" currently but it also affects cluster stability
2. It's a pretty serious problem, and by default RHEL-based OSs are still shipping with THP enabled, so out-of-the-box experience is poor if they don't disable this.

Comment by Dave Rigby [ 12/Feb/14 ]
MB to update the installer to warn about this.
Comment by Ruth Harris [ 04/Apr/14 ]
Does this apply to ONLY 2.1.0?

In 2.1.0, the Known Issue can also be noted re affecting cluster stabiity.
If the INSTALLER needs to be updated, then this Issue should be transfered to the BUILD team (not documentation).

This Known Issue in 2.1.0 is not carried over into subsequent releases.

--ruth
Generated at Mon Sep 22 07:46:09 CDT 2014 using JIRA 5.2.4#845-sha1:c9f4cc41abe72fb236945343a1f485c2c844dac9.