How to keep more active data in RAM?

Couchbase Server
1

Hello,

We are using couchbase server 4.0.0 cluster with 7 nodes, 2 replica and each node has 100 GB RAM.
Our problem is, replica data is using more RAM then the active data inside the bucket.

While searching, we came across one ejection threshold: “pager_active_vb_pcnt”.
And we have tried to change this threshold to 5 (Default is 40). So what we understand is, it will eject only 5% active data and 95% replica data from RAM when ejection happens.

  1. The above threshold did not worked as desired. So are we missing anything ?
  2. Are we looking at wrong place ? Becuase our goal is to utilize more RAM for active data and not for replica. So is there any other settings that we should do here ?
2

Note the change won’t take effect immediately - well it will, but until active, non-resident data is requested by a client, the active won’t be loaded into cache. In other words, the pager_active_vb_pcnt value is only applied when the item pager runs and considers which items to eject from RAM.

Note also that the item pager will only run when the high watermark is reached.

If you let the cluster run for a while (and non-resident items to be fetched from disk) you should see the active percentage increase over time.

3

Could “pager_active_vb_pcnt” be set by default to desired value somewhere in config files or there is only a cli interface to dynamically change params for running cluster ?
UPDATE: I see there is a https://github.com/couchbase/ep-engine/blob/9ba9f4b9f1f6999dc678fa32b2776e70af63cf21/configuration.json
Maybe a “tiny shortcut” about how “EventuallyPersistentEngine::configuration” is populated for today-a-little-bit-lazy-for-code-reading ones ? :wink:

4

Thanks for the reply, but how did you find the ep engine’s configuration.json file in linux ?
Can you provide us the path for this configuration file where it is located in linux ?

5

@ronak.pandya,
“Use the source, Luke!”
Actually, i don’t need all these features (at least now), so, i just don’t wanna to use my time digging to the roots of “where initialization happens”. But if you have enough time - all the sources are yours!
I hoped @drigby could give a hint about this, but looks like he is " quietly on the side of hardworkers way".

P.S. Probably, basic params are just pre-generated before compilation and then “softcoded”, but this assumption needs to be verified in more detailed way then a “brief look at the one file” ( ep-engine/tools/genconfig.cc at 9ba9f4b9f1f6999dc678fa32b2776e70af63cf21 · couchbase/ep-engine · GitHub )

6

There’s not a UI mechanism exposed for this, but you can use the diag/eval endpoint on ns_server to change this (and a number of other parameters). Note some of these require a restart of the memcached process (and consequently the data for that node will be offline until it warms up again), but you should be ok without a restart for pager_active_vb_pcnt.

See for example the comments on this MB: Loading...

That is changing the maximum number of writers, but you should be able to do something similar for the pager_active_vb_pcnt - note: this is untested - please test before running on a live system.

curl -u Administrator:password -XPOST -d \
    'ns_bucket:update_bucket_props("default", [{extra_config_string, "pager_active_vb_pcnt=5"}]).' \
    http://localhost:8091/diag/eval

Edit: fixed syntax of curl command

7

Nice, that’s useful, thank you.