Cgroup memory usage is above the critical threshold, while the current memory is only about 30-40% of the memory limit

Couchbase Server
,
1

Hi everyone,
(Sorry if my English grammar is not good andI cannot post more than one embedded media)

I’m running Couchbase 7.1.3 on Docker and I limit Couchbase container memory to 20Gb.

When I exec to Couchbase container anh call API localhost:8091/pools/default I met some message like:

alert
alert1267×259 10.1 KB

Then I check current bucket memory quota of my main bucket dt2base using couchbase-cli bucket-list. The usage rate is about 65%

To be more careful, I check current Couchbase usage memory by using docker stats, and I see memory usage barely being around 50% of the maximum, mostly 6-10Gb / 20Gb.

Then I check reports.log file:

=========================NOTICE REPORT=========================
    alarm_handler: {set,{system_memory_high_watermark,[]}}
[error_logger:info,2024-04-12T03:20:36.640Z,ns_1@cb.local:alarm_handler<0.130.0>:ale_error_logger_handler:do_log:101]
=========================NOTICE REPORT=========================
    alarm_handler: {clear,system_memory_high_watermark}
[error_logger:info,2024-04-12T03:22:36.666Z,ns_1@cb.local:alarm_handler<0.130.0>:ale_error_logger_handler:do_log:101]

With the above results, I really don’t understand what the critical messages about memory are really talking about.
I’m a DevOps newbie, so when I saw critical notices about memory, I was very scared because this is a production environment.

Please help me solve this problem huhu.
Thank you for reading my question!

2

In 7.1.4, cache is included in the cgroup memory calculation. Since this cache can be expired if needed for something else, it should not really be counted in the calculation.

The way cgroup memory usage is calculated is fixed in 7.2.0 - MB-55037

1 Like
3

Thank you for your feedback. Currently, I am not confident enough to upgrade Couchbase to 7.2.0 myself in the production environment. So as you said, the critical messages are about cache, so can I ignore it and let Couchbase continue running?

4

There is a note from the developer on the CBSE ticket that points to the MB ticket.

They can disregard cgroups memory alerts. Host alerts should still be valid though. They can’t be turned off separately from each other unfortunately.

and

you can disable the memory alert completely. It can be done the same way as any other alert (by means of UI or CLI)

1 Like
5

Thank you very much!

6

Hi,

I accessed Couchbase container and using ulimit -a, the result of max locked memory is: max locked memory (kbytes, -l) 64.
Does this configuration affect cgroup memory notifications?

Thank you for reading my question.