Greetings all
We have got an issue where one of the nodes became heavily imbalanced in the cluster.
Can you please share some troubleshooting/analysis steps on how troubleshoot this issue?
Couchbase server 4.5.0-2601
Regards
Luke
As a quick check, assuming Linux, I’d check for all of the user couchbase processes and sort them by rss with ps (or top, if you’d rather). There are multiple processes and first order is to know which one it is.
Then, after that, have a look at the log file for that process. The docs have a section on log locations.
Thank you
It would seem like the indexer is 'hoarding over 20G of memory:
[root@rcouch2 ~]# top -M
top - 14:43:04 up 71 days, 2:19, 1 user, load average: 0.32, 0.62, 0.62
Tasks: 310 total, 1 running, 309 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2%us, 1.2%sy, 0.0%ni, 95.3%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 31.341G total, 30.653G used, 704.645M free, 87.035M buffers
Swap: 4095.996M total, 2983.781M used, 1112.215M free, 217.262M cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3445 couchbas 20 0 1213m 184m 8136 S 24.3 0.6 15869:00 cbq-engine
3141 couchbas 20 0 10.2g 8.5g 1964 S 8.1 27.0 17313:14 beam.smp
3494 couchbas 20 0 543m 89m 2164 S 4.1 0.3 1725:01 memcached
3419 couchbas 20 0 1059m 192m 3300 S 1.4 0.6 1412:31 projector
59973 couchbas 20 0 26.2g 20g 6444 S 1.4 66.2 31261:14 indexer
91218 root 20 0 15168 1420 948 R 1.4 0.0 0:00.11 top
1 root 20 0 19356 564 328 S 0.0 0.0 0:39.51 init
2 root 20 0 0 0 0 S 0.0 0.0 0:03.13 kthreadd
Although in the couchbase config it has been set up to use 512M maximum for the index.
