Short:
How/where to configure how much memory beam.smp reserves?
Long:
I have successfully compiled CB on BSD. I’m compiling from source. CB is running fine but I need to adjust the memory reserved by beam.smp. this seems to default to 15GB but I need to adjust this to the hardware I’m trying to use it on. I asked on Stack Overflow but I was given the answer that the installer asks about to amount of memory and should take care of that. Since I’m not using any installer, that answer didn’t really help to find where to configure this in a running system.
Thank you for your answer! Can I ask you to please look at my screenshots at stack overflow? The answer I got from Matthew Groves on StackOverflow suggests that there technically should be a limit, which is set by the installer. It’s just that this seems to default to 15GB in my installation and I don’t know where to change that. As you can see from the gray “screenshot”, top reports that beam-smp has reserved 15GB of which 285MB have been used.
A load of 38 means that it took me minutes to even log into via console and kill the beam.smp process that used the most memory. The machine then immediately became responsive again and after a few minutes everything was back to normal.
I was hoping that I could set the max memory that Erlang “thinks” has available so that it even wouldn’t try to use that much memory.
Maybe I understood this one wrong then: recommended RAM is " 2GB of RAM free per node above and beyond what is needed to hold the data". So with my 4GB machine and less than 512 MB of actual data, I should be on the safe side, shouldn’t I? Or do you mean the document says I should have 16GB AND additional 2GB per node?
I appreciate your help, I really do, but I cannot agree to give 16GB of RAM to a machine that I need for 120 MB of data just to work around that I couldn’t figure out how to properly set the limit for memory usage. As far as my experience goes, even 8 or 14GB RAM wouldn’t solve the issue, because the Erlang VM constantly increases its memory footprint and I assume (but I haven’t tested) that it will continue to do so until it hits the 15GB reservation limit. I have no idea what will happen once it reaches 15GB, but I assume that it’ll start freeing memory again.
Which brings me back to my original question: Matthew Groves answer suggested that there actually is some limit that can be configured and the installer does that. But what exactly does the installer change?
That blog is a reasonable guide; but note it’s from over 4 years ago (when Couchbase was at ~version 3); resource usage may have increased since then.
You generally need something in the region of ~1GB for the cluster manager (beam.smp) and other fixed overheads on say a couple-node cluster; plus whatever memory quota you’ve allocated to each services’ quotas.
If you haven’t already, I’d not configure any services you arn’t using (GSI, Query, FTS, etc).
So that’s not something I’ve seen with Erlang on Linux - perhaps that’s some kind of BSD-specific behaviour.
I’m not sure what Matthew was alluding to, but I’m pretty sure there’s no limit for Erlang VM setup in the installer. You might find some more general settings for Erlang’s VM in the documentation for BSD Erlang.
I’m pretty sure there’s no limit for Erlang VM setup
Did I mention I appreciate your help? I do! So please don’t get me wrong, I’m not arguing with you. However, in Erlang there is “+hmax Size” and “max_heap_size”. These sound to me like candidates for what I’m looking for. I guess I’ll have to look for these options in the CB sources then.