Managing Disk and Memory for Nodes in the Cluster

Couchbase Server automatically manages storing the working set between disk and memory resources for nodes in a cluster. This allows an installation to have a working set that is larger than the available RAM in the nodes participating in the cluster. To keep throughput high and latency low, Couchbase Server will always keep metadata about all items in memory.

When configuring a Couchbase Server, a memory quota is set. Couchbase Server will automatically migrate items from memory to disk when the configured memory quota is reached. If those items are later accessed, they will be moved back into system memory. For efficiency purposes, these operations are performed on a regular basis in the background.

At the moment, there is no ability define a quota for the on-disk persistent storage. It is up to the administrator to appropriately monitor the disk utilization and take action (either deleting data from Couchbase or adding capacity by upgrading the nodes themselves or adding more nodes).

Couchbase Server monitors and reports on statistics for managing disk and memory. As with any multi-tier cache, if the working set of data is greater than the available amount of the bucket RAM quota (the first level of caching), performance will drop due to disk access latencies being higher and disk throughput being lower than RAM latencies and throughput. Acceptable performance of the system is application dependent. Statistics should be monitored in case tuning adjustments are required.

Server Quotas

Each server node has a memory quota that defines the amount of system memory that is available to that server node on the host system. The first node in a cluster sets a memory quota that is subsequently inherited by all servers joining the cluster. The maximum memory quota set on the first server node must be less than or equal to 80% of the total physical RAM on that node. A server cannot join a cluster if it has less physical RAM than 1.25x the RAM quota (the same maximum allocation of 80% of physical RAM to the cluster). If a server that was a standalone cluster joins another cluster, the memory quota is inherited from the cluster to which the node is added.

Server nodes do not have disk quotas. System administrators are responsible for monitoring free disk space on individual server nodes. Each server node in a cluster has its own storage path - the location on disk where data will be stored. Storage paths do not need to be uniform across all server nodes in a cluster. If a server that was a standalone cluster joins another cluster, the storage path for that server remains unchanged.

Bucket Quotas

Memory quota allocation is also controlled on a bucket-by-bucket basis. A fixed amount of memory per node is allocated for use by a bucket. Adding or removing nodes will change the size of the bucket.