Relationship between #buckets and node (hardware) configuration?
I had read in some documentation that CB does not recommend more than 3-4 buckets on a small cluster.
What constitutes a small cluster, though is a bit of a mystery. Can someone clarify what is considered a small cluster?
My question is how to determine what cluster (hardware) configuration will support how many buckets? Also, does increasing the CPUs on a node allow for more buckets?
We have 4 CPU nodes and they can take at best 5 buckets. After which if we create more buckets, the nodes keep failing every now and then we have to keep adding back and rebalancing the cluster.
In Couchbase Server 2 most of the "background processes" that are used for replication, I/O, disk/file compaction, views, and other cleanup are per bucket.
So the reason why the documentation is mentioning as a best practice to reduce/control the number of bucket per cluster is really to avoid consuming too many resources on your servers.
We do not have a magic formula about the relation between node and buckets as it depends a lot of the volume of data and type of operations you do on them. (for example lot of mutation or not? views or not? ....)
I usually take another approach when I talk to developer about their project:
- you want multiple buckets, ok, but why? Can you explain why you need more?
Then we see if this "best practice" limit is an issue or not.
Some interesting reading in relation to your question:
- Sizing 1: http://blog.couchbase.com/how-many-nodes-part-1-introduction-sizing-couc...
- Sizing 2: http://blog.couchbase.com/how-many-nodes-part-2-sizing-couchbase-server-...