Does empty disk space on a new node need to be the same (or higher) as the current nodes? I have some large value ejection buckets and a few servers with some very large disks to handle overflow. If I add some new nodes with RAM but smaller disks (still greater than RAM) will that negatively impact the maximum size the value ejection buckets can grow to?
Internally, CB will partition the data across all the nodes in the cluster. So when you add a new node in to the cluster, the amount of data you need to store on each node will be reduced. Corresponding, the disk requirements on each individual node will be reduced as well. The only time when CB will require more disk space is during compaction or failover.
So going back to your case, it’s not required that each new node must have the same(higher) empty disk space as the current node. But enough empty disk space should be preserved for the compaction or just in case another node fails.
Please see the excerpt from the blog about disk space here:
Depending on workload, your required disk size may range anywhere from 2-3x your total dataset size (active and replica data combined) due to the append-only disk format. Heavier update/delete workloads will increase the size more dramatically than insert and read heavy workloads. In reality, the size is likely to grow and shrink significantly over the course of time as the automatic compaction process runs. The 2-3x number comes more from this need to expand rather than your data actually taking up more space on disk.