Hi Team,
Before using Magma as our backend storage engine, we ran a pillowfight test comparing couchstore and Magma with 128 vbuckets. Our CB cluster has three nodes (16 vCPUs, 32 GB RAM) with data and query services on all nodes.
We used the following command to populate data. The set-pct is 100 to ensure all six threads perform writes only.
date ; /opt/couchbase/bin/cbc-pillowfight -U couchbase://xx.xx.xx.xx/data2 -u cbadmin -P cbadmin -m 1000 -M 1000 -J --set-pct 100 --sequential -B 1 --start-at 0 -I 55000000 --populate-only -t 6 --rate-limit 30000 -p A -T ; date
We observed high disk reads during Magma (eviction policy full) bucket writes (verified via iostat till writes completes) on the data storage disk. The memory quota is 300MB, and the resident ratio is under 1%. For couchstore bucket (eviction policy value only) writes, we did not see such high reads; the resident ratio was 16%. As a result, CPU usage during writes is up to 38% higher with Magma than couchstore.
I am unable to interpret this load comparison. Could you please assist if anyone has observed a similar increase in CPU utilization when using Magma as backend storage, or if I am overlooking an aspect in my load test comparison?
Thanks,
Debasis
Hi Team,
Could you please provide your input on the abov equery.
Thanks,
Debasis
Hi Debasis,
I’m Anirudh, and I work on Magma at Couchbase. Thank you for taking the time to run some benchmarks on your own. I’ve discussed your query with the team, and there are some observations about the setup and workload we have that might be impacting your benchmarking resource utilization.
First, for a more uniform and comparable benchmarking setup, we’d recommend setting the same bucket eviction policy for both storage engines. This is especially important for policies like value-only eviction, which keeps metadata in memory and can eliminate the I/O otherwise required for metadata reads.
Second, the memory quota is a critical factor for comparable results. Setting different memory quotas between Magma and Couchstore is not recommended, as lowering the memory quota results in higher write amplification and higher I/O. Magma has higher fixed memory overheads than Couchstore at lower quotas. When memory is constrained, Magma’s write cache must be flushed very frequently, which triggers internal LSM Tree compactions that consume higher I/O. For this reason, and for any performance-sensitive benchmarking, we recommend giving Magma a minimum of 1GB memory quota. At this level or higher, the overheads are negligible and performance should be comparable.
Finally, the workload itself can influence the results. A purely sequential key workload is naturally favorable to Couchstore, which generates a highly optimized B-Tree. However, in most real-world workloads, key insertion order is more random. Magma is designed to handle this more efficiently; if you use a random key order writes or an update-heavy workload, Magma will perform better, showing significantly lower compaction compared to Couchstore, which can build up fragmentation quickly and require frequent compactions.
In short, if you use the same eviction policy and memory quota of 1 GB or higher, the CPU utilization should be similar for Couchstore and Magma.
Best,
Anirudh, and the Magma team