Optimize Couchbase for map-reduce vs. key-value store
I'm test driving couchbase as a possible switch from couchdb. Couchdb is quite slow when creating new views and doesn't exude the same level of polish as couchbase. I already developed an app to export our 4M records from couchdb and bulk import them into couchbase using 10 concurrent threads with back-off strategy when couchbase write queues are running behind. On my macbook, using centos under vmware fusion, I get about 3.5K records/sec when uploading which is good enough for our immediate needs.
However, it has one thing going for it that is currently a deal killer to switch to couchbase for us. Couchbase seems to insist to keep all keys in memory, which is useless to us since we write records only once and then map-reduce them. The killer feature for us in couchdb/couchbase has been its view management. Every once in a while, there might be a corrupted record that makes its way into the views. When that happens, we only need to find the offending record, delete it, and the views are automatically updated to reflect the dataset change. This capability alone is what I really love about couchbase/couchdb. Couchdb is however excruciatingly slow. Adding a new view takes ~36hrs to compute, which is not acceptable. Couchbase, no the other hand, does it in ~1hr.
That said, couchbase keeps running low on memory because it keeps the 4M keys in memory, which is totally pointless for our application. We generally store documents only once and then never directly retrieve them again.
My question, is there are a way to configure couchbase to meet our needs? Or should I be looking at a different solution?