Disk Write Optimization?
I'm working up a scenario where I update an object several times over a short period. After the first retrieval, I can pretty safely assume that the object is in memory, so I can reasonably bank on the notion that subsequent retrievals will come from cache. For each inbound event, I'll retrieve and update -- every time -- replacing the cached image.
So my question is about the write behind log. Let's say I've got 1M keys/values queued to write to disk, but only 500K of the keys are unique -- there are several values for the same key in line to be written to disk. At this point, the ones earlier in the queue are already stale -- only the last one matches the current cache image.
Is there a mechanism in place that optimizes the write-behind so that the earlier images get removed from the queue?
Thanks!
Paul
To some degree. Certainly writing the current cached image makes a lot of sense.
Is the pointer data structure some sort of tree so that we have a guarantee it doesn't exist twice (i.e. then the cached image would get written twice)? That would certainly be consistent with the potential out-of-order writes (which you're correct, is not a big deal).
Thanks,
Paul
Once an item is marked as "dirty", it doesn't get put on the queue if it changes again so you don't have to worry about it existing twice.
Thanks Perry!
Paul
Sounds good Paul.
Paul,
Membase doesn't actually store any values in the disk write queue...it is simply a list of pointers to items that have been changed.
When the disk write process goes to actually send an item to disk, it retrieves the latest value that is in memory.
This is a very effective way of doing write deduplication, though it will also lead to "out-of-order" writes to the disk...usually not a big deal, just thought I'd bring it up.
Does that answer your question?
Perry
Forum support is great for free but sometimes you need a guaranteed response time and dedicated resources for your questions or issues.
Consider purchasing enterprise-level support from Couchbase: http://www.couchbase.com/products-and-services/overview
Call or email "sales -at- couchbase-dot- com" today!