Durability / Consistency Enhancement
It would be nice to be able to tweak the durability of the system. Basically add some of the features of other cluster systems where quorum reads/writes are supported.
I like the approach that a certain apache product has in that you can specify on a read/write how consistent you want the data. Something like:
set(Key, Value, Expiration, Consistency, Durable)
Where Consistency is a set of possible values:
ONE : Any replica has received the data. (Weaker consistency than today)
MASTER_ONLY : Master node has received the data. (Same as today)
MAJORITY / QUORUM : Majority of replicas have received the change. (Better consistency, slower)
ALL : All replicas have received the change. (Slowest)
Where Durable is either:
TRUE: All nodes only report when data is committed to disk.
FALSE: Nodes report immediately upon receipt of the data. (Memory write is ok.)
Similar with read:
ONE: Any replica responds. (Weaker consistency, more HA)
MASTER_ONLY: Master node responds with data. (Same as today)
QUORUM: Majority of replicas respond with consistent values. (More consistent, possibly better HA since master node can be down)
ALL: All replicas must respond with consistent values.
I read the very little information about what is planned with the "observe" command soon. Would it be possible to build this framework on top of that?
This would allow a client to write/read data in a fashion that matches its requirements for the various pieces of data.
If easier, this could be set on the bucket itself instead, at the expense of not being able to store data in the same bucket using different schemes.