Supplying CAS to set_multi() or replace_multi() ...

Gentlefolk,

Neither set_multi() nor replace_multi() appear to have a way for me to supply them a CAS value per item. delete() does supply such a mechanism. It is documented in the Python SDK connection.delete() section:

Remove multiple keys with CAS:

            oks = cb.delete({
                "key1" : cas1,
                "key2" : cas2,
                "key3" : cas3
            })

Oddly, delete_multi() does not support supplying CAS. It does not appear that set() or replace() allows a dictionary as delete() does for the keys.

There are three best practices I am trying to follow in building my app. 1) Use optimistic locking. 2) Use idempotent mutation operations to allow optimistic locking with retries. And 3) use bulk operations. To follow these practices, I must track and maintain the key/value/cas triple tuple per document. If the bulk APIs do not support CAS, then I have to abandon bulk operations or use pessimistic locking and endure the performance hit that strategy entails. I doubt that either is the intention of the Couchbase team.

In my opinion, the Couchbase Python SDK is making it difficult to follow these three best practices. Clearly, the Couchbase team and the user community needs to have a discussion around how to actually follow these best practices using Python.

Finally, if the Couchbase Python SDK documentation is in massive error, then that needs to be corrected. While this product has open source roots, your salesmen remind me in email that it is a commercial product. They seem to be in a hurry for me to finish my prototyping effort. Perhaps the documentation might be in the way of my completing this prototype? Or perhaps the SDK is not really a v1.0 design and it has been inappropriately promoted to that status?

This SDK could be better. Lets start to talk about how and why.

Andrew

P.S. to put my operational bona fides in place, I have a pair of daemons inserting and deleting items to a 10+ million document database. I do bulk reads and use single operation CAS set()s and delete()s. I am happy with the results so far but I haven't really started stressing things nor aggressively using the database.

P.P.S. I've also attended on of your Couchbase Dev Days. Hence, I think I have a pretty good intellectual understanding of your technology. While I'm a Python and Couchbase noob, I know my data and its structure very well.

1 Answer

« Back to question.

set_multi does allow a CAS. However this is currently not supported or documented (it will likely be exposed in future versions). The key issue is allowing the "idiomatic" way of setting values using a dictionary, and also being able to supply the CAS.

There's actually support for this. The key issue with not being able to supply CAS to set_multi is because of the ambiguity of "is it a value? or is it a set of parameters". For this reason there is an 'Arguments' class designed for this purpose. Unfortunately it has not been documented in the current version, as tests still need to be written for it.

It's available from couchbase.Arguments; and it's a dict subclass which accepts "value", "cas", and "ttl" fields. You use this as a dictionary value for each item you pass into the dict for set_multi.

In general; the set() family of methods accept more than a single parameter for each key; for example there is "ttl", "flags", "format", "value", and "cas" (the former is not applicable in add() though).

Feel free to file bugs on our issue trackers if you think there are features missing.