As per documentation and other posts, transactional APIs do not allow setting an expiry for documents.
However, since I need transactions as well as need to set expiry, is the following a valid strategy?
do transactional insert
Use touch API to set expiry
Couchbase documentation strongly suggests against using transactional and non-transactional writes in parallel for a document. In above case, the non-transactional API is setting expiry, and not updating the document data. Is touch an atomic operation, and can I assume that in this approach, for 1 document, the data will never in inconsistent state?
I am thinking of this scenario-
Time T1- D1 created
T2- txn starts on T1 and stages an updated version.
T3- D1’s expiry time is changed
T4- txn commits
In this case, when T2 commits, it would end up resetting the expiry to 0. Is there a way to preserve the document expiry by a transaction? A txn anyway does not allow setting the expiry, so why should it reset it?
Hi @Ankur-Shukl
First to confirm, yes expiry is not currently supported in transactions. We are monitoring that request and it has come up a few times, and we do encourage the community to let us know what features you’d like us to prioritise.
I wouldn’t recommend the approach there. It’s mixing transactional and non-transactional writes for one. For another, the document could expire mid-transaction, creating unintended effects. You will have to make assumptions about the staged insert that are based on current implementation details that could change in the future. And yes, the commit will not preserve the expiry so it will be lost anyway.
Could you apply the expiry after the transaction fully commits and returns? At that point it is a regular document, and as long as you are not mixing transactional and non-transactional writes on that document, all will be well. Of course you won’t have the transactional guarantees of ACID and rollback. But whether that is a crucial distinction will depend on how vital the expiry is.
A txn anyway does not allow setting the expiry, so why should it reset it?
This is the default behaviour of all KV operations, including the ones used under-the-hood by transactions - unless explicitly told to, they will not preserve the expiry. They can be explicitly told to either by re-sending the expiry each time, or using the preserveExpiry() option that’s supported by Couchbase Server 7.0+. However, like expiry itself, neither is presently supported by transactions.
Hi @graham.pople
I guess I did not write the example correctly. I was suggesting what you have mentioned.
Ti- denotes a timestamp
Di- denotes a document
Txn- denotes a transaction
T1- Txn1 creates D1 and commits
T2- Txn2 begins and stages old D1 in XATTR
T3- Non-transactional expiry add to D1 via touch call
T4- Txn2 commits
There can be many parallel threads performing writes via different APIs eg: transactional insert, touch etc. while I am not running both a txn and then touch call within same method, another thread could run a touch call. So txn2 will end up removing the expiry set by another parallel thread since you mention like expiry itself, neither is presently supported by transactions.
So even if I am not mixing transactional and non-transactional writes within one method, they can get mixed since there are many parallel threads.
" like expiry itself, neither is presently supported by transactions."- There are 2 statements of importance here-
a. txn and non-txn writes should not be mixed i.e. do not use both types of APIs for a document
b. txn does not support any kind of expiry, not even preserveExpiry().
This means again that documents created via transactions were not intended to have an expiry. From a DB perspective, documents created via or without txn should have no distinction. Then I do not understand why expiry was intentionally left out.
Statement b above means that even if I set expiry outside of a transaction, next transactional write to same document will always reset expiry. So there appears to be no way to set expiry for a document.
Is there a recommendation you can give around how expiry can be added to documents which were created via a transaction? If this is not feasible, what could be a good way to build atomicity with CB so that I can do bulk inserts and add expiry atomically? (I am thinking of creating a meta docment to emulate a lock, but that requires building a lot of application logic)