Lag and data inconsistency with Transactions

java sdk: 3.4
Couchbase server version: 7.0.4

I was going through the documentation here- Error Handling | Couchbase Docs

As per this, even if txn is complete, the documents from ATRs are still not unstaged and thus non-txn reads will see old version.

  1. While the documentation says txn and non-txn APIs should not be used together, you could easily have a case where you perform txn writes, and then do non-txn APIs to perform reads further. This design puts too much burden on the developer to be well aware of internals of txns and use mandatorily txn APIs only if you do not want to degrade performance with waits after txn.

  2. From a user perspective, this behavior is data inconsistency, and eventually consistent view of data.

  3. I benchmarked txn vs non-txn write performance and the difference in 2-7x easily. Single document non-txn insert takes ~2ms while txn write takes ~12ms. Can someone share the details which would cause this? i understand ATR writes can cause this. How can this be reduced?

“even if txn is complete, the documents from ATRs are still not unstaged and thus non-txn reads will see old version.”

If a txn is successful, the txn is unstaged and non-txn and txn reads will see the new version. If a txn fails, it remains staged, and non-txn and txn reads will see the old version.

  1. Yes, txns function as described. So to use them as described, one must understand how they function. If you understand how txns function, then you won’t be surprised by the behavior. Graham can correct me if I’m wrong, but reading with a non-txn should be okay as long as you understand that non-txns will not see txn changes until the txn is committed.

  2. The data will be what is expected given the behavior of txns and non-txns.

  3. Txns require more work than non-txns, thus require longer to execute. There is no way to avoid the extra work in a txn. That extra work is the staging, application of the staged txn and unstaging.

If transactions are not for you, you may wish to use CAS (optimistic locking), which has almost no overhead and is easier to understand.

Hi @Ankur-Shukl
Thank you for your feedback. To add to what my colleague Michael has written above -

First, I would strongly advise using the latest Couchbase SDK and server so you have access to all fixes and performance improvements.

As per this, even if txn is complete, the documents from ATRs are still not unstaged and thus non-txn reads will see old version.

Apologies if you got this impression from the documentation, but this certainly is not the case. After a transaction is complete all documents will be either committed or rolled back.

There are a few edge cases where this isn’t possible - if the transaction times out mid-commit, for instance. In these scenarios the asynchronous background transactions cleanup will find and finish the transaction soon after.

  1. Yes as Michael says the restriction is only on using transactional and non-transactional write operations together. Read operations can be safely intermingled.

  2. Totally agree with you! Hopefully these answers are going some way to put your mind at ease.

  3. When you benchmarked, did you have the non-transactional writes using durability? If not, that will be a significant part of the performance difference. Transactional KV ops by default are at MAJORITY durability while non-transactional KV ops are at no durability by default. Durable writes are somewhat slower by nature.

The main things you can do to improve transaction performance are:

  • If you can, structure your data model to avoid them - JSON documents provide rich capabilities to embed data. While we aim to keep transactions performance as high as possible, there is an inevitable penalty over single-document writes.
  • Lower the durability level to MAJORITY (the default), if you are running at one of the PERSIST levels. Unless you need that level of data safety, of course.
  • Stage your writes inside the transaction in parallel - see example. This should have a dramatic effect, especially on larger transactions.
  • Wait for the forthcoming Java SDK release that is going to parallelise the commit phase, which should dramatically improve transaction times.

And of course, anything you do to improve performance of non-transactional reads and writes, will also impact the transactional versions.

1 Like