Active Transaction Records Cleanup

Hi there,

I’m using .NET Client library v3.1.4 and a few days ago I introduced Couchbase.Transactions (v.1.0.0) in my project.
I read that this library leverages couchbase transaction functionalities by using Active Transaction Records documents (prefix _txn:atr-) but i noticed that these documents were never deleted even if the transaction is completed correctly.
Documentation says that these documents, which may persist indefinitely in the bucket, are maintened by Couchbase Server (Transaction Documentation)

Do these documents live forever or is something missing in my configuration?
I’m using Couchbase v6.6 Community Edition.

Thanks to everyone who will clarify this aspect for me.

@Alessandro_Gargiulo -

Welcome to Couchbase Forums! @Richard_Ponton should be able to help you with this question.

Jeff

Thank you @jmorris

@Richard_Ponton can you help me with this question?

Hi @Alessandro_Gargiulo. There are a static number of ATR documents (1024 in most SDKs, and .NET Transactions doesn’t supply a way to change that, currently) that will persist. The individual entries inside that document are cleaned up, both when a transaction finishes and by a background thread looking for expired transactions from any and all processes. So if you have no transactions running, all of those documents should be empty.

Thank you for the answer @Richard_Ponton . This is what we were looking for :slight_smile:

For educational purposes: what is the motivation behind the choice to persist 1024 ATR documents?

The implementation uses the sub-document API extensively, under the covers. A fixed number of documents that doesn’t grow steady states into atomic updates, rather than dealing with the edge cases of multiple simultaneous access where one client is trying to update and another client has cleaned up the root document.

@Alessandro_Gargiulo each transaction attempt needs to write 3 times to an ATR, so having a large fixed pool of them helps us distribute that load and avoid any contention issues. That’s the main reason.

In addition, the Couchbase architecture involves partitioning all key-value data into a number of ‘virtual buckets’, or vbuckets, and there are 1,024 of these by default on Linux and Windows, per bucket. Having at least this number of ATRs allows us to guarantee that the ATR modified by the attempt is on the same vbucket as the first document mutated in that attempt - this should allow some performance wins under the hood, as those writes will often be conflated in the same fsync.

The reason not to set it to something even higher than 1,024, is mainly that the cleanup algo regularly polls all the ATR documents. So there’s a balance to strike there. 1,024 ATRs means ~17 reads per bucket, per second, which we expect to be negligible traffic on most deployments.

We keep the ATR documents around because generally on any system regularly doing transactions, deleting them would just be adding unnecessary writes - e.g. the ATRs would simply be recreated within seconds.

It’s very interesting understand how the things work under the hood.
Thank you for the detailed explanation @graham.pople @Richard_Ponton