Couchbase.Core.Exceptions.KeyValue.DurabilityImpossibleException

.NET SDK
1

Hi all,
we have an issue using the transaction library.
Every time that we try to perform and insert we receive the Couchbase.Core.Exceptions.KeyValue.DurabilityImpossibleException

There is something wrong in the configuration ?

Exception
Transaction did not reach commit point Couchbase.Transactions.Error.TransactionFailedException: Transaction failed.
—> Couchbase.Core.Exceptions.KeyValue.DurabilityImpossibleException: Exception of type ‘Couchbase.Core.Exceptions.KeyValue.DurabilityImpossibleException’ was thrown.
at Couchbase.Core.Retry.RetryOrchestrator.RetryAsync(BucketBase bucket, IOperation operation, CancellationTokenPair tokenPair)
at Couchbase.KeyValue.CouchbaseCollection.MutateInAsync(String id, IEnumerable`1 specs, MutateInOptions options)
at Couchbase.Transactions.DataAccess.AtrRepository.MutateAtrPending(UInt64 exp, DurabilityLevel documentDurability)
at Couchbase.Transactions.AttemptContext.b__55_0()

Cluster configuration
Enterprise Edition 7.2.0 build 5325
11 Data Nodes
4 Query /Index Nodes

Bucket Configuration
Type: Couchbase
Bucket RAM Quota: 558GiB
Cluster RAM Quota: 51.1GiB
Replicas: 1
Server Nodes: 11
Ejection Method: Full
Conflict Resolution: Sequence Number
Compaction: Not active
Compression: Passive
Storage Backend: Magma
Minimum Durability Level: none

SDK Version
Couchbase.Extensions.DependencyInjection 3.4.14
Couchbase.Transactions 3.4.14

Code

public async Task<bool> SaveData(Request application)
{
	try
	{
		var transactions = Transactions.Create(await _bucketManager.GetCluster(), TransactionConfigBuilder.Create());
	
		await transactions.RunAsync(async (ctx) =>
		{
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "Customer_v1", application.CustomerInfo.CustomerId, await _couchbaseMapper.ToCustomer(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "Account_v1", application.AccountInfo.AccountId, await _couchbaseMapper.ToAccount(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "Card_v1", application.CardInfo.CardId, await _couchbaseMapper.ToCard(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "Cardholder_v1", application.CardholderInfo.CardholderId, await _couchbaseMapper.ToCardholder(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "CardBalance_v1", application.CardInfo.CardId, _couchbaseMapper.ToCardBalance(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "CustomerBalance_v1", application.CustomerInfo.CustomerId, _couchbaseMapper.ToCustomerBalance(application));
	
			await _bucketManager.InsertCouchbaseItemAsync(ctx, "AccountStatement_v1", _couchbaseMapper.GetAccountStatementId(application), await _couchbaseMapper.ToAccountStatementAsync(application));
			
			await ctx.CommitAsync();
		});
	}
	catch (TransactionCommitAmbiguousException)
	{
		Logger.Warn("Transaction possibly committed");
		return await _bucketManager.ExistAsync("AccountStatement_v1", _couchbaseMapper.GetAccountStatementId(application), "TrySaveApplicationOnCouchAsync");
	}
	catch (TransactionFailedException ex)
	{
		Logger.Error(ex, "Transaction did not reach commit point");
		return false;
	}
	
	return true;
}

public class BucketManager : IBucketManager
{
	private readonly INamedBucketProvider _apolloBucketProvider;
	
	public BucketManager(INamedBucketProvider apolloBucketProvider)
	{
		_apolloBucketProvider = apolloBucketProvider;
	}

	public async Task InsertCouchbaseItemAsync<T>(AttemptContext context, string collectionName, string key, T entity)
	{
		var bucket = await _apolloBucketProvider.GetBucketAsync();
		var collection = await bucket.CollectionAsync(collectionName);
		await context.InsertAsync(collection, key, entity);
	}

	public async Task<ICluster> GetCluster()
	{
		try
		{
			var bucket = await _apolloBucketProvider.GetBucketAsync();
			return bucket.Cluster;
		}
		catch (AggregateException ae)
		{
			ae.InnerExceptions.ToList().ForEach(x => { ExtractInnerException(x, "GetCluster", "", ""); });
		}
		catch (CouchbaseException ce)
		{
			ExtractInnerException(ce, "GetCluster", "", "");
		}
		catch (Exception e)
		{
			ExtractInnerException(e, "GetCluster", "", "");
		}

		return null;
	}
}

Regards
Federico

2

The Durability in the transaction cannot succeed.
Maybe it’s needing to do Majority and there are no replicas.

3

I suppose that I don’t need to set the durability because, getting form documentation, i found out this:

  • majority — replicate to a majority of the replicas before acknowledging the write. This is the default level.

Regarding the replica i fond out this:

If using a single node cluster (for example, during development), then note that the default number of replicas for a newly created bucket is 1. If left at this default, then all Key-Value writes performed at with durabiltiy will fail with a DurabilityImpossibleException .

But, I’m using a cluster with 11 nodes (not single node cluster)

Regards
Federico

4

Try setting the transaction durability to NONE like suggested in the linked forum post.

5

Yes, I could try it but in the documentation I found out this sentence:

Caution: A level of None is present but its use is discouraged and unsupported. If durability is set to None, then ACID semantics are not guaranteed.

I’m in an enterprise company, and I’m not sure that I can use unsupported features.

Regards
Federico

6

Try it for troubleshooting purposes. Not as a permanent solution.

7

Setting the log to DEBUG I have also retrieved these couchbase logs:

Couchbase.Transactions.DataAccess.AtrRepository…ctor|85||Requested Durability = Majority

  • Couchbase.Transactions.DataAccess.AtrRepository…ctor|85||Requested Durability = Majority
  • Couchbase.CouchbaseBucket.SendAsync|85||Sending op SubMultiMutation with _txn:atr-325-#a69 to servername.serverdomain.net:11207 using VBID: 325
  • Couchbase.Core.ClusterNode.SendAsync|85||CB: Current state is Disabled.
  • Couchbase.Core.ClusterNode.ExecuteOp|85||Executing op SubMultiMutation on servername.serverdomain.net:11207 with key _txn:atr-325-#a69 and opaque 41477
8

@Federicus.One

That’s very odd. With 11 nodes and 1 replica, Majority durability should not be failing.

Could you possibly try doing a non-transactional KV operation with Majority durability following these docs? This is just to get an extra data point though, I assume it will fail in the same way.

1 Like
9

One possibility is - As described in the linked post, if the bucket was created with 0 replicas, then changed to 1 replica, the vbuckets will not be replicated until a Rebalance is done.