Weird revision for new documents in CB 5

I like the new CB release as it means we can update documents in the bucket without needing a shadow bucket.
Thus allowing shared access to buckets via REST and sync Gateway.
Also CB + XDCR use quite a bit less cpu which is good.

However we have an issue with a bucket that does not use Sync Gateway.

The spec is as follows…

Couchbase 5.0.0 or 5.0.1
Bucket with no Sync Gateway.
Create a new document via REST or web GUI and the CB revision is NOT 1. It could be 5 or 473 for example.
We normally use XDCR however this is with XDCR disabled.
Any subsequent updates to documents correctly increment the revision by 1.

I suspect what is happening is that the document is created with rev 1, but then it’s internally updated multiple times for some reason.
The reason I think this is that if XDCR is enabled and a doc is created then the replicated copy on the other CB server has rev 1 whereas the original doc has the bogus rev count.

This weird behaviour totally messes up XDCR as this sequence occurs…

Create a new doc on Server A, rev is 5 for example.
XDCR doc appears on Server B, rev is 1. (Suggesting the rev was 1 originally)
Update or delete doc on Server B rev is 2.
Rev 2 is lower than 5 so doc is never synced and doc on Server A is stale.
Update or delete doc on Server A rev is 6.
XDCR doc appears on Server B, rev is 6. (So rev 2 is lost)

If we manually remove all the documents from the bucket the behaviour stays broken.
If we purge the bucket then add a document the revision ID is 1.
But then after a while this fault returns.

We have been able to purge, then restore all documents and it works OK for a while.

Any one using CB 5 see this behaviour ?

A question in Couchbase is there a way to show revision history (like there is in CBL) either in the Web GUI or programatically.

Note that we have been using CB 4.5 for a long time and this has been working fine. You don’t see the rev count like you do in CB 5. I guess it’s done in a different way, but changes to documents on any server replicated correctly using XDCR.

Thanks,

Martin

1 Like

There’s no guarantee that the per-document revID is an absolute count of how many times a document has changed - it’s only guaranteed to be ordered with respect to other changes to the same document.

In other words - a document can start from a non 1 revID, but whenever it’s changed it’ll be guaranteed to get a higher revID.

The reason for this is somewhat complex; but it relates to what happens when a key is re-created after it’s been deleted - we don’t want to re-create the key at revID:1 as that can mess up conflict resolution (using revID-based conflict resolution this “new” key would not win.

This has been the case for a long time - (2.0 era) - see http://review.couchbase.org/#/c/11901/

Indeed that makes sense for reused keys.

However these are all documents with new keys. So the documents have never existed before.

Also if XDCR is on, then the replicated document has revision 1.

Thanks,

Martin

It applies to /all/ keys - if a document was deleted (and the tombstone has been purged), we have no record of that keys’ last revID; therefore there’s simply a maxDeletedRevId per vBucket. See the comments in the patch I linked.

For example; you’ll see the following series of revIds, assuming starting from a new Bucket. I’m also assuming all of these keys below to the same vBucket:

INSERT('a') -> revid=1
DELETE('a') -> maxDeletedRevID=1
INSERT('b') -> revid=2 (maxDeletedRevID+1)
INSERT('c') -> revid=2 (maxDeletedRevID+1) 
DELETE('b') -> maxDeletedRevID=2
INSERT('e') -> revid=3 (maxDeletedRevID+1) 

(It’s possible I’m slightly out by one - this is just from memory - but you get the idea hopefully).

Thanks for this explanation. We see varying initial revision rather than the same number.
I just created 4 new documents (without deleting anything) and got revisions 8, 37, 44 and 11.

However this behaviour would be OK if the replicated documents had the same revision.
But they don’t, they always have revision 1. Then if you edit the replicated doc, it’s revision becomes 2, which is lower than original doc revision so XDCR does not replicate it. Then we have a stale doc on the other server.

Unless I pause XDCR then create a doc, it gets revision 13 for example, then resume XDCR the replicated doc also has revision 13. Then editing the doc on either server bumps the revision to 14 on both.

So our issue is that when XDCR is active it incorrectly replicates the revision for a new doc.

Martin

Thanks for clarifying.

I can’t explain what you’re seeing - that doesn’t sound expected. What XDCR conflict resolution mode (revID/ time-based) are you using?

You might want to raise an issue on XDCR at http://issues.couchbase.com if you believe you have a regression since 4.x.