Documents truncated on downwards sync from CouchbaseLite 2.0

Mobile Couchbase Lite
1

This occurs with both the android and ios SDKs on db23:

1)Set up two devices with couchbase using PULL only replication.
2)Sync down the database on both devices. At this point, all documents are available on both devices
3)On device 1, update a document (for example, change the value of an email field) via the Sync Gateway REST API (we can’t use PUSH replication because our validation logic is too complex for the sync gateway’s validate hook).
4)On device 2, use PULL replication to pull down the new changes. Notice that the document updated by device 1 is truncated- the document will exist and will not be deleted, but it will have no fields.

This will be the case even if you uninstall and reinstall the app, starting with a fresh installation- the initial sync will have a truncated document.

When looking at it on the Couchbase server, the document has data. When querying it via the REST API for use on web, the document has data.

Example of one of those documents on the server:

SELECT *, meta() FROM data WHERE email LIKE “barb%”

[
    {
        "$1": {
            "cas": 1523467936225034240,
            "expiration": 0,
            "flags": 33554432,
            "id": "648f0f18-cc8a-4150-8e9f-cb56239e58eb",
            "type": "json"
        },
        "data": {
            "$schema": "",
            "_sync": {
                "cas": "",
                "channels": {
                    "Demo": {
                        "rev": "2-31c7ccc90fa7011d9757d2b61b658eb0",
                        "seq": 27
                    }
                },
                "history": {
                    "channels": [
                        null,
                        [
                            "Demo"
                        ],
                        null
                    ],
                    "parents": [
                        2,
                        -1,
                        1
                    ],
                    "revs": [
                        "3-6717498ab5a8a0b8a61001c5df18f961",
                        "1-e91e28f561027aa8f8e631ae194d2e14",
                        "2-31c7ccc90fa7011d9757d2b61b658eb0"
                    ]
                },
                "recent_sequences": [
                    7,
                    27,
                    28
                ],
                "rev": "3-6717498ab5a8a0b8a61001c5df18f961",
                "sequence": 28,
                "time_saved": "2018-04-09T18:04:09.901965149Z"
            },
            "email": "barbara@yaplc.com",
            "hs": {
                "account": "Demo",
                "channels": [],
                "create": [
                    "Demo@REP",
                    "Demo@DIRECT"
                ],
                "ctime": "2018-04-09T18:04:09.877Z",
                "cuser": "buyer@customer.com",
                "delete": [
                    "Demo@REP",
                    "Demo@DIRECT"
                ],
                "mtime": "2018-04-09T18:04:09.877Z",
                "muser": "buyer@customer.com",
                "type": "Demo::Customer",
                "update": [
                    "Demo@REP",
                    "Demo@DIRECT"
                ],
                "version": 1
            },
            "name": "Thomas PLC",
            "uuid": "648f0f18-cc8a-4150-8e9f-cb56239e58eb"
        }
    }
]

Any help is appreciated, this is a showstopping bug.

2

Hmm, so my initial questions would be:

  1. Can you share exactly how you’re updating the document? The exact request that hits Sync Gateway would be perfect here.
  2. The document has 3 revisions - one from when it was created, and then two updates. Only the 2nd revision has the document in any channels - Demo. What channels are you expecting this document to be in at each stage?
  3. What channels does the user you’re connecting as have access to?
3

Hmm. I would have expected the channels to go unchanged. The user should have access to DEMO. The actual path our data takes is: modify on client, send to our server api (which does our validation), then the server writes to the SyncGatewatay. I’ll work with the server folks here to figure out what the exact query was, and if channels was getting dropped on our end. I hope its that simple, I’d much rather this be our bug and CB is working perfectly.

4

I would be curious- if it was a channels problem, why would there be an empty document rather than none? None would make sense, an empty document seems odd. Although our document names are just UUIDs, so it isn’t a security issue if we do have blank documents.

5

So, assuming your sync function just assigns the document to the channels in the top level channels property, it looks like your server is emptying that array as it writes it in.

I’d have to check the API spec on that, but under the covers if the user has lost access to the document, they have to be synced something in order to remove it locally.

@hideki - do you know off the top of your head what gets returned for a _removed revision?

Edit: I hadn’t noticed you were using iOS as well, @pasin maybe a question for you then on _removed revisions…

6

This bug is on us, not on you guys.

We dove into this and found an inconsistency in our schema data where part of our codebase wasn’t applying channels and part was. Depending on which path you moved through, the channels could have been stripped. When they were stripped, we got saw this effect. I find the empty documents a bit odd, but they won’t be a problem.

Thank you for your advice, and hope I didn’t panic anyone too much.

1 Like
7

The empty document is to reflect the fact that the client no longer has access to it. The document isn’t strictly-speaking deleted, but CBL shouldn’t keep the previous obsolete revision either. Our compromise was to send the client an empty revision (actually it isn’t entirely empty, it has a property _removed:true.)