Revoking access to a document by clearing its channels

Hello :wave:

We have some difficulties with Couchbase Lite, when we decide to revoke the access to a document.
For example, we want to remove the access to a document for a user after a validation from our API.

By referring to the documentation and various answers on the forum, clearing the document channel allows the user to no longer access the document but also to automatically purge it from the local database.

However, we do not achieve this result: documents with an empty channel are still present on the phone and they are still downloaded.

We test in this way:

  • New application, therefore new database, we connect with a user.
  • We wait a little while the synchronization is done.
  • We recover the local database and analyze it (adb + cblite). We check for the presence of a document X.
  • We empty the channel of the document X on the remote Couchbase side.
  • We wait for the synchronization to be successful.
  • The database is recovered and we check the absence of document X. However, it is still present.

We also checked that the user does not have access to the admin channel (*) and that the document is not public (!).

A remark we made is that the document contains in these metadata a channels property still containing the user’s email address:

     "channels": {
        "user@mail.fr": {
          "seq": 985436,
          "rev": "94-51c704ba1cb4bf4816b4817d600e4cde"
        }
      }

We check the list of documents to a channel, thanks to the get /db/_changes REST API.
We saw a lots of documents, with "remove": ["user@mail.fr"] :

{
  "results":[
   ...
  ,{"seq":985436,"id":"X","removed":["user@mail.fr"],"changes":[{"rev":"94-51c704ba1cb4bf4816b4817d600e4cde"}]}
  ],
  "last_seq":"986249"
}

Does this have anything to do with caching revisions and / or channels? What did we miss ?

CBL : 2.8.5 and SGW : 2.7.1

Thanks for your help

1 Like
  • Can you elaborate on how are you are “clearing the document channel”?
  • Can you please include the CBL and SGW versions .
  • Why are you “recovering” the database and inspecting it manually with cblite tool ? What happens when you query for documents in database in the app via the CBL API (or just retrieve a document via Id) ? Do you see that document returned in the query response

It’s a bit confusing with the email address etc - Is that the channel Id and the docId?
In any case, to scope the troubleshooting

  • for the document in question, please check the output of the _raw REST command. See what docs the document is attached to
  • For the specific user, please check the output of the user REST call. See what channels the user has access to

Auto purge requires that the user lose access to the document via all the channels that user has access to. So the output of the above should confirm state of system

Next is to look into sync itself. How do you know that the sync was in fact successful.

First of all, thanks for your response.

Can you elaborate on how are you are “clearing the document channel”?

In our API, we have something like this :

executeQuery("UPDATE " + bucketName() + " SET channels = [] WHERE conditions"); 

I also test manually by clearing the document channels directly on the Admin UI (Documents > Document ID = > Edit Document > Clear the channel > Save)

Can you please include the CBL and SGW versions .

I updated the initial post (CBL : 2.8.5 and SGW : 2.7.1)

Why are you “recovering” the database and inspecting it manually with cblite tool ?

I’m sorry, “recovering” was not the good word. I simply pull the database into my computer thanks to adb, and then open it with cblite.
I think this is the best way to check what is inside my database because we don’t display every documents in our app (some are hidden depending of their state).

What happens when you query for documents in database in the app via the CBL API (or just retrieve a document via Id) ? Do you see that document returned in the query response

The following code check if the document 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7472(with an empty channel) exist :

    fun getById(id: String) {
         db.getDocument(id)
               ?.let { Timber.d("The document $id does exist") }
               ?: Timber.d("The document $id does not exist")
    }

And the output is :

2021-04-16 17:51:15.533 D/VisiteDao: The document 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7472 does exist

It’s a bit confusing with the email address etc - Is that the channel Id and the docId?

The channel ID is the user email address. This is not the document ID.

Great, I didn’t think about this command. The _raw REST command give us a strange document indeed :

{
  "channels": [],
  ... // Document properties
  "_sync": {
    "rev": "94-51c704ba1cb4bf4816b4817d600e4cde",
    "sequence": 985436,
    "recent_sequences": [
      976147,
      976152,
      976155,
      976157,
      985430,
      985432,
      985434,
      985436
    ],
    "history": {
      "revs": [
        "66-8a685bb4c4eaee47106fd2d2dcbc19af8caeca45",
        "7-272bdbd92997ca536ef4ec5d78e8486b0e5c2f9d",
        "82-279cbe86130884d13f8e5115d7e1c711073f2c49",
        "45-ca7c35d2bc22d1a04af5e2685a8a3104b19a90b0",
        "54-17c209c8104e4ec1e2d60e07388c7718fbbd66d0",
        "29-550d1b17b344f833387c167d7f0d9c21307aaeab",
        "80-a1db141b1bb3d83ac4c114bdeec81bba9cb2fd0f",
        "76-67c9e30b8d7ad377a1c0bc47fb8ab4bef88e5fe7",
        "49-80ce59106d35739e238421fbe3e7a0d9938645a3",
        "38-f0e0d5058f5b494a34a131fb4f833561a3e28db9",
        "77-fa6da5fbb611f69fc1b9cf16a79175736a2daf74",
        "48-a07cc935c4a2afcd35f6c5caff275dc17e17655a",
        "46-94e4060ba068b1e3bb1ef9a41ca237b6f5119aa6",
        "13-8dafe128081cedad2c0afa722f1ca510be1c485a",
        "42-722b7750fb43423cdab6ecbfc9deea0075d70ef3",
        "24-6544ebec6160c6c0c898dcf90c29ebc0ad03904b",
        "2-ea81340dc2ea8c9f3c746dbe960241e4",
        "8-6d93c566124a3caf086cea6f87ff3ba47a30f25f",
        "69-330362ef3ee326b3396278c8ec93938eeea0aa40",
        "61-8a1cb8457a734ce207b23ee3756733ef626c09b3",
        "81-cf8411f0238ff6dfb1fc3a26aed14260dfb82cfe",
        "55-c09058d375e5bccbad9229572e6110894ac4f6fb",
        "30-6a7f1f8b5ec6ccb0434681e799c1da8d6b437a5a",
        "78-6edd2981122ab4a2f6b50b565e0b0ce3764bdd0d",
        "11-99bf03d3dcccc96be3940f5930c422918ba0fd2a",
        "41-839b19fd778c910ecdb6ac7d7113e6a260187f93",
        "94-51c704ba1cb4bf4816b4817d600e4cde",
        "37-2f4c239e96e3b9ca7902cf8056cca4a9561a8df0",
        "6-21b13abb618ad772cd9e9e257dd69a046048c04e",
        "34-42c6956a2694c8e1f5b1d4da82e596ad4232b97b",
        "84-52bf56c2420e1793414b25ea46145c6de5d12f5d",
        "86-d70680a7b8910fbcf01b8f0fc424aabd575f3e68",
        "4-100e96ac3dbf59e3cd5d25e78050992ac4fc0c7c",
        "22-4c79bcab3233def578d1e4d066fb2ef10daa0397",
        "67-318ff26ee44134feaf2dd84d565579466e3a4b33",
        "20-14e8de81682a1682fbc0322c8470ecfd83e82110",
        "56-ff5c46cdd9a7c19cb1a9b73a0ac0b93ee88f4efd",
        "51-b2af787cfd066a163f8312ab6ec7b04690e2cbc7",
        "50-a4c50e914e5d1687650cf0875288c310376c2be8",
        "65-b425b239f4cd5512d99838283c6405b49b190ed7",
        "31-299de0e18402881b02939ca5395b5620c67e7690",
        "9-1fa3b9d08c0d3f146c2458686f7ef92a0a508512",
        "28-5fd889737a9236351621175bc5ed7f9a272a3a8d",
        "17-7e2208a1dc8cda92b62dbf3a79de822157e03d2c",
        "87-2370d369cd3a43ed59ffda8b9cfbed910f826b00",
        "39-00a382cde69fbc6bf7fdc42d329c9cef9ce33894",
        "36-5df3f36a2ec7ef6bc4a021899ed9f7862372fc1a",
        "33-40319b428b8a1dee5402b6f9ec8446c492a7f80a",
        "58-42a3e92b2e82b64da2d60051e45d71ed27546d37",
        "35-d77533426b59878b62811f61fe3a8642964b2259",
        "5-0407753d2835557b9e68348fa9e837ffa68ccd86",
        "47-d0a4cbf898b0fd4e0ea79882a0cdfbb30af41f1f",
        "14-5c59498f333a7339108561aaba64abd835a9649c",
        "10-73e9198c940eba1322a6e8e8a34b1403e1fe77ac",
        "52-164f1d8b1dfef1a6ad9367919efd9c45e5801c96",
        "64-5d1fb34e15693236f3e03d8689f6f5646504fc00",
        "27-1c1c41b9cad278c310e9727db88f1402028cfca9",
        "72-eaca7c6d2ee150f3274886418d7c2ab3f41cead5",
        "53-67975ddd331a1bc50670760eaff73bddab1add30",
        "57-c49a55f7920fef92c296be39d6ffdd5933a23f5b",
        "26-4c010d2747eac3e15cedb8a7771984ea5667747a",
        "59-2bba72d557d4970c66b292af370e14dc6cd4a96b",
        "62-3221f6559e60def1891f984f9b499a8efca52e00",
        "1-19fbc33c2f43e42cd4f833b44cf154d5",
        "83-f1aacc344c7301a3db6f89de966a159fe2d1888b",
        "3-c656d7404bd3f7d3425cd7f86a789961",
        "18-62b3be883f77f1efa0221649c797a6230619c685",
        "70-eea4498fd04212a1ef94e46b049bf82bc3af1f9c",
        "88-f6376866c9bcc1103517bc079d6b8322346accc9",
        "15-cd6dcde87b9dfb75b1a27282810b4eb8f9896583",
        "75-dadd9aa7df65456f4ef264b3d3046c5e472a613a",
        "71-84c7e88ceda89782d0bac55fa3d6a5d5e290ef48",
        "21-a27e412c0b6a83f550f395de8061f352e49af0ad",
        "43-251b726d187933e65120500edd7953ed5a545d2a",
        "19-710dcf4d26c4fb45672aef6ce072753287ff0cf1",
        "68-f78957c9ee7d5fdfdc3628ca0f6b574e36ae5ad2",
        "74-0d142e8c59571676b7ac434da04bd36ae473a8a4",
        "79-bc4967b365951e8d2b9754a7cba578d34f9fcf5a",
        "93-e5af72ba52ad611976c57bf6d359199d62dac665",
        "12-729685bd28b7ab3daf19f3db11771618fedfb8de",
        "63-b6dfbff2d89e1748849021dbe39309f9b5bcee7c",
        "90-6ecd86d46b3c6ad2c2cd5fc6584a2845fe06982e",
        "44-2bc4984b5646e882c09d39025cd79faf194c91d8",
        "23-d8c3a6d0e7fbde64ab591d4d0fc944be8e4bfcef",
        "91-12a9c6df05630f26e4b40a1f2872e77a7904073d",
        "85-a0a264f0a54ee2eea017ecb1ddeb14f325bafcd2",
        "92-6ffe7dae72909ac267e146844fad9e2a8a2c74f3",
        "60-041649db7818dc30752b47cfdb60c44ed9706b6d",
        "16-d6d72c0774ad7e95cd5e42ccbd578508130351e4",
        "73-cdd667dc11bfdecce84dd9c2fa12114b20ce171c",
        "89-47000b8b2af732ce0cea3dae20172ce1a2b11f97",
        "40-1d46f9920b914d1e07e55175ce1c20a80ab2adaa",
        "32-66452c21ce773234a97e3d841c091d4f00242f7a",
        "25-ad01f9222af77c4e98f1c59b04ca8a9dcb072c34"
      ],
      "parents": [
        39,
        28,
        20,
        82,
        58,
        42,
        77,
        70,
        11,
        27,
        7,
        51,
        3,
        79,
        25,
        83,
        63,
        1,
        75,
        87,
        6,
        4,
        5,
        10,
        53,
        91,
        78,
        46,
        50,
        47,
        64,
        85,
        65,
        72,
        0,
        74,
        21,
        38,
        8,
        55,
        22,
        17,
        56,
        88,
        31,
        9,
        49,
        92,
        59,
        29,
        32,
        12,
        13,
        41,
        37,
        80,
        60,
        71,
        54,
        36,
        93,
        48,
        19,
        -1,
        2,
        16,
        43,
        18,
        44,
        52,
        76,
        67,
        35,
        14,
        66,
        34,
        89,
        23,
        86,
        24,
        62,
        90,
        73,
        33,
        81,
        30,
        84,
        61,
        69,
        57,
        68,
        45,
        40,
        15
      ],
      "channels": [
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        [
          "user@mail.fr"
        ],
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ],
        null,
        null,
        [
          "user@mail.fr"
        ]
      ]
    },
    "channels": {
      "user@mail.fr": {
        "seq": 985436,
        "rev": "94-51c704ba1cb4bf4816b4817d600e4cde"
      }
    },
    "cas": "0x00007ad280fb7516",
    "value_crc32c": "0x54eb5ff9",
    "time_saved": "2021-04-15T10:42:51.81432436+02:00"
  }
}

I don’t understand the _sync.history.channels property. Why is there a difference between the channels property from the document and channels property in the metadata ?

If the document doesn’t have cleared channels, it explains why the auto purge is not working.

For the specific user, please check the output of the user REST call. See what channels the user has access to

We already had checked the channels that the user has access, and we didn’t saw anything particular :

{
  "name": "anothername",
  "admin_channels": [
    "!",
    "agence:107731933",
    "user@mail.fr"
  ],
  "all_channels": [
    "!",
    "agence:107731933",
    "user@mail.fr"
  ],
  "email": "user@mail.fr"
}

Next is to look into sync itself. How do you know that the sync was in fact successful.

I tested on visibles documents : we use a live query, so when the sync is successful, the live query is updated, and the view is also updated.

Yes. That could explain it. The document still exists in the channel.

It looks like you have been putting the doc in and out of channels. And I see that you are doing it by modifying some property in the document and the sync function is enforcing access.

Would recommend to start with a clean slate with a new doc

  • Pls share your sync function to ensure that the channel assignment is done correctly
  • Create new dummy doc with channel assignment property .
  • Share output of _raw
  • Sync doc to client. Confirm client has doc ( see preferred option of using replication listener log below)
  • Update the doc by changing the channel assignment property to remove it from channel
  • Sync doc to client.
  • See output of replication listener log
  • Share output of _raw

Thats fine. I had understood what you meant :slight_smile: . I was getting to the point that you shouldn’t be have to manually inspect the database outside of the app. What you get through the public API of CBL will match exactly whats in the database.
I understand, that you don’t display all of docs in your app. But CBL also supports events of different granularity that you can listen for from within your app and then log such events for debugging purposes.
For testing purposes, for instance, it would be simpler if you log to console every time thee is a database change event.
Or more specifically, in this particular case, you must listen for DocumentFlagsAccessRemoved replication event that you can listen to by registering for the Replication event listener. Log event to console. You can also use the replication event listener to track any replication event which will be simpler for debugging.

Again, thanks for your help.

Here our SG configuration :

{
  "adminInterface": "0.0.0.0:4985",
  "databases": {
    "vistest-visite": {
      "bucket": "bucket_name",
      "enable_shared_bucket_access": true,
      "feed_type": "DCP",
      "import_docs": true,
      "import_filter": "function(doc) { return true; }",
      "num_index_replicas": 0,
      "oidc": {
        "default_provider": "AuthorizationCodeFlow",
        "providers": {
          "AuthorizationCodeFlow": {
            "callback_url": "callback_url",
            "client_id": "sync-gateway",
            "include_access": true,
            "issuer": "issuer_url",
            "register": true,
            "validation_key": "validation_key"
          }
        }
      },
      "password": "password",
      "server": "server_url",
      "username": "username"
    }
  },
  "logging": {
    "console": {
      "log_level": "debug"
    },
    "log_file_path": "/var/log/sync_gateway"
  }
}

I created a new dummy document 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 :

{
  "channels": [
    "user@email.fr"
  ],
  "_sync": {
    "rev": "1-30b46ab4120ed2f893a1449fb5ed518b",
    "sequence": 988292,
    "recent_sequences": [
      988292
    ],
    "history": {
      "revs": [
        "1-30b46ab4120ed2f893a1449fb5ed518b"
      ],
      "parents": [
        -1
      ],
      "channels": [
        [
          "user@email.fr"
        ]
      ]
    },
    "channels": {
      "user@email.fr": null
    },
    "cas": "0x0000c1a70f357716",
    "value_crc32c": "0xe094e3ac",
    "time_saved": "2021-04-19T10:28:52.417998936+02:00"
  }
}

In the application, I received the new document :

V/CouchbaseLite/REPLICATOR: {N8litecore4repl11IncomingRevE#186} Received revision '9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471' #1-30b46ab4120ed2f893a1449fb5ed518b (seq '988292')
D/DatabaseManager$pullReplicationFilter: Document 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 allowed to be pulled by filters
V/CouchbaseLite/REPLICATOR: {N8litecore4repl8InserterE#66}     {'9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471' #1-30b46ab4120ed2f893a1449fb5ed518b <- } seq 2406
I/CouchbaseLite/REPLICATOR: {Repl#4} documentEnded 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 1-30b46ab4120ed2f893a1449fb5ed518b flags=00 (0/0)
D/DocumentReplicationLoggerListener: Document created or modified notification received: 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471

Then, I removed its channels :

{
  "channels": [],
  "_sync": {
    "rev": "2-3a9d647e30650d2f1c684ee993eefb9e",
    "sequence": 988293,
    "recent_sequences": [
      988292,
      988293
    ],
    "history": {
      "revs": [
        "1-30b46ab4120ed2f893a1449fb5ed518b",
        "2-3a9d647e30650d2f1c684ee993eefb9e"
      ],
      "parents": [
        -1,
        0
      ],
      "channels": [
        [
          "user@email.fr"
        ],
        null
      ]
    },
    "channels": {
      "user@email.fr": {
        "seq": 988293,
        "rev": "2-3a9d647e30650d2f1c684ee993eefb9e"
      }
    },
    "cas": "0x00005bcb60357716",
    "value_crc32c": "0x5d5874ef",
    "time_saved": "2021-04-19T10:34:40.907552134+02:00"
  }
}

And in the application, I received the updated document :

V/CouchbaseLite/REPLICATOR: {N8litecore4repl11IncomingRevE#195} Received revision '9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471' #2-3a9d647e30650d2f1c684ee993eefb9e (seq '988293')
D/DatabaseManager$pullReplicationFilter: Document 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 allowed to be pulled by filters
V/CouchbaseLite/REPLICATOR: {N8litecore4repl8InserterE#66} Inserting 1 revs:
V/CouchbaseLite/REPLICATOR: {N8litecore4repl8InserterE#66}     {'9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471' #2-3a9d647e30650d2f1c684ee993eefb9e <- 1-30b46ab4120ed2f893a1449fb5ed518b} seq 2407
I/CouchbaseLite/REPLICATOR: {N8litecore4repl8InserterE#66} Inserted   1 revs in   8.11ms (  123/sec) of which 99.9% was commit
V/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#35} Database changed!
V/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#35} Notified of 1 db changes #2407 ... #2407
I/CouchbaseLite/REPLICATOR: {Repl#4} documentEnded 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 2-3a9d647e30650d2f1c684ee993eefb9e flags=00 (0/0)
I/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#35} Found 0 changes up to #2407
V/CouchbaseLite/REPLICATOR: {Repl#4} Saving remote checkpoint 'cp-3BPtEyNBRkQzdXl/ZioB69Xu4TM=' with rev='0-456': {"time":1618821276,"local":2407,"remote":985137} ...
V/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#35} Asking DB for 200 changes since sequence #2407 ...
V/CouchbaseLite/REPLICATOR: {N8litecore4repl6PusherE#35} Waiting for db changes...
I/CouchbaseLite/REPLICATOR: {Repl#4} Saved remote checkpoint 'cp-3BPtEyNBRkQzdXl/ZioB69Xu4TM=' as rev='0-457'
I/CouchbaseLite/REPLICATOR: {Repl#4} Saved local checkpoint 'cp-3BPtEyNBRkQzdXl/ZioB69Xu4TM=': {"time":1618821276,"local":2407,"remote":985137}
D/CouchbaseLite/REPLICATOR: C4Replicator.documentEndedCallback @7437d47200, pushing: false
D/DocumentReplicationLoggerListener: 1 documents mutation received
D/DocumentReplicationLoggerListener: Document created or modified notification received: 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471

The line documentEnded 9c99d8bf-5f5e-4c56-96d8-b8c40b3e7471 2-3a9d647e30650d2f1c684ee993eefb9e flags=00 (0/0) means that we do not lost access to the document, because there is no flag AccessRemoved ( because of flags=00(0/0)), isn’t it ? :thinking:

So the document is still in the local database.

Does it have a link with the method define in import_filter ?

Hello :wave:

We still have the problem, so I made more search.

  1. I create new document with a channel thanks to a POST request :
{
    "id": "659e2164e1ad117a7e01a709a638f8ac",
    "ok": true,
    "rev": "1-3306b118cf1c107afd18c2491a527c13"
}
  1. I check that the document is synchronized and pulled by the application (in the logs) :
2021-04-29 15:09:54.565 20516-20636/fr.o2.visite.app.vistest D/DatabaseManager$pullReplicationFilter: Document 659e2164e1ad117a7e01a709a638f8ac allowed to be pulled by filters
2021-04-29 15:09:54.733 20516-20641/fr.o2.visite.app.vistest D/DocumentReplicationLoggerListener: Document created or modified notification received: 659e2164e1ad117a7e01a709a638f8acac
  1. I modify the document thanks to a PUT request, and replace the previous channel by a new channel "old" for example :
  • Request body :
{
  "channels": ["old"]
}
  • Response body :
{
    "id": "659e2164e1ad117a7e01a709a638f8ac",
    "ok": true,
    "rev": "2-13dd9f5ce30885cda72a35ba14a597f6"
}
  1. I check that the document is synchronized and pulled by the application (in the logs). But I didn’t see a document with flag AccessRemoved :
2021-04-29 15:15:38.023 20516-20634/fr.o2.visite.app.vistest D/DatabaseManager$pullReplicationFilter: Document 659e2164e1ad117a7e01a709a638f8ac allowed to be pulled by filters
2021-04-29 15:15:38.189 20516-20641/fr.o2.visite.app.vistest D/DocumentReplicationLoggerListener: Document created or modified notification received: 659e2164e1ad117a7e01a709a638f8ac
  1. I check the document with a GET request :
{
    "channels": [
        "old"
    ],
    "_sync": {
        "rev": "2-13dd9f5ce30885cda72a35ba14a597f6",
        "sequence": 1003241,
        "recent_sequences": [
            1003231,
            1003241
        ],
        "history": {
            "revs": [
                "1-3306b118cf1c107afd18c2491a527c13",
                "2-13dd9f5ce30885cda72a35ba14a597f6"
            ],
            "parents": [
                -1,
                0
            ],
            "channels": [
                [
                    "previous-channel"
                ],
                [
                    "old"
                ]
            ]
        },
        "channels": {
            "previous-channel": {
                "seq": 1003241,
                "rev": "2-13dd9f5ce30885cda72a35ba14a597f6"
            },
            "old": null
        },
        "cas": "0x000057e683567a16",
        "value_crc32c": "0xf67573b7",
        "time_saved": "2021-04-29T15:15:40.497978839+02:00"
    }
}

Here the code our logger :


class DocumentReplicationLoggerListener : DocumentReplicationListener {

    override fun replication(replication: DocumentReplication) {

        if (replication.isPush) {
            Timber.d("${replication.documents.size} documents mutation pushed")
            replication.documents
                .forEach { Timber.d("Document mutation pushed: ${it.id}") }
        } else {
            Timber.d("${replication.documents.size} documents mutation received")

            replication.documents
                .forEach {
                    if (it.flags().contains(DocumentFlag.DocumentFlagsAccessRemoved)) {
                        Timber.d("Document access removed notification received: ${it.id}")
                    }
                    if (it.flags().contains(DocumentFlag.DocumentFlagsDeleted)) {
                        Timber.d("Document deleted notification received: ${it.id}")
                    }
                    if (it.flags().isEmpty()) {
                        Timber.d("Document created or modified notification received: ${it.id}")
                    }
                }
        }
    }
}

After that, I tried to add some changes to the document (example add a new property). The application did not received the change because the user did not subscribe to the channel "old".

I decided to purge the document thanks the request. The SG removed the document, but the document is still in the application, on the device, because I didn’t have any logs about deletion.

I check the channel that the document has thanks to this request : {rest-api-admin-pfx}#/database/get __db___all_docs[/{db}/_all_docs. Here the response :

...
         {
            "key": "659e2164e1ad117a7e01a709a638f8ac",
            "id": "659e2164e1ad117a7e01a709a638f8ac",
            "value": {
                "rev": "2-13dd9f5ce30885cda72a35ba14a597f6",
                "channels": [
                    "old"
                ]
            }
        },
...

Assumption : The Sync Gateway does not delete the document once the channel is emptied or modified. The channel keeps a reference to the document thanks to the last revision which has access to the channel.

"channels": {
  "previous-channel": {
     "seq": 1003241,
     "rev": "2-13dd9f5ce30885cda72a35ba14a597f6"
   },
     "old": null
},

For example, the document is accessible to user who subscribe to the channel "previous-channel". The document version will be the one at revision 2-13dd9f5ce30885cda72a35ba14a597f6. User that subscribe to channel "old" have access to the last version of the document (null).

Do I understand properly how the SG is working ? Is there a way to remove the reference to the document in the channel ?

NB : I also tried to modify properties max_length and min_length from channel_cache to 1 and 0 but nothing changed.

Based on what you indicated, the doc should have been auto purged on the client if it were removed from all channels that user had access to. So its interesting that its not happening.

I noticed that you are using default sync function. In a prod app, you should define your own sync function so you have full control over access grants. In your case, since you are using “channels” property, the default sync function would automatically route the document to channel with Id corresponding to “channels” property. So it should have worked in your case even with default sync function.

As a next step in troubleshooting, I would recommend that you implement a custom sync function. Use console.log() to log to output. That way you can track the documents as they are processed by sync function. You can log for instance the channels property associated with document so you know the channel it is routed to and so on…

Purges don’t get replicated. You must delete the document (even that won’t purge document in CBL app but client will at least hav the opportunity to react. Again, does not solve your problem but a FYI.

Yeah- that won’t matter. That controls the size of channel cache for optimizing read performance (pull replication)- so SGW hits cache for channel info instead of querying the bucket…

Thanks for your help.

I made a new sync function and apply the same steps (create a document, update the channel, …).

The sync function :

"sync": `function (doc, oldDoc) {
    console.log("doc : " + doc._id + " " + doc.channels);
    console.log("doc : " + doc._id + " " + doc._rev);
    console.log("doc : " + doc._id + " " + doc._deleted);
    if (oldDoc !== null) {
         console.log("oldDoc : " + oldDoc._id + " " + oldDoc.channels);
         console.log("oldDoc : " + oldDoc._id + " " + oldDoc._rev);
         console.log("oldDoc : " + oldDoc._id + " " + oldDoc._deleted);
    }
    channel(doc.channels);
 }`

And I check the log of the SG with tail -f -n 500 *.log | grep a5b3fcac15d70537b2268a5e9b2170bc where a5b3fcac15d70537b2268a5e9b2170bc is the new document.

2021-04-30T17:32:42[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc user@mail.fr
2021-04-30T17:32:42[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc 1-3306b118cf1c107afd18c2491a527c13
2021-04-30T17:32:42[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc false
2021-04-30T17:32:42[INF] CRUD: 	Doc "a5b3fcac15d70537b2268a5e9b2170bc" / "1-3306b118cf1c107afd18c2491a527c13" in channels "{user@mail.fr}"
2021-04-30T17:33:59[INF] HTTP:  #003: PUT /vistest-visite/a5b3fcac15d70537b2268a5e9b2170bc?rev=1-3306b118cf1c107afd18c2491a527c13 (as ADMIN)
2021-04-30T17:33:59[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc old
2021-04-30T17:33:59[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc 2-13dd9f5ce30885cda72a35ba14a597f6
2021-04-30T17:33:59[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc false
2021-04-30T17:33:59[INF] Javascript: Sync oldDoc : a5b3fcac15d70537b2268a5e9b2170bc user@mail.fr
2021-04-30T17:33:59[INF] Javascript: Sync oldDoc : a5b3fcac15d70537b2268a5e9b2170bc undefined
2021-04-30T17:33:59[INF] Javascript: Sync oldDoc : a5b3fcac15d70537b2268a5e9b2170bc undefined
2021-04-30T17:33:59[INF] CRUD: 	Doc "a5b3fcac15d70537b2268a5e9b2170bc" / "2-13dd9f5ce30885cda72a35ba14a597f6" in channels "{old}"

I didn’t read a log about cleaning previous channel from the document a5b3fcac15d70537b2268a5e9b2170bc.Do you know what the CRUD line is about ? :thinking:

I checked log with the revision of the document tail -f -n 500 *.log | grep1-3306b118cf1c107afd18c2491a527c13 :

2021-04-30T17:32:42[INF] Javascript: Sync doc : a5b3fcac15d70537b2268a5e9b2170bc 1-3306b118cf1c107afd18c2491a527c13
2021-04-30T17:32:42[INF] CRUD: 	Doc "a5b3fcac15d70537b2268a5e9b2170bc" / "1-3306b118cf1c107afd18c2491a527c13" in channels "{user@mail.fr}"
2021-04-30T17:33:59[INF] HTTP:  #003: PUT /vistest-visite/a5b3fcac15d70537b2268a5e9b2170bc?rev=1-3306b118cf1c107afd18c2491a527c13 (as ADMIN)
2021-04-30T17:30:51[WRN] c:#003 Sync fn exception: TypeError: Cannot access member '_id' of null; doc = map[_deleted:false _id:b3bb7c19ee8d36a3eb65797d2552250b _rev:1-3306b118cf1c107afd18c2491a527c13 channels:[user@mail.fr]] -- db.(*Database).getChannelsAndAccess() at crud.go:1636

What does the last line mean ? It seems the SG can not delete the document from the channels. What do you think ?

What was the PUT command that you issued?

I did the following PUT request : /db-name/a5b3fcac15d70537b2268a5e9b2170bc?rev=1-3306b118cf1c107afd18c2491a527c13

Where a5b3fcac15d70537b2268a5e9b2170bc is the document ID, and 1-3306b118cf1c107afd18c2491a527c13 is the first revision of the document with channels : [user@mail.fr].

The body of the request was :

{
  "channels": ["old"]
}

This is the line where the code throws an exception : crud.go in version 2.7.1.

I looked at the code of the SG in version 2.8.2 : the code seems to work differently with channels. We are going to discuss in team to update the current version of the SG to the last version. I keep your informed if the upgrade solves our problem :slightly_smiling_face:

We went from 2.7.1 to 2.8.2 : we didn’t solve the problem by upgrading the SG.

When I’ll have some time, I’ll set up a new clean environment and I’ll try same steps.