Couchbase Server Community Edition 6.5.1 build 6299
Sync gateway 2.7.3
Couchbase Lite Android 2.8.1
We are experiencing a problem where deleted documents are reappearing on our Couchbase server.
Here is the process of what is happening:
- Document is created on CBL with a TTL of 15 days and synced to sync gateway.
- Document is updated on CBL and synced to sync gateway.
- Document is deleted from the Couchbase Server bucket with a DELETE N1QL query.
- After the document is deleted from the bucket it gets randomly added again within a few days.
- Only documents that are still on the devices i.e. not older than TTL of 15 days, are added back to the bucket.
We tried increasing the Metadata Purge Interval to more than 15 days but this did not resolve the problem.
Does anybody have any suggestions or possibly know what could be the problem here?
Thanks in advance!
PS: Here is our Sync Gateway config with the sync function:
"log": [
"*"
],
"adminInterface": "0.0.0.0:4985",
"interface": "0.0.0.0:4984",
"databases": {
"prod": {
"server": "http://localhost:8091",
"bucket": "prod_bukcet",
"username": "sync_gateway",
"password": "XXX",
"enable_shared_bucket_access": true,
"import_docs": "continuous",
"use_views": true,
"users": {
"user_X": {
"password": "XXX",
"admin_channels": ["*"],
"disabled": false
}
},
"sync":`
function sync(doc, oldDoc) {
/* sanity check */
// check if document was removed from server or via SDK
// In this case, just return
if (isRemoved()) {
return;
}
//Only sync down documents that are created on the server
if (doc.deviceDoc == true) {
channel("server");
} else {
if (doc.siteId) {
channel(doc.siteId);
} else {
channel("devices");
}
}
// This is when document is removed via SDK or directly on server
function isRemoved() {
return (isDelete() && oldDoc == null);
}
function isDelete() {
return (doc._deleted == true);
}
}`,
}
}
}```
1 Like
Also as responded to in stackoverflow where the same question was posted
In shared bucket access mode (enable_shared_bucket_access:true),a N1QL delete on a document creates a tombstone. Tombstones are always synced. The metadata purge interval setting on the server determines the period after which the tombstone gets purged on the server. So it is typical to set it to a value that matches the maximum partition window of the client- that is to ensure that all disconnected clients have the opportunity to get the deleted document. So setting it to > 15 days just means that the tombstone will be purged after 15 days and so tombstoned documents will be synced down to clients in the meantime.
In your case, if you don’t want documents to be synced down to clients because the lifetime of the document is managed independently on CBL side via the expirationDate(), then purge the document instead of deleting it on server.
1 Like
Thanks for the explanation!
But I think you maybe misunderstood my issue.
We have a scenario where documents are created on CBL. These documents are synced up to the server. The user realizes an error has been made and flags the document as incorrect. On the server the admin can then view all of the flagged documents and delete them from the server. The sync gateway has been setup to only sync up these types of documents, i.e. once an edit has been made to these documents on the server the changes are not synced back down to CBL.
I am unable to immediately purge the document from the device seeing that the document updates need to be synced with the server.
I probably still don’t understand your question
Which user? server side or CBL client user?
What is the purpose of flagging document as incorrect and then have a separate process to delete ? Why not just delete the document if it’s incorrect?
How are you ensuring that? Deletes will always be synced to client. You will have to purge the document on server as explained in earlier post.
Do not understand how this fits into what was described earlier. Which document is this?
We have 2 different types of users in our system
- General users
- Admin users
General users use CBL to create documents on Android devices. If they realise an error has been made they flag the incorrect document on CBL.
These documents are created by an interaction with a general user and a client and contain sensitive information. Thus we do not want to delete/purge the document immediately because the admin user needs to review the flagged document first before permanently deleting it.
We ensure that these documents are only synced up by using the modified sync function as shown in the original question. As can been seen here:
//Only sync down documents that are created on the server
if (doc.deviceDoc == true) {
channel("server");
} else {
if (doc.siteId) {
channel(doc.siteId);
} else {
channel("devices");
}
}
If the document is a “deviceDoc” (it was created on a device) it is put into the “server” channel. All of the CBL devices are set up to only sync the “devices” channel (this includes all of the required config documents).
I hope this clarifies our situation? Please let me know if anything is unclear?
And I am still unsure about purging on the server? Could you please explain how to do this?
Two options as discussed in previously linked doc
- Call N1QL or SDK delete. Creates tombstone. Tombstones are auto magically purged when metadata purge interval expires
- Call SGW purge API to manually purge document