We’ve encountered a synchronisation issue in our edge-device application that’s impacting data consistency. Our approach involves aggregating data at the day level, storing it in a single document. This means any transaction within a day is saved to the same document, leading to multiple mutations of the same document within a day.
Our edge app uses Sync Gateway to replicate data to edge devices. However, we’ve received reports from customers about missing data for specific days. Our initial investigation revealed that certain documents are not being synchronised to the Sync Gateway.
A notable observation is that all missing documents lack specific Sync Gateway attributes in their metadata, as seen in the UI. We also tried manually updating these docs which changes the revision number but looks SG rejects these docs because the attributes are never populated for these docs
We seek guidance on the following:
Diagnosing the Issue: How can we identify the root cause of why these specific documents are not syncing with the Sync Gateway?
Forced Synchronisation: Is there a possibility to force-sync these missing documents to the Sync Gateway using the REST API, considering that they differ only in data content but include other required fields?
Any insights or recommendations on resolving this synchronization issue would be greatly appreciated.
To confirm my understanding, this is an issue where documents in Couchbase Server are not being imported by Sync Gateway? And thus not being synced to your edge devices?
Can you clarify which revision number you’re talking about when you observe that changing?
I suspect this is a case of Couchbase Server-side modifications not being imported by Sync Gateway. This could be for a number of reasons but it’s worth checking the most obvious ones first:
Is import_docs enabled on the SG database?
Do you observe any errors in the logs during import? You should be able to search for specific document IDs
Is your Sync Function rejecting documents being written by the import process?
Yes, documents in Couchbase server are not being imported by Sync Gateway and thus not being synced to edge devices.
I’m referring to the revision number that gets updated whenever we mutate a document. This revision number is shown in the metadata of the document.
Yes, import_docs in enabled and works for 99% docs, however, a very few times it doesn’t replicate data to edge(Couchbase lite).
Hence wanted to know if there is a REST endpoint to see
force push items so that they are replicated to Couchbase lite
I think you may be hitting a bug we’ve previously fixed. The result is that a subset of documents aren’t imported until SG restarts and restarts its connections to Couchbase Server.
Unfortunately there are no APIs or even even Sync Gateway logging to see unobserved mutations that haven’t been imported, as SG is never notified about the document update in the first place to know to run import.