Routing documents entered using SDK (CB 5.0 & SG 1.5)

Mobile
1

As I understood from documentation, since versions Couchbase server 5.0 and Sync Gateway 1.5 you can create/update documents using SDK (directly in DB) and they will be available for syncing with mobile devices as is. My question is how document routing is done in this case? Does Sync Gateway implicitly call sync function on all of those documents created/updated using SDK or is there some other way to put document in the right channel/channels?

2

You are still responsible for implementing the sync function that routes documents to the right channels for consumption by mobile clients . But you can specify a import_filter function that determines what SDK uploaded documents to import / sync.

You will have to specify a couple of new configuration options in Sync Gateway Config file
– enable_shared_bucket_access to true: This enables Sync Gateway to use XAttrs
– import_docs to continuous: This causes the Sync Gateway to automatically import documents added via the SDK

Here is a sample Sync Gateway config file for reference.

Deployment Note: In a non-accel SG deployment, only a single SG node should have import_docs=continuous to avoid duplicate processing of documents

3

So if I understood correctly, sync function will do the routing for SDK inserted documents. And in import filter function we determine which documents will be available for syncing and go through sync function, etc.

4

Yes, That is correct

5

Thanks a lot Priya. Now I am having trouble when deleting documents via SDK, because they remain on local CB Lites. I guess that problem is because tombstones were not created using bucket.remove(…) method, just documents where deleted from server. Is something missing from Sync Gateway configuration file or there is some other SDK method similar to Sync Gateway Delete request?

Config:
“import_docs”: “continuous”,
“enable_shared_bucket_access”: true,
“import_filter”: function(doc) { return true; },
…

6

Deletes , even if done directly on the server will get imported and synced over to the mobile clients.

  • What does your sync function look like ?
  • What do the sync gateway logs show ? (Make sure you enable logging). Are there any import errors reported.
  • How are you testing to see if the change made it’s way to the client ? Do you have a database change listener ?
7

Thanks Priya! In my case I get following error in Sync Gateway logs:

Create doc:

2017-12-18T11:48:12.460Z Import+: Attempting to import doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f”…
2017-12-18T11:48:12.461Z CRUD+: Doc BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f is not an SG write, based on cas. cas:15016114f7ca0000 syncCas:""
2017-12-18T11:48:12.461Z Import: Created new rev ID 1-ff3911914f8ece88df969db8160aba54
2017-12-18T11:48:12.461Z CRUD+: Invoking sync on doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” rev 1-ff3911914f8ece88df969db8160aba54
2017-12-18T11:48:12.469Z CRUD: Doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” in channels "{OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40, OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40-COMMON}"
2017-12-18T11:48:12.469Z CRUD+: Saving doc (seq: #90, id: BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f rev: 1-ff3911914f8ece88df969db8160aba54)
2017-12-18T11:48:12.470Z CRUD: Stored doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” / "1-ff3911914f8ece88df969db8160aba54"
2017-12-18T11:48:12.470Z Import+: Imported BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f (delete=false) as rev 1-ff3911914f8ece88df969db8160aba54
2017-12-18T11:48:12.470Z Cache: Received #90 after 1ms (“BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” / “1-ff3911914f8ece88df969db8160aba54”)
2017-12-18T11:48:12.470Z Cache: Initialized cache for channel “OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40” with options: &{ChannelCacheMinLength:50 ChannelCacheMaxLength:500 ChannelCacheAge:1m0s}
2017-12-18T11:48:12.470Z Cache: #90 ==> channel "OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40"
2017-12-18T11:48:12.470Z Cache: #90 ==> channel "OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40-COMMON"
2017-12-18T11:48:12.470Z Cache: #90 ==> channel ““
2017-12-18T11:48:12.470Z Changes+: Notifying that “posdev” changed (keys=”{, OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40, OWNER::a928c352-4bc5-43eb-98bb-27caa5ad1a40-COMMON}”) count=5

Delete doc:

2017-12-18T11:48:58.568Z Import+: Attempting to import doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f”…
2017-12-18T11:48:58.568Z CRUD+: Doc BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f is not an SG write, based on cas. cas:1501611fb4140000 syncCas:"0x000084f814610115"
2017-12-18T11:48:58.569Z Import: Created new rev ID 2-c9d09e11c060b3736b1a3e7d2ad1f242
2017-12-18T11:48:58.569Z CRUD+: Invoking sync on doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” rev 2-c9d09e11c060b3736b1a3e7d2ad1f242
2017-12-18T11:48:58.569Z CRUD+: No old revision “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” / “1-ff3911914f8ece88df969db8160aba54”
*2017-12-18T11:48:58.570Z WARNING: Sync fn exception: TypeError; doc = map[_deleted:%!s(bool=true) _rev:2-c9d09e11c060b3736b1a3e7d2ad1f242 _id:BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f] – db.(Database).getChannelsAndAccess() at crud.go:1199
2017-12-18T11:48:58.570Z CRUD+: Did not update document “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” w/ xattr: 500 Exception in JS sync function
2017-12-18T11:48:58.571Z Import: Error importing doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f”: 500 Exception in JS sync function
2017-12-18T11:48:58.571Z WARNING: Unable to import doc “BRAND::383859bc-9889-499f-b375-a7d7e3eabb7f” - external update will not be accessible via Sync Gateway. Reason: 500 Exception in JS sync function – db.(*changeCache).DocChangedSynchronous() at change_cache.go:394

So somehow SG can’t find document’s old revision => oldDoc is null which creates error in my sync function. Because I expect that oldDoc exists in case of deleting document => doc._deleted = true.

How come Sync Gateway can’t find document’s old version in this case, when delete was performed using SDK?

8

So when a doc is deleted via the SDK, the system no longer stores copy of the body. So oldDoc will be nil. This when it imports, the oldDoc which refers to the body of the parent revision (typically) will be nil

So in your sync function , do something like this
if (isRemoved()) { return; }

where the isRemoved() function is defined as follows -
// This is when document is removed via SDK or directly on server function isRemoved() { return( isDelete() && oldDoc == null); }

1 Like