Issue with sync after upgrading to 2.5

I’ve just updated an old test app and the sync. gateway in my dev. environment from 2.1.2 to version 2.5

I was trying to solve an issue where deletions in the Couchbase bucket (through a web app using the Java SDK) did not sync. the deletions to the mobile test app. Therefore, I tried to update the abave (and the Java SDK to 2.7.4 - from 2.7.3)

I now have the issue that a new documents in the Couchbase bucket are NOT synced to the app???

How can I troubleshoot this?

Hi @jda,

Can you confirm you have the following properties in your Sync Gateway config?

"enable_shared_bucket_access": true
"import_docs": true

Hi @bbrks

I have this:

        "databases": {
                "data": {
                "use_views":true,
                "bucket": "data",
                "server": "http://localhost:8091",
                "username": "xxxxx",
                "password": "......",
                "enable_shared_bucket_access": true,
                "import_docs": "continuous",
                "users": { "GUEST": { "disabled": true, "admin_channels": ["!"] } },
                "sync": `function (doc, oldDoc) {
  // Document type is mandatory
  if (oldDoc._deleted){
    // Server doc. deleted -> just propagate this!
    return;
  }
: 
: 

So not exactly… My import_docs is different…

Seems I get an error:

{Couchbase.Lite.CouchbaseLiteException: CouchbaseLiteException (LiteCoreDomain / 26): Panic: interface conversion: interface {} is nil, not string.}

Not sure what this means…?

It’s the output from these lines in my OnReplicationEvent method:

if (arg.Status.Error != null){
   Console.WriteLine($"Error :: {arg.Status.Error}");
}

@jda Are you able to get the logs from Sync Gateway? This is a rather interesting error that I’d like to take a look at ASAP!

… Ok, I think this was just due to setting the loglevel incorrectly :wink:

Hi @jda, are you still able to get the Sync Gateway logs around this particular error?

We’d like to improve the error handling around this, but could do with seeing exactly what is happening in SG when this error occurs.

Well, yes, it is all in my local dev. environment. So what do you want me to give you?

I have the strange situation now that after having deleted the local db, I now only get a small subset of the docs out on the app…

… I have event tried to change the sync formula to:

  if (oldDoc._deleted){
    // Server doc. deleted -> just propagate this!
    return;
  }
return;  // Just allow everything...!!!
: 
: 

… but it seems to make no difference to the number of docs. sync’ed…???

Ok, looking into sg_warn.log I found entries like:

2019-05-01T23:15:09.917+02:00 [WRN] gocb: Config block decode failure (read tcp 192.168.77.9:5818->192.168.77.9:8091: use of closed network connection) -- base.GoCBCoreLogger.Log() at logger_external.go:49
2019-05-01T23:15:19.919+02:00 [WRN] gocb: Config block decode failure (read tcp 192.168.77.9:5880->192.168.77.9:8091: use of closed network connection) -- base.GoCBCoreLogger.Log() at logger_external.go:49
2019-05-01T23:15:29.922+02:00 [WRN] gocb: Config block decode failure (read tcp 192.168.77.9:5942->192.168.77.9:8091: use of closed network connection) -- base.GoCBCoreLogger.Log() at logger_external.go:49
2019-05-01T23:15:39.928+02:00 [WRN] gocb: Config block decode failure (read tcp 192.168.77.9:6008->192.168.77.9:8091: use of closed network connection) -- base.GoCBCoreLogger.Log() at logger_external.go:49
2019-05-01T23:15:49.931+02:00 [WRN] gocb: Config block decode failure (read tcp 192.168.77.9:6070->192.168.77.9:8091: use of closed network connection) -- base.GoCBCoreLogger.Log() at logger_external.go:49

What could that be???

The latest error (“config block decode failure”) looks symptomatic of connection problems between Sync Gateway and Couchbase Server. I haven’t seen that particular error before, but it looks more like a networking issue than anything Sync Gateway is doing.

Hmmmm… Well, the connection is there (and has been working for ages) - it’s on the same box. But perhaps “…decode…” could indicate that some “invalid” character has been put in the config file while editing?

One other thing. In the documentation for 2.5 you show all these options:

"import_docs" : true
"import_docs" : false
"import_docs" : "continuous"

The last one is not explained (but shown in the sample of the import_filter. So what is correct??

I really hope I can get this fixed before this afternoon… as I’m going to do a presentation on Couchbase to a bunch of “colleagues” (other consultants) tonight… :flushed:

It’s always like this with timing of such issues - thank you Murphy!! :zipper_mouth_face:

@adamf and @bbrks I have just reviewed my sync_gateway.json after the above. Still not sure about the “import_docs” setting that does not seem to be consistent in the documentation. But one of the settings that was “left over” from my initial setup was the "use_views":true - which I changed to “false”. Following the documentation related to that I also changed the “num_index_replicas” setting for my test environment (two Couchbase server nodes and one SyncGateway) - but “obviously” I now get this error as I’m using the Community edition:

May 02 11:34:27 sg1 bash[9185]: - will retry. -- db.(*SGIndex).createIfNeeded.func1() at indexes.go:242
May 02 11:34:38 sg1 bash[9185]: 2019-05-02T11:34:38.156+02:00 [WRN] Error creating index sg_tombstones_x1: Error creating index with statement: CREATE INDEX `sg_tombstones_x1` ON `data`(meta().xattrs._sync.tombstoned_at) with {"num_replica":1,"retain_deleted_xattr":true,"defer_build":true}.  Error: [5000] GSI CreateIndex() - cause: Index Replica not supported in non-Enterprise Edition

Question: Should (could) I create this indexes manually on each db server (as I have code doing for the query indexes)? The formula is shown, so I can easily do it :wink: - but not sure if the SyncGateway server uses/benefits from it?

Oh, and it appears that I have to set num_replicas to 0… - or the SG server won’t start :frowning:

For reference this is what my sync_gateway.json file looks like now:

{
	"log": ["HTTP+"],
	"adminInterface": "0.0.0.0:4985",
	"interface": "0.0.0.0:4984",
	"databases": {
		"data": {
			"use_views":false,
			"num_index_replicas":0,
			"bucket": "data",
			"server": "http://db1,db2:8091",
			"username": "xxxxxxx",
			"password": "yyyyyyyyyyyy",
			"enable_shared_bucket_access": true,
			"import_docs": true,
			"users": { "GUEST": { "disabled": true, "admin_channels": ["!"] } },
			"sync": `function (doc, oldDoc) {
  // Document type is mandatory
  if (oldDoc._deleted){
    // Server doc. deleted -> just propagate this!
    return;
  }
  // Document type is mandatory
  if (!doc.type){
    throw({forbidden: "Document type is required."});
  }
  if ((doc.type == 'EnvLake' || doc.type == 'EnvMeasurement' || doc.type == 'Feedback' || doc.type == 'ActivityLog') && oldDoc){
    throw({forbidden: "Document type not allowed to sync to mobile..."});
  }
  // All public docs are available in the app
  if (doc.ispublic) {
    channel('!');
    //  return;
  }
  // All non-club fishing trips and catches are available (for stats)
  if ((doc.type == 'FishingTrip' || doc.type == 'Catch') && doc.clubonlykey == undefined) {
    channel('!');
  }
  // All non-specific user info is available (for stats)
  if (doc.type == 'User') {
    channel('!');
  }
  // Only docs "owned" by user can be updated
  var key;
  if(doc.type == 'User'){
    key = doc.key;
  } else if(doc.type == 'FishingTrip' || doc.type == 'Catch' || doc.type == 'Photo' || doc.type == 'Private'){
    key = doc.userkey;
  }
  if(key){
    requireUser(key);
    channel('channel.' + key);
    access(key,'channel.' + key);
  }

  // Creation of new Feedback docs (perhaps without userkey)?
  // Creation of new docs?
  // Updates to existing docs?
  // Readonly for all non-user specific docs...
}`,
      		"allow_conflicts": false,
      		"revs_limit": 20
        }
    }
}

Edit:
It seems that after I edited the config file in an external editor and uploaded it, stopped and started the sync gateway, took the db offline, ran resync, and put it back online - and then deleted local app and rebuild (and resync’ed data) it seems to be behaving more as expected :slight_smile:

Still have the question about indexes for the sync gateway though… - and deletions not sent to mobile (but asked that as a separate question to avoid cluttering everything up)…

“continuous” is an older configuration option, but is not applicable when using enable_shared_bucket_access, so we suggest you stick with true. In reality, using either true or "continuous" will work, as they map to the same underlying value in the code.

As the error message you got back stated, index replicas are only available in Enterprise Edition of Couchbase Server. If you wish to use indexes in Sync Gateway, you can set num_replicas to zero as you’ve found.

Thanks @bbrks. As I understand it I should use index in sync gateway now…?

I now I cannot use numReplicas on the community edition - but I just created an index_1 on db1-server and index_2 on db2-server from code. That seems to work Ok.

Should I do the same for the Sync Gateway indexes? Otherwise, the system will not be tolerant to any outages although I have two servers in a cluster (kind of spoiling the main idea of the cluster)