Syncing the gateway with Couchbase Server

Mobile Sync Gateway
1

Hi everyone,

I have a newbie question which I couldn’t find an answer for that.

We have a system that uses both couchbase mobile platform & couchbase server. In order to run syncing without an issue I have learnt that I need to use restful api of sync gateway such as inserting document etc. However, in case of I got an couchbase database whose data is already inserted and there is no metadata, is there a way that I can realize syncing between couchbase server and gateway?

Thanks a lot.

2

Hi
If you are using Sync Gateway 1.4 or earlier and Couchbase Server 4.x, then your understanding is right. There is a concept of bucket shadowing that will allow you to handle the sync w/ mobile clients but that is being deprecated so I wouldn’t recommend taking that approach.
The good news is that with Sync Gateway 1.5 / Couchbase Server 5.0 , you should have two-way sync working seamlessly between the mobile clients and server. Developer builds are available to try out -https://developer.couchbase.com/documentation/mobile/1.5/whatsnew.html.
Is moving to Sync Gateway 1.5 / Couchbase Server 5.X an option for you ?

1 Like
3

good news,

documents can be inserted on the Server directly (SDKs) or through the Sync Gateway REST API.

is that means that I can update documents on the Server directly by N1QL (update/migrate/delete/insert),and SG admin REST API will be called by CB Server automatically?

4

Hi - Yes, with the latest release (now Dev), you can run N1QL queries directly against your CB and manipulate documents and it will be synced over to the mobile clients via the sync gateway.
Under the hood, the server does not call REST API on sync gateway. At a high level, there is sophisticated high performance DCP protocol between the CB Server and Sync Gateway that handles the sync. Also some core architectural changes to the way the sync metadata is stored - its moved away from the _sync field in document to XATTRs…

5

Thank you for the explanation .
where is [quote=“priya.rajagopal, post:4, topic:13037”]
XATTRs
[/quote] store?

6

Thanks Priya. We will think about that but this is a exciting news.

7

The xattrs are metadata associated with documents stored on Couchbase server .

8

Hi Priya,
I have Couchbase server 5.0 and SG15 running. When I insert a document into the CB with the .NET SDK I get following warning in SQ Log:

12:14:46.835+02:00 WARNING: changeCache: Doc "b21a71ff-676c-45a8-8390-f7b26221609c" does not have valid sync data. -- db.(*changeCache).DocChanged.func1() at change_cache.go:385

What did I do wrong?

9

“SQ log” ?

Can you indicate the exact builds that you are using for SG and .NET ? (Please refer to version compatibility table Couchbase Capella for Mobile Developers)

10

Hi Priya,

My apologies:

SG 1.5 (477)
Couchbase Server 5.0.0-3217
Couchbase .Net client 2.4.8

UPDATE:
I meant SG log instead of SQ. Again, my apologies

11

No worries (I assumed as such but wanted to be sure :-)) . Can you share your SG config file ?

12

Left one database because the other are all alike.

{
	"log": ["*"],
	"interface": ":4984",
	"adminInterface": ":4985",

	"databases": {
		"user": {
			"server": "http://127.0.0.1:8091",
			"username": "xxxxxxx",
			"password": "xxxxxxx",
			"bucket": "user",
			"users": {
				"GUEST": {
					"disabled": false, "admin_channels": ["*"]
				}
				

			},
			"unsupported": {
				"replicator_2": true
			}

		}

		
		

	}
}
13

Hey Can you please add this flag for XAttr support

"unsupported": {
        "enable_extended_attributes": true,
        "replicator_2":true
      }
14

Hi,

I am using syncgatway 2.0, and couchbase server 5.0. I have loads of existing documents in the system as per the above discussion i assumed there would be a two way sync between sync_gateway and couchbase bucket. But Any document created or updated does not reflect in the sync_gatway.

Please find the config in my sync_gateway below.
{
“log”: ["*"],
“interface”: “:4984”,
“adminInterface”: “:4985”,

"databases": {
	"####": {
		"server": "http://localhost:8091",
		"username": "###",
		"password": "###",
		"bucket": "###",
		"users": {
			"GUEST": {
				"disabled": false,
				"admin_channels": ["*"]
			}


		},
		"unsupported": {
			"enable_extended_attributes": true,
			"replicator_2": true
		}

	}
}

}
please let me know if we should make any config changes in couchbase server, else two way sync between couchbase server and sync gateway no longer supported.

Regards
Ashok Sati

15

You are using obsolete config parameters in the “unsupported” section - those are no longer applicable in GA version .

You should use the import_docs and enable_shared_bucket_access …

{
 "interface":":4984",
 "log": ["*"],
 "databases": {
   "my_db": {
   "import_docs": "continuous",
   "bucket":"my_db",
   "server": "http://localhost:8091",
   "enable_shared_bucket_access":true
  }
 }
}

Check out the blog which discusses the same.

16

That helped thank you.

17

Heh sorry, the above config works fine when we create a new document after sync docs are created the documents which were present before the config does not get synced.

Is there a way to achieve the sync documents created before the sync was setup.

18

Do a resync of the database.

Although if you restarted the SGW with the new config it should import pre-existing docs in server and add metadata to it…but try a resync as suggested

And before you do the resync, one more thing to double check is the bucket access credentials with which SGW is accessing the bucket. It needs full bucket access and read only admin access…

19

Hi Priya,

After implementation of two way sync. I took the backup of the bucket with cbbackup which is working great. But when we use cbrestore to import the bucket with documents it leads to issue usually conn.sen() failure it is happening only on the buckets with two way sync.

What is the best approach to take backup for the same. One would always prefer to have a file backup don’t they.

Regards
Ashok Kumar Sati