How to: SG 3.1 with scopes&collection replication

Mobile Sync Gateway
,
1

Hi,

I’m using Couchbase 7.1.3 EE and SG 3.1.0 EE.
Up to the latest news, this set up should allow the replication of named scopes and collections by the Inter-SyncGateway Replication.

Anyway, I didn’t find how to map a scope/collection in the replication or at the database level.
The documentation seems to be the same from 3.0 to 3.1.

Can anyone explain me how to map scopes and collections in the SG?

Many thanks!

2

Here is a sample config.

3

Hi @househippo, many thanks for the fast answer and for the shared example!

I tried to map a scope and a collection, but I’m noticing some limitation.
I’m creating a replication to push docs in the specified source collection by Inter-SyncGateway Replication.

After setting the replication, I see in the logs the following WRN:

2023-05-11T12:56:16.963Z [INF] Replicate: c:sgr-mgr- db:dummy_bucket Initializing replication test31-scope-coll
2023-05-11T12:56:16.963Z [DBG] Replicate+: c:sgr-mgr- db:dummy_bucket couldn’t find existing local checkpoint for ID “_sync:m_dummy_bucket:sgrStatus:sgr2cp:pull:test31-scope-coll”
2023-05-11T12:56:16.964Z [DBG] Replicate+: c:sgr-mgr- db:dummy_bucket couldn’t find existing local checkpoint for ID “_sync:m_dummy_bucket:sgrStatus:sgr2cp:push:test31-scope-coll”
2023-05-11T12:56:16.964Z [INF] Replicate: c:sgr-mgr- db:dummy_bucket Couldn’t load initial replication status for “test31-scope-coll”: Replication status not found - stats will be reset
2023-05-11T12:56:16.964Z [WRN] c:sgr-mgr- db:dummy_bucket Error initializing replication default collection is not configured on this database: %!v(MISSING) – db.(*sgReplicateManager).RefreshReplicationCfg() at sg_replicate_cfg.go:811

The replication is in running state, but I’m not seeing any changes in the remote.
The database is configured as follows:

{“bucket”:“dummy_bucket”,“scopes”:{“scope1”:{“collections”:{“c2”:{},“c1”:{}}}},“guest”:{“disabled”:true},“enable_shared_bucket_access”:true,“import_docs”:true,“num_index_replicas”:0}

The replica is configured as follows:

{“batch_size”: 1000, “continuous”: true, “direction”: “pushAndPull”, “max_backoff_time”: 5, “purge_on_removal”: false, “remote”: “https://user:pass@remotesgendpoint/testdb”, “replication_id”: “test31-scope-coll”}

Any advice?

4

Add collections_enabled to your replication config, along with a list of collections_local you wish to replicate with the remote. You can omit collections_local if you want to replicate all collections.

Note the remote must already be configured with the set of collections being replicated.

Docs: Admin REST API | Couchbase Docs

5

Thanks for the advice.

Just to see if I understood:

"collections_enabled": true,
  "collections_local": [
    "scope1.collection1",
    "scope1.collection3",
    "scope1.collection6"
  ],
  "collections_remote": [
    "scope1.collectionA",
    null,
    "scope1.collectionF"
  ]

This maps “scope1.collection1” → “scope1.collectionA”, “scope1.collection3” → “scope1.collection3” (null seems to mean the same scope/collection on target) and so on?

What if I don’t specify collections_local and collection_remote?
It tries to replicate to the a similar named scope/collection?

6

Yes, you only need to specify collections_remote if your local and remote collections do not match in name. You can omit the property (or single items inside it) if your local and remote scopes/collections match in name.

7

Hi Ben,

I reuse this thread to ask if it’s possible to specify different scopes in the same database or there’s a need to specify multiple databases for different scopes in the same bucket.
If so, can you give me an example?

Many thanks for the help! :wink:

8

You’ll need to create a database per-scope

1 Like