CBlite replication - Channels that are not part of cblite replication are syncing

Problem: We do have two sets of channels 1. Primary Channels(All configuration documents) 2. Secondary Channels (Documents that contain attachments)

After user login we are starting the cblite replication with only primary channels. When the primary channels sync is in Idle state, we are stopping the cblite replication and updating to secondary channels and restarting the replication. When we restarted the cblite replication with new channels (both with primary and secondary) we are getting documents that are not part of cblite replication channels.

Cblite Version: 2.8.4

Initialization of Replication:

func setupSyncWith(syncUrl: String, syncBucketName:String ,authenticator : DEAuthenticator?) {

    if Defaults[.isIntialSyncCompleted] {
        Defaults[.channels] = Defaults[.primaryChannels] + Defaults[.secondaryChannels]
    }else {
        Defaults[.channels] = Defaults[.primaryChannels]
    }
    
    DEDBHelper.shared.initWithUrl(syncUrl: syncUrl + syncBucketName, authenticator: authenticator, Channels: Defaults[.channels])
    
    }

//// Initialize Replicator
public func initWithUrl(syncUrl:String, authenticator: DEAuthenticator?, Channels channels:Array?) {

    self.syncUrl = syncUrl

    self.dbName = NSString(string: self.syncUrl).lastPathComponent
    
    if channels != nil {
        self.channels = channels!
    }

    if self.database == nil {
        self.database = self.database(name: self.dbName)
    }
    
    if authenticator != nil {
        self.authenticator = authenticator
        self.startSync()
    }
}

///Start Replicator
internal func startSync() {

    if database == nil {
        return
    }

    //startLogging()
    print("\(Date().description(with: Locale.current)) Start Logging")
    let targetEndpoint = URLEndpoint(url: URL(string: self.syncUrl.replacingOccurrences(of: "http", with: "ws"))!)
    replicatorConfig = ReplicatorConfiguration(database: self.database, target: targetEndpoint)
    replicatorConfig.replicatorType = .pushAndPull
    
    if channels.count > 0 {
        replicatorConfig.channels = self.channels
    }
    
    replicatorConfig.continuous = true
    
    // Add authentication.
    if let authenticator = getAuthenticator() {
        replicatorConfig.authenticator = authenticator
    }
    
    // Create a replicator.
    self.replicator = Replicator(config: replicatorConfig)
    
    // Listen to replicator change events.
    self.listenerToken = self.replicator.addChangeListener { (change) in

        print("\(Date().description(with: Locale.current))\(change.status.progress)")
        //Checking whether app registered for callbacks or not
        if self.delegate != nil {
            
            var syncChange = DESyncChange()
            syncChange.status = self.getSyncStatus(change: change)
            syncChange.error = self.getSyncError(change: change)
            
            self.delegate.syncListener(status : syncChange)
        }
    }
    
    self.documentListenerToken = self.replicator.addDocumentReplicationListener { (replication) in
        print("Replication type :: \(replication.isPush ? "Push" : "Pull")")
        for document in replication.documents {
            if (document.error == nil) {
                print("Doc ID :: \(document.id)")
            } else {
                if self.delegate != nil {
                    self.delegate.documentSyncError(error: document.error, documentId: document.id)
                }
            }
        }
    }

    
    // Start replication.
    self.replicator.start()
}

///
func getAuthenticator() → Authenticator? {

    if (self.authenticator == nil) {
        return nil
    }
    
    if let sessionAuth = self.authenticator as? DESessionAuthenticator {
        
        if sessionAuth.cookie.isEmpty {
            return SessionAuthenticator(sessionID: sessionAuth.token)
        }
        return SessionAuthenticator(sessionID: sessionAuth.token, cookieName: sessionAuth.cookie)

    }
    return nil
}

//Stop Sync
///To stop sync
internal func stopSync() {

    if replicator != nil {
        if self.replicator.status.activity != .busy {
            print("\(Date().description(with: Locale.current)) Stop Replicator")
            if let token = documentListenerToken {
            self.replicator.removeChangeListener(withToken: documentListenerToken)
            }
            self.replicator.stop()
            print("\(Date().description(with: Locale.current)) Replication status after replication stop executed: \(self.replicator.status.activity)")
        }else {
                             print("\(Date().description(with: Locale.current)) Retry To Stop Replicator: \(self.replicator.status.activity)")
                self.stopSync()
        }
    }
}

Logs:
cbllog.zip (4.5 MB)

There is not much to comment on here since we can only take your word that things “that are not part of cblite replication channels” are being replicated. Do you have a specific example (include document body and the logs which show it being replicated) that you can show? Can we see the sync function on Sync Gateway?

@borrrden The following post has the sample documents.

Before updating the replicator with additional channels these are not syncing but when the replicator was updated to include the additional channels the following docs are syncing.

I don’t see a document named “certrax” in the logs. In order to help, we need specific information:

  1. What is the document ID and body (including all revisions) of a document that is not being replicated as expected?
  2. Why do you expect it to behave the way that it does?
  3. How does it behave instead?

@borrrden

  1. What is the document ID and body (including all revisions) of a document that is not being replicated as expected?
    DocID: ef15b650b8b8dd72491cf2da9b6ddfd5
    Document Body:

{
“_sync”: {
“rev”: “2-1c82d431ceaa1b95f0cfb083f3e71597”,
“sequence”: 406420,
“recent_sequences”: [
406419,
406420
],
“history”: {
“revs”: [
“1-f9fc4a537c66fa41a899ef758a021118”,
“2-1c82d431ceaa1b95f0cfb083f3e71597”
],
“parents”: [
-1,
0
],
“channels”: [
[
“sync1”,
“sync23”
],
[
“sync_remove”
]
]
},
“channels”: {
“sync1”: {
“seq”: 406420,
“rev”: “2-1c82d431ceaa1b95f0cfb083f3e71597”
},
“sync23”: {
“seq”: 406420,
“rev”: “2-1c82d431ceaa1b95f0cfb083f3e71597”
},
“sync_remove”: null
},
“cas”: “”,
“time_saved”: “2022-06-10T08:55:54.584117658Z”
},
“date”: “2022-06-10”,
“removeChannel”: true,
“type”: “_fh_dummy_data_channel_doc”
}

  1. Why do you expect it to behave the way that it does?
    The above document has the latest channel set to “sync_remove”. Previously, the channels sync23 and sync1 are the active channels for the same document. When we no longer needed access to the particular document that met conditions, using the SyncFunction we are setting it to sync_remove. The expected behavior is to have documents sync_remove never sync to the mobile.

  2. How does it behave instead?
    In Mobile we are using following channels for cblite replication
    channels = {
    primary = (
    default,
    AUTHENTICATED,
    sync23,
    ghapps1594403452248,
    “ghapps1594403452248_webAdmin”,
    2c5f42f823262bca754f08d8b63f4ac6
    );
    secondary = (
    “ghapps1594403452248_attach”
    );
    }

We are expecting the documents that need to sync to the mobile with above channels not all the documents.

Logs:
cbllog.zip (1.0 MB)

Try as I might I can’t understand what you are trying to explain. Is this correct?

In a database that doesn’t contain this document, you start a pull replication that has a set of channels that does not include sync_remove, and then this document is pulled?

Yes thats correct. We raised a ticket in Enterprise support and more details available with Ticket No: #47095