Web service as a client or use webhooks

Mobile Sync Gateway
1

Hi Team,

I need an architectural help.

We have 2 repositories with one cluster for each. One is where we will keep app related data. It will be having data changed by apps. And second repository is for other business requirements. Both repositories will have different granularity. The app repository will generally have documents broken into different parts so that less amount of data will be synced. We will be keeping static and volatile data in different documents and technologies as diffgrams are not implemented yet and complete documents are synced. Also App repository will not have complete data, it will have data only required by the apps.
But we need to keep both repository in sync.
To do this I think there are 2 options which I can think of right now

  1. Using web hooks :- We will create a web service on which Sync gateway will post documents using web hooks and this service will apply business logic to merge multi documents coming from app repository into one and post it on sync gateway of other repository. But what if the services are down, Then the queue can fill out quickly and the changes will not sync. Or if there is an error on the service and we are setting very less time for timeout in web hooks. In this case we will need to write a fallback method. What do you suggest as the best way to write fallback. Which will make sure all the data is processed and nothing is missing even if some documents missed with web hook method.One method is to create a daemon service which will ask sync gateway about the data changed from last sync periodically. But I need to know if there are better methods then this.

  2. creating the service as a client by embedding CBL in the service. This will work like a mobile device. And when the service will be saving document in its local DB we will set TTL of the document very short so that the service local DB wont have much documents as in reality it doesnt need. But then I am skeptical about the performance of this approach. Theoretically the sync will always happen. But will the CBL be able to cater the load. Will speed be an issue. Please suggest. I also want to know at what number of documents CBL will start having issue.

Please suggest what can be done in this scenario.

Regards
Pankaj Sharma