We’re currently designing an offline-first app but are a bit unsure how to best leverage the great sync features of CBL while not violating the service-oriented principles.
To be more specific, we’re concerned that having the app write directly into the synced DB would bypass any validation / logic implemented in the services managing these entities, unless we duplicate this service logic on the client.
We have been considering 2 options:
1: We could implement a shared service layer, deployed in both client and server. The mobile client would just call this locally as a direct library call, while other consumers that don’t have offline sync would just call the web service - this is doable as client and server are developed in the same language, but still seems like duplicating things - and there’s always a risk of the client and server logics to be on different versions of the service code.
2: Alternatively, we are considering taking the approach of:
Letting read flows access the local mobile db directly however it needs - basically considering that for read flows, the db schema is a committed contract and the read logic is basic enough to be duplicated on client and servers
For write flows, essentially ban direct local writes. All writes go through the server, and we’ll wait for them to be synced back to the device for reading. And to support an offline mode, we’d persist the service call requests locally into a distinct db (not synced, or synced just for backup purposes, but not actually doing any functional transactions into our main entities).
Of course that’s not really a Couchbase question, I’m just wondering if anyone has thoughts based on experience, or best practices to share…
I’m not sure I understand your question, so these answers might be off base.
we’re concerned that having the app write directly into the synced DB would bypass any validation / logic implemented in the services managing these entities
The app’s local database only belongs to the app, so it doesn’t need any special validation. Validation happens when the app pushes changes back to the server. Specifically, the “sync function” you register with Sync Gateway can reject invalid updates.
Note also that since the app is your own code, the only way invalid data can be entered into its database is if your app code writes it there. Assuming the app validates user input, that shouldn’t be a serious problem. Of course you still need the server validation layer in case of application bugs or malicious access to the local database or the replication protocol, but it’s more of a backstop than the first line of defense.
So let’s take a trivial example.
Say my app records, for instance, orders for a particular thing.
Say that, for instance, I only want to allow users to order up to 10 such things at a time.
For online users (e.g. using my website), my service will implement this check server-side for all users.
Mobile offline mobile users however, aren’t going through my service and will record the orders locally in CBL, which will then be synced up when back online. I need to duplicate the logic of that check so that it’s enforced both in the service, AND in the app.
Even worse, if I need to check that I still have inventory for this order, then clearly I cannot do that offline, so I cannot really take the locally-persisted order as a confirmed one, I’ll still need to validate it through an API service call on my server, so it doesn’t really make sense to synchronize this offline order.
Basically, having a local db that synchronizes with the cloud is great, but clearly violates the service-oriented concepts, and I’m trying to figure out an elegant way to handle it… so far, it seems we’ll need to either duplicate the service logic on the client, or give up on syncing altogether and consider the local data as a queue for future service requests.
The short answer to your question is - “it depends on the use case”
Not all use cases are candidates for offline first mode . Order processing for instance isn’t a good use case for offline mode because in order to process an order, you would need online access to payment processing systems, updated inventory etc.
So let’s say you will allow this ordering functionality only when there is network connectivity. In that case, the inventory information in your local database should be expected to be more or less in sync with what’s in the sever. Couchbase does real time sync - it depends on the network. So orders that you make are stored in local CBL database and will be synced up immediately to the remote servers where the validation logic is applied by your backend app servers and order rejected if it cannot be fulfilled. Since it’s connected, then the chance of order getting rejected is low as the local database will likely be in sync. At the end of the day, the server would have to still do the validation in addition to what you do at the client side because you are dealing with a distributed system with number of concurrent clients .
Anytime you expect a client to be disconnected from the backend servers, you will face the issue of data being potentially out of sync or inconsistent with the backend systems - so it’s a question of how tolerant your application is to the inconsistency or if that’s even an issue. For instance, this is a non-issue for a field app and you are updating field reports in offline mode and that list gets eventually synced at a later time to backend . A locally synced database is the only way to guarantee real time responses to data queries and ensure that your app is usable even when disconnected - of course, the results of the query would be based on snapshot of last synced data but that is acceptable in many cases .
Indeed, placing orders really wouldn’t be a typical use case for an offline-first mode… but it is for us.
Our users are field agents recording “orders” for subsidised agricultural products in rural east Africa, so the lack of connectivity is something we have to deal with.
I’m not so concerned about conflict resolution - these can of course happen, and we’ve been used to handling this for 10+ years using Access - I’m a bit more concerned with having to duplicate logic between client and server, as this means we’ll need to redeploy the app pretty frequently… The above example is quite trivial, but we can have much more complex rules deciding who can get what kinds of products…
For sure I was not expecting a magic solution… just curious if anyone had any experience and recommendations dealing with this…
