I use couchbase lite android with a continuous push pull replicator.
I have certain documents that are written to multiple times a second and I have noticed this causes huge strain on the replicator and on the server that processes the incoming changes as it receives each change individually. The main reason I write every change to the db is because of the persistence.
I had a few thoughts myself
Have a collection specifically for this document and switch the sync on and off as I need, down side is I forsee a lot of edge cases where the sync isn’t switched on when it is needed which is very difficult to debug as all we will see is the document not in the state we need it.
Place a sync filter on the replicator that looks at a flag on this document that I set when I want the document to sync. Down side is its a lot of work on the replicator as its processing every change coming in multiple times a second.
Cache the document somewhere else and copy it to the database whenever I want to sync it. Same downsides as 1.
I was wondering if anyone has any advice on different strategies / if there is some mechanism built into couchbase to help minimize this strain.
As you might expect, I have much more experience implementing the replicator than I do using it. My first take on this (simply what I’d try first) is solution #1. I guess a lot depends on how fresh the documents have to be and how long you can wait for them to be fresh.
In Android applications I’ve built in the past, it was often sufficient to give users a view of locally cached data and to update it whenever they actually looked at it.
Updating a document that often is not a good idea. It results in very large revision histories, which make updates and replication more expensive.
If you must do it, one fix is to not keep a continuous replicator running all the time. Instead, when you make changes, schedule a one-shot push replicator to run sometime in the near future.
If you still want to receive updates immediately, keep the continuous replicator but make it pull-only.
Sounds like you are storing non-user created document , but some metric or time series data. Can you store all the changes in memory for that second and write it to a document every second or even better a document every minute. That way you can also add some aggregate data like min , max , average of that second or minute already in that document? If its time series data Couchbase has some update features around it. Time Series Data | Couchbase Docs