We are developing a mobile app to exchange media files between community members. Since we need to support offline use and then sync when connection (or wifi) is available, we choosed Couchbase mobile for its replications features.
My first approach was to store the media as binary BLOB file on the CBL database, replicate it to the Couchbase server then allow members to get it on their device via pull replication and use it offline on their devices, if needed. Then I read that it is not a good idea to store BLOB on any database, including on this Kirk Kirkconnell’s excellent article.
I’m sensitive to all the exposed arguments especially the cost effectiveness. But, since the SGW replication is a key feature we need to use, do I face an issue storing BLOB on the file system? Is there a robust way to keep replicating BLOB between Couchbase server and client’s devices even if they are stored on the file system?
Does someone know a better way to manage that use case?
The BLOB storage story in Sync Gateway has some serious limitations at the moment – notably the Couchbase Server document limit of 10 or 20 MB (sorry, I don’t remember exactly).
At the moment, if you need to store larger blobs, your best bet is to use a separate BLOB storage service like S3, or roll your own, and include links to the blobs in your documents.
If the document limit is still within your limits and you want to store the blobs directly in Couchbase Mobile, you should definitely considering using CBL 2.0 since the new websocket-based sync protocol has much better multiplexing capabilities, so large attachments will have less of an impact on the overall sync throughput.
If I use a separate BLOB storage in S3 for example, do is there a way to keep syncing BLOB files throught the sync Gateway? I seriously consider that option because of database storage costs and calculations cost on EC2.
I wioll read more about new websocket based sync protocol on CBL 2.0
Is there a viable architecture considering storage and sync files from file system to file system?
Unfortunately not at the moment. It would be possible to extend Sync Gateway to be able to support storing BLOBs in S3 (or similar BLOB stores). It’s been discussed, but hasn’t made it onto the immediate development roadmap yet.