ForestDB, iOS, Background

Mobile Couchbase Lite
1

Hey,

Just looking for guidance on background usage (notably, with iOS Data Protection) on Couchbase Lite and ForestDB. An app I’m working with has just switched over to ForestDB in development and it seems like there’s very few things we can do once protected data is no longer available – even queries seem to trigger errors.

e.g.

[FDB ERR] Error in OPEN on a database file '/var/mobile/Containers/Data/Application/536281B4-4C4A-4C8A-8C37-8A055E99ECEA/Library/Application Support/CouchbaseLite/cbase.cblite2/Device.viewindex.0', errno = 1: 'Operation not permitted'
2016-08-29 14:16:21.706 Org APP[9923:3497076] WARNING: ForestDB error: A file operation is not permitted (-50)
{at FDBLogCallback:68}

* thread #1: tid = 0x355c74, 0x000000018275411c libsystem_kernel.dylib`__pthread_kill + 8, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
frame #0: 0x000000018275411c libsystem_kernel.dylib`__pthread_kill + 8
frame #1: 0x0000000182820ef8 libsystem_pthread.dylib`pthread_kill + 112
frame #2: 0x00000001826c5dc8 libsystem_c.dylib`abort + 140
frame #3: 0x00000001821f93f4 libc++abi.dylib`abort_message + 132
frame #4: 0x0000000182215e80 libc++abi.dylib`default_terminate_handler() + 280
frame #5: 0x0000000182220264 libobjc.A.dylib`_objc_terminate() + 152
frame #6: 0x0000000182220264 libobjc.A.dylib`_objc_terminate() + 152
frame #7: 0x0000000182212f44 libc++abi.dylib`std::__terminate(void (*)()) + 16
frame #8: 0x0000000182212fd0 libc++abi.dylib`std::terminate() + 92
frame #9: 0x00000001007658cc OrgCore`__clang_call_terminate + 16
frame #10: 0x0000000100796ee4 OrgCore`c4View::c4View(c4Database*, cbforest::slice, cbforest::slice, fdb_config const&, cbforest::slice) [inlined] c4Internal::RefCounted<c4View>::~RefCounted() + 432 at c4Impl.hh:116
frame #11: 0x0000000100796e7c OrgCore`c4View::c4View(this=0x0000000115f5fe48, sourceDB=<unavailable>, path=<unavailable>, name=<unavailable>, config=<unavailable>, version=<unavailable>) + 328 at c4View.cc:56
frame #12: 0x0000000100795034 OrgCore`::c4view_open(C4Database *, C4Slice, C4Slice, C4Slice, C4DatabaseFlags, const C4EncryptionKey *, C4Error *) [inlined] c4View::c4View(this=<unavailable>, sourceDB=<unavailable>, config=<unavailable>) + 156 at c4View.cc:54
frame #13: 0x0000000100795010 OrgCore`::c4view_open(db=<unavailable>, path=<unavailable>, viewName=<unavailable>, version=<unavailable>, flags=<unavailable>, key=0x0000000000000000, outError=0x0000000000000006) + 120 at c4View.cc:102
frame #14: 0x000000010076c460 OrgCore`-[CBL_ForestDBViewStorage openIndexWithOptions:status:](self=0x0000000125ec83b0, _cmd=<unavailable>, flags=<unavailable>, outStatus=0x0000000000000000) + 368 at CBL_ForestDBViewStorage.mm:181
frame #15: 0x000000010076c248 OrgCore`-[CBL_ForestDBViewStorage lastSequenceIndexed](self=0x0000000125ec83b0, _cmd=<unavailable>) + 32 at CBL_ForestDBViewStorage.mm:132
frame #16: 0x000000010082f3b0 OrgCore`-[CBLDatabase(self=0x0000000125e8b4f0, _cmd=<unavailable>, viewName=<unavailable>, options=<unavailable>, ifChangedSince=-1, outStatus=0x000000016fdd95ac) queryViewNamed:options:ifChangedSince:status:] + 124 at CBLQuery.m:485
frame #17: 0x000000010082dda8 OrgCore`-[CBLQuery run:](self=0x0000000125dc9550, _cmd=<unavailable>, outError=domain: class name = CBLQuery - code: 0) + 152 at CBLQuery.m:219
frame #18: 0x00000001006f351c OrgCore`(view="Device", descending=true, startKey=nil, endKey=nil, limit=0, database=0x0000000125e8b4f0, self=0x0000000125da52d0).query(String, descending : Bool, startKey : AnyObject?, endKey : AnyObject?, limit : UInt, database : CBLDatabase) -> [DatastoreModel]? + 676 at Datastore.swift:189
frame #19: 0x00000001006f2acc OrgCore`Datastore.query(view=DeviceModel, database=nil, self=0x0000000125e85d20) -> [DatastoreModel]? + 360 at Datastore.swift:135
frame #20: 0x000000010014c558 Org APP`HomeAPPGraphViewModel.(self=0x0000000125ede180).getter + 72 at HomeAPPGraphViewModel.swift:125
frame #21: 0x00000001001497c8 Org APP`HomeAPPGraphViewModel.(heartRate=71, todayDailySummary=0x0000000125d88180, self=0x0000000125ede180)(Int?, DailySummaryModel?) -> [String : AnyObject]? + 440 at HomeAPPGraphViewModel.swift:80
frame #22: 0x000000010014a1c0 Org APP`HomeAPPGraphViewModel.(oldValue=70, self=0x0000000125ede180).didset + 236 at HomeAPPGraphViewModel.swift:37
frame #23: 0x000000010014a46c Org APP`HomeAPPGraphViewModel.(newValue=71, self=0x0000000125ede180).setter + 76 at HomeAPPGraphViewModel.swift:0
frame #24: 0x000000010014e694 Org APP`HomeAPPGraphViewModel.(event=(Next = 71)
...

I’m going over some of these uses, I’m sure some of them are unnecessary, but just from a guidance perspective – should I assume CouchbaseLite + ForestDB + Background (protected data not available) means we shouldn’t attempt any CouchbaseLite calls during that time?

2

The difference in behavior is probably because our ForestDB storage uses separate files for view indexes, whereas with SQLite everything is in the same database file. The view indexes are kept open for one minute after a query, then closed, to avoid using up too many file handles. Unfortunately this means you can get errors if you query a view while your app is in the background.

One workaround is to switch to a file protection mode that allows files to be opened while the device is locked. Either that, or don’t perform any view queries while locked.

(This ought to be filed as an issue so we can think about improving the situation in the future. I don’t think we have an issue covering this, so I’ll file one.)

3

I just looked at your backtrace and noticed that it appears to be a crash. That definitely shouldn’t be happening — at worst you just just get an error from the query. I’ll file another bug report :confused:

4

Without knowing what’s in the ForestDB index files (I haven’t looked much at the internals yet), is it possible that the indexes could be written with different data protection flags than the database, leaving them amenable to being opened in the background? In any case, yes, the crash is definitely a bigger deal than a query not working in the background. Thanks for filling it.

5

They could, but I’d be loath to do that by default since it lowers security somewhat. It would also be somewhat difficult to implement because those files get created by ForestDB code, sometimes at unpredictable times (i.e. during compaction that runs on a background thread.)

6

Still, even if compaction and other scenarios that cause ForestDB to create files at unpredictable times, every call in ForestDB should be guarded using the native flag UIApplication.sharedApplication().protectedDataAvailable to check if it is able to perform the action or back off until it can. This check should be run regardless of the storage type.

7

Hm. There’s no hook in ForestDB to enable me to call that when the background compaction starts. I’d need to ask them to add some API for it.