What are some tips for achieving maximum insert performance with Cb Lite on Android?
At current, I’m trying to load about 50K documents and can only load about 10-20 documents per second using a Galaxy Tab 4.
I’m also creating my own id. Is is more performant to do that or let cblite create the id and use a view instead?
I’ve tried using transactions, and that works well using the .NET driver. But I"m getting out of memory exception on Android with this many documents.
Is it possible and supported to bulk load directly to the sqlite file maybe?
Update: I tried bulk insert with letting cblite create the id and am getting speeds of 39 docs/second. That seems to be the better route so far.
Update2: I should clarify that not using my own id and wrapping the insert process in a transaction was providing a rate of 39 docs/sec. But it crashed after about 15K documents. Without a transaction I’m getting aobut 23 docs/sec. Perhaps, wrapping batches of documents in a transaction will get a little more peformance.
I assume you are using CBL .NET. I am not .NET developer, but I work for CBL Android/Java. Following is my tips for Android.
You already uses. Transaction makes database operation faster.
Find out best batch size for each Transaction. By using Transaction, data is stored in memory till commit. In case app uses a lot of memory, Android system tries to release releasable memory, it makes app/system really slow.
Same reason with previous one, release unused object/memory as soon as possible.
Yeah. That’s pretty much what I’m figuring. Need to find a magic number for the batch size and wrap each in a transaction.
Even still, the next challenge is indexing… This devices takes close to 5 minutes to apply an index against 42K documents. And the index is doing nothing more than emitting two properties if they exist (not null). Not sure what more I can do to improve that.
Refer to the inBatch API. Also assuming you are on a version that is much more recent than 2.0
public void inBatch(@NonNull [Runnable] runnable) throws [CouchbaseLiteException])
Runs a group of database operations in a batch. Use this when performing bulk write operations like multiple inserts/updates; it saves the overhead of multiple database commits, greatly improving performance.
`runnable` - the action which is implementation of Runnable interface
`CouchbaseLiteException` - Throws an exception if any error occurs during the operation.