CBL 2.6.0 Performance: saving 11,000 MutableDocuments takes 20 seconds; what am I doing wrong?

Mobile Couchbase Lite
9

Hi @benjamin_glatzeder, thanks for pitching in to help!

When I said “batch” above, I meant “partitioning” the 11k models into smaller “batches”, iterating over the batches, and then saving each batch of documents inside of a CBL database transaction. Yes, using a CBL transaction is faster for inserts, and we’re already doing that. I was just saying that if you do several units of work consecutively instead of concurrently, there’s very little you can gain from the order in which you structure consecutive units of work.

By way of interest, here are some updated timings which confirm that using a CBL database transaction is 24% - 30% faster:

Saved 10935 documents in batches of 50 to database in 11294ms which is 24% faster than 14910ms by using a transaction
Saved 10935 documents in batches of 200 to database in 10938ms which is 27% faster than 14980ms by using a transaction
Saved 10935 documents in batches of 1000 to database in 10471ms which is 35% faster than 16049ms by using a transaction
Saved 10935 documents in batches of 3000 to database in 10733ms which is 30% faster than 15413ms by using a transaction

Here is a single model with real-world data that has been obfuscated without changing the character count of the JSON; it’s 1375 characters. We’re already explicitly passing the entryGuid value as the first parameter of the MutableDocument constructor:

{
  "entryGuid": "7kwb5269-28a8-4c23-b90c-37c53867dcc7",
  "type": "person",
  "firstName": "Andy",
  "lastName": "Huntsman",
  "displayName": "Huntsman, Andy",
  "organizationName": "Huntsman, Andy",
  "locationGuid": "baf01c9f-18fb-4b60-b709-93051143696a",
  "siteGuid": "2fdba5c0-d6dc-468f-9046-1cab227fab8f",
  "siteName": "Main Site",
  "userStatus": "Regular User",
  "departmentName": "Information Technology > Apps",
  "residenceArea": "Anytown - NY (USA)",
  "workAreaName": "Office 1",
  "contacts": [
    {
      "type": "Work",
      "method": "Phone",
      "contact": "58119"
    },
    {
      "type": "Work",
      "method": "Fax",
      "contact": "58419"
    },
    {
      "type": "Work Alternate",
      "method": "Phone",
      "contact": "51000"
    },
    {
      "type": "Personal",
      "method": "Mobile",
      "contact": "+1 8885551212"
    },
    {
      "type": "Personal",
      "method": "Email",
      "contact": "andrewhuntsman@acme.com"
    },
    {
      "type": "Work",
      "method": "Email",
      "contact": "AJHUNTSMAN@acme.com"
    }
  ],
  "spouseEntryGuid": "2281d49d-3e84-4f5c-be90-00a505c011f2",
  "personPrimaryPhotoGuid": "3510be56-47c6-4546-8954-9116743f2efc",
  "primaryPhotoUrl": "https:\/\/myphotoserver.blob.core.windows.net\/primaryphotos\/8891be56-47c6-4546-8954-9116743f2efc?sv=2018-03-28&sr=b&sig=vJTpHXMYefdVrehAo%2FJiDHNnq60rMlwk3d4QHZQSRmk%3D&se=2019-11-15T23%3A00%3A48Z&sp=r",
  "emergencyContactGuids": [ ],
  "teamName": "Blue Team",
  "lastUpdated": "0001-01-01T00:00:00+00:00",
  "expirationDate": "0001-01-01T00:00:00+00:00",
  "couchbaseType": "Person",
  "couchbaseId": "7kwb5269-28a8-4c23-b90c-37c53867dcc7"
}