mutableDoc.setBlob() not syncing blob - tiny Android project demonstrating the issue on Github

benjamin_glatzeder · March 12, 2021, 4:08pm

The issue could be either in CBL, SG or CB Server.

Setup:
CBL 6.6 CE
SG 2.8
CBL 2.8.1

GitHub: GitHub - benjaminglatzeder/Animallover: Demonstrating blob syncinc issue

App screenshots:

Couchbase Server web console:

Setting the blob for the first time does work:

mutableDocument.setBlob(
                IMAGE_KEY,
                Blob("image/jpeg", byteArray)
            )
            database.save(mutableDocument)

After clicking the buttons multiple times (and slowly! … so no race/thread issue) the image stops updating. In the screenshots the “I love cats” is the latest update to the document. Please also see the revsionIDs and the length of the image file. The revisionIDs match but the lengths do not! Couchbase Server shows the change, too, i.e. the length is correct. But downloading the image via SG Admin REST API shows the dogs instead of the cat:

sudo curl -X GET "http://localhost:4985/database/animallover/blob_%2fimage" -H  "accept: application/json" --output image.jpg

Most basic SG config and sync function:

// ...
"users": {
    "GUEST": {
    "disabled": false,
        "admin_channels": [
			"*"
        ]
    }
},
"sync":`
function (doc, oldDoc) {
    // Your code here
}
`
// ...

Workaround

Removing the key which points to the blob, then saving the document (then it’s synced, too), then add the new blob and save again.
There is the boolean variable removeBlob which can be set to true to enable this workaround in the code.

if (removeBlob) {
    mutableDocument.remove(IMAGE_KEY)
    database.save(mutableDocument)
}
// edit mutableDocument further and save again

Expected

Replacing the blob with another blob, then saving should sync the blob across CBL clients, SG and Couchbase Server.

priya.rajagopal · March 14, 2021, 12:05am

Wasn’t clear - Does this happen the first time you try to replace the blob or after a certain number of updates?

In any case, blob replacement should work. I tested out the userprofile sample app in iOS from our tutorials page and blob updates work as expected . See screen recording below

The postman query

curl -X GET \
  http://localhost:4984/userprofile/user::demo/blob_%2fimageData \
  -H 'Accept: application/jpeg' \
  -H 'Authorization: Basic ZGVtbzpwYXNzd29yZA=='

Blob updates in iOS app are done as follows

  if let imageData = record?.imageData {
            let blob = Blob.init(contentType: "image/jpeg", data: imageData)
            mutableDoc.setBlob(blob, forKey: UserRecordDocumentKeys.image.rawValue)
        }

benjamin_glatzeder · March 14, 2021, 11:18am

I started over, tried again and ran into the reported issue. Creation works, 1st replacement works, 2nd replacement fails. Below is the sg_info log with comments:

# Click I love dogs on Device A
2021-03-14T11:51:47.247+01:00 [INF] SyncMsg: c:[eaa7b42] #4: Type:proposeChanges #Changes: 1
2021-03-14T11:51:47.352+01:00 [INF] CRUD: c:[eaa7b42]   Added attachment "sha1-awlXHxl5/EygmXwcySWXaVfsEDA="
2021-03-14T11:51:47.353+01:00 [INF] Cache: c:[6a68937d] GetCachedChanges("*", 1) --> 1 changes valid from #2
2021-03-14T11:51:47.353+01:00 [INF] Cache: c:[eaa7b42] GetCachedChanges("*", 1) --> 1 changes valid from #2
2021-03-14T11:51:47.353+01:00 [INF] Cache: c:[7d2e1a91] GetCachedChanges("*", 1) --> 1 changes valid from #2
2021-03-14T11:51:47.359+01:00 [INF] SyncMsg: c:[eaa7b42] #6: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w=
2021-03-14T11:51:47.364+01:00 [INF] Sync: c:[eaa7b42] Sent 1 changes to client, from seq 2
2021-03-14T11:51:47.405+01:00 [INF] SyncMsg: c:[eaa7b42] #7: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w= Rev:0-1
2021-03-14T11:51:47.466+01:00 [INF] Sync: c:[7d2e1a91] Sent 1 changes to client, from seq 2
2021-03-14T11:51:47.467+01:00 [INF] Sync: c:[6a68937d] Sent 1 changes to client, from seq 2
2021-03-14T11:51:47.473+01:00 [INF] SyncMsg: c:[7d2e1a91] #4: Type:getAttachment Digest:sha1-awlXHxl5/EygmXwcySWXaVfsEDA=
2021-03-14T11:51:47.474+01:00 [INF] SyncMsg: c:[6a68937d] #4: Type:getAttachment Digest:sha1-awlXHxl5/EygmXwcySWXaVfsEDA=
2021-03-14T11:51:47.594+01:00 [INF] SyncMsg: c:[7d2e1a91] #5: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:51:47.594+01:00 [INF] SyncMsg: c:[7d2e1a91] #6: Type:proposeChanges #Changes: 0
2021-03-14T11:51:47.605+01:00 [INF] SyncMsg: c:[6a68937d] #6: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:51:47.605+01:00 [INF] SyncMsg: c:[6a68937d] #5: Type:proposeChanges #Changes: 0
2021-03-14T11:51:47.606+01:00 [INF] SyncMsg: c:[6a68937d] #6: Type:setCheckpoint   --> 409 Document update conflict Time:531.713µs  <-- Conflict?! The document was just created. The document didn't exist before
2021-03-14T11:51:47.610+01:00 [INF] SyncMsg: c:[6a68937d] #7: Type:getCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:51:47.614+01:00 [INF] SyncMsg: c:[6a68937d] #8: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-1
2021-03-14T11:51:47.624+01:00 [INF] SyncMsg: c:[6a68937d] #9: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-2

# Click I love cats on Device A
2021-03-14T11:51:58.772+01:00 [INF] SyncMsg: c:[eaa7b42] #8: Type:proposeChanges #Changes: 1
2021-03-14T11:51:58.862+01:00 [INF] CRUD: c:[eaa7b42]   Added attachment "sha1-jIWXYmOvMOBwOFE6oQywqTIYN3Y="
2021-03-14T11:51:58.866+01:00 [INF] Cache: c:[7d2e1a91] GetCachedChanges("*", 2) --> 1 changes valid from #2
2021-03-14T11:51:58.866+01:00 [INF] Cache: c:[6a68937d] GetCachedChanges("*", 2) --> 1 changes valid from #2
2021-03-14T11:51:58.866+01:00 [INF] Cache: c:[eaa7b42] GetCachedChanges("*", 2) --> 1 changes valid from #2
2021-03-14T11:51:58.871+01:00 [INF] SyncMsg: c:[eaa7b42] #10: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w= Rev:0-2
2021-03-14T11:51:58.877+01:00 [INF] Sync: c:[eaa7b42] Sent 1 changes to client, from seq 3
2021-03-14T11:51:58.926+01:00 [INF] SyncMsg: c:[eaa7b42] #11: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w= Rev:0-3
2021-03-14T11:51:58.933+01:00 [INF] Sync: c:[6a68937d] Sent 1 changes to client, from seq 3
2021-03-14T11:51:58.933+01:00 [INF] Sync: c:[7d2e1a91] Sent 1 changes to client, from seq 3
2021-03-14T11:51:58.938+01:00 [INF] SyncMsg: c:[7d2e1a91] #7: Type:getAttachment Digest:sha1-jIWXYmOvMOBwOFE6oQywqTIYN3Y=
2021-03-14T11:51:58.939+01:00 [INF] SyncMsg: c:[6a68937d] #10: Type:getAttachment Digest:sha1-jIWXYmOvMOBwOFE6oQywqTIYN3Y=
2021-03-14T11:51:59.057+01:00 [INF] SyncMsg: c:[7d2e1a91] #8: Type:proposeChanges #Changes: 0
2021-03-14T11:51:59.057+01:00 [INF] SyncMsg: c:[7d2e1a91] #9: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-1
2021-03-14T11:51:59.058+01:00 [INF] SyncMsg: c:[6a68937d] #11: Type:proposeChanges #Changes: 0
2021-03-14T11:51:59.058+01:00 [INF] SyncMsg: c:[6a68937d] #12: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-3
2021-03-14T11:51:59.058+01:00 [INF] SyncMsg: c:[7d2e1a91] #9: Type:setCheckpoint   --> 409 Document update conflict Time:634.318µs
2021-03-14T11:51:59.068+01:00 [INF] SyncMsg: c:[6a68937d] #13: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-4
2021-03-14T11:51:59.068+01:00 [INF] SyncMsg: c:[7d2e1a91] #10: Type:getCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:51:59.078+01:00 [INF] SyncMsg: c:[7d2e1a91] #11: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-4
2021-03-14T11:51:59.078+01:00 [INF] SyncMsg: c:[7d2e1a91] #11: Type:setCheckpoint   --> 409 Document update conflict Time:592.995µs  <-- Conflict?! Only one device is updating the document. Both devices are open and connected
2021-03-14T11:51:59.087+01:00 [INF] SyncMsg: c:[7d2e1a91] #12: Type:getCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:51:59.095+01:00 [INF] SyncMsg: c:[7d2e1a91] #13: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-5
2021-03-14T11:51:59.101+01:00 [INF] SyncMsg: c:[7d2e1a91] #14: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-6

# Click I love dogs on Device A. Picture is not synced to Device B. It is still showing the cat on Device B
2021-03-14T11:52:05.447+01:00 [INF] SyncMsg: c:[eaa7b42] #12: Type:proposeChanges #Changes: 1
2021-03-14T11:52:05.467+01:00 [INF] Sync: c:[eaa7b42] proveAttachment successful for doc animallover (digest sha1-awlXHxl5/EygmXwcySWXaVfsEDA=)
2021-03-14T11:52:05.471+01:00 [INF] Cache: c:[eaa7b42] GetCachedChanges("*", 3) --> 1 changes valid from #2
2021-03-14T11:52:05.471+01:00 [INF] Cache: c:[6a68937d] GetCachedChanges("*", 3) --> 1 changes valid from #2
2021-03-14T11:52:05.471+01:00 [INF] Cache: c:[7d2e1a91] GetCachedChanges("*", 3) --> 1 changes valid from #2
2021-03-14T11:52:05.474+01:00 [INF] SyncMsg: c:[eaa7b42] #14: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w= Rev:0-4
2021-03-14T11:52:05.477+01:00 [INF] Sync: c:[eaa7b42] Sent 1 changes to client, from seq 4
2021-03-14T11:52:05.524+01:00 [INF] SyncMsg: c:[eaa7b42] #15: Type:setCheckpoint Client:cp-DDyIuiz54CDqPHi9/Uwy+L+lx5w= Rev:0-5
2021-03-14T11:52:05.592+01:00 [INF] Sync: c:[7d2e1a91] Sent 1 changes to client, from seq 4
2021-03-14T11:52:05.594+01:00 [INF] Sync: c:[6a68937d] Sent 1 changes to client, from seq 4
2021-03-14T11:52:05.626+01:00 [INF] SyncMsg: c:[6a68937d] #14: Type:proposeChanges #Changes: 0
2021-03-14T11:52:05.627+01:00 [INF] SyncMsg: c:[7d2e1a91] #15: Type:proposeChanges #Changes: 0
2021-03-14T11:52:05.672+01:00 [INF] SyncMsg: c:[7d2e1a91] #16: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-7
2021-03-14T11:52:05.672+01:00 [INF] SyncMsg: c:[6a68937d] #15: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-5
2021-03-14T11:52:05.673+01:00 [INF] SyncMsg: c:[6a68937d] #15: Type:setCheckpoint   --> 409 Document update conflict Time:568.331µs
2021-03-14T11:52:05.685+01:00 [INF] SyncMsg: c:[6a68937d] #16: Type:getCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:52:05.695+01:00 [INF] SyncMsg: c:[6a68937d] #17: Type:setCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo= Rev:0-8

# Killed app on Device B and reopened it. Still showing the cat
2021-03-14T11:52:38.161+01:00 [INF] HTTP: c:[6a68937d] #001:    --> BLIP+WebSocket connection closed
2021-03-14T11:52:38.161+01:00 [INF] HTTP: c:[7d2e1a91] #002:    --> BLIP+WebSocket connection closed
2021-03-14T11:52:38.161+01:00 [INF] Changes: c:[6a68937d] MultiChangesFeed done
2021-03-14T11:52:38.161+01:00 [INF] Changes: c:[7d2e1a91] MultiChangesFeed done
2021-03-14T11:52:43.576+01:00 [INF] HTTP:  #004: GET /database/_blipsync (as GUEST)
2021-03-14T11:52:43.576+01:00 [INF] HTTP+: #004:     --> 101 [45551c9c] Upgraded to BLIP+WebSocket protocol (as GUEST)  (0.0 ms)
2021-03-14T11:52:43.576+01:00 [INF] WS: c:#004 Start BLIP/Websocket handler
2021-03-14T11:52:43.584+01:00 [INF] SyncMsg: c:[45551c9c] #1: Type:getCheckpoint Client:cp-RN3DPwhn/YWsE7hOUjqESy8MGXo=
2021-03-14T11:52:43.588+01:00 [INF] SyncMsg: c:[45551c9c] #2: Type:subChanges Since:4 Continuous:true
2021-03-14T11:52:43.588+01:00 [INF] Sync: c:[45551c9c] Sending changes since 4
2021-03-14T11:52:43.588+01:00 [INF] Changes: c:[45551c9c] MultiChangesFeed(channels: {*}, options: {Since: 4, Limit: 0, Conflicts: false, IncludeDocs: false, Wait: true, Continuous: true, HeartbeatMs: 0, TimeoutMs: 0, ActiveOnly: false}) ...
2021-03-14T11:52:43.589+01:00 [INF] Sync: c:[45551c9c] Sent all changes to client
2021-03-14T11:52:43.589+01:00 [INF] SyncMsg: c:[45551c9c] #3: Type:proposeChanges #Changes: 0

Revision id, sequence and text match in the app on both devices (see app screenshots in my first post). The “image” key does not match.

Couchbase Server web console matches both cat and dog images!!
length 574686 is the image with the dogs.
length 519576 is the image with the cat.

In the json data part it’s the dogs. In the metadata it’s the cat.

Data:

{
  "image": {
    "@type": "blob",
    "content_type": "image/jpeg",
    "digest": "sha1-awlXHxl5/EygmXwcySWXaVfsEDA=",
    "length": 574686
  },
  "text": "I love dogs!",
  "type": "animallover"
}

Metadata:

{
  "meta": {
    "id": "animallover",
    "rev": "3-166c2ffbf27600000000000000000000",
    "expiration": 0,
    "flags": 0,
    "type": "json"
  },
  "xattrs": {
    "_sync": {
      "rev": "3-ee23e70230be4eb346ed7d0da2a12f508c3a9568",
      "sequence": 4,
      "recent_sequences": [
        2,
        3,
        4
      ],
      "history": {
        "revs": [
          "1-264463b12501359da1acfcc9b0a324c4586cc252",
          "2-43d849ada09f5f9242a00ebae96e102b5fad42a0",
          "3-ee23e70230be4eb346ed7d0da2a12f508c3a9568"
        ],
        "parents": [
          -1,
          0,
          1
        ],
        "channels": [
          null,
          null,
          null
        ]
      },
      "cas": "0x000076f2fb2f6c16",
      "value_crc32c": "0x7615919c",
      "attachments": {
        "blob_/image": {
          "content_type": "image/jpeg",
          "digest": "sha1-jIWXYmOvMOBwOFE6oQywqTIYN3Y=",
          "length": 519576,
          "revpos": 2,
          "stub": true
        }
      },
      "time_saved": "2021-03-14T11:52:05.469711192+01:00"
    }
  }
}

Running the curl command shows the cat

sudo curl -X GET "http://localhost:4985/database/animallover/blob_%2fimage" -H  "accept: application/json" --output image.jpg

blake.meike · July 21, 2021, 6:00pm

Are you using a conflict resolver? Could it be this bug:

https://issues.couchbase.com/browse/CBL-2180

benjamin_glatzeder · July 21, 2021, 7:13pm

There’s no custom conflict resolver in the test project on github. So the fix from @Giallon - thanks! - may or may not fix the issue I observed. I’m happy to give the test project another spin with a new CBL release.