{"id":8755,"date":"2020-06-02T10:01:46","date_gmt":"2020-06-02T17:01:46","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=8755"},"modified":"2023-09-29T13:48:47","modified_gmt":"2023-09-29T20:48:47","slug":"store-sync-binary-data-attachments-blobs-couchbase-mobile","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/store-sync-binary-data-attachments-blobs-couchbase-mobile\/","title":{"rendered":"Handle Binary Data Attachments & Blobs with Couchbase Mobile"},"content":{"rendered":"

Couchbase Mobile<\/a> supports a JSON document style NoSQL data model. In addition to supporting the standard JSON data types, Couchbase Mobile also supports binary data that include images, audio, video, PDF files, etc. A JSON document can be associated with one or more elements of binary data referred to as \u201cattachments\u201d or \u201cblobs\u201d. The binary data can be synced between Couchbase Lite clients and the server via the Sync Gateway. In this post, we discuss how to create binary data attachments, how to retrieve and update them. We also take a look under the hood at how attachments are internally represented, related idiosyncrasies and how to deal with them.<\/p>\n

Everything in this post applies to a Couchbase Mobile 2.x based deployment.<\/p>\n

Background: Attachments and Blobs<\/h2>\n

Support for associating binary data with JSON documents within Couchbase Mobile has evolved over the years. The internal representation of binary data within the JSON document has changed across versions of Couchbase Mobile. In Couchbase Mobile 1.x, binary data was stored in the form of \u201cattachments\u201d within a top-level \u201c_attachments<\/em>\u201d attribute. Couchbase Mobile introduced the blob<\/a> data type for storing binary data. In most cases, the discrepancy between the representations across versions is seamlessly handled by Couchbase Mobile so end users don\u2019t have to do anything special within their apps to deal with it. However, there are certain cases wherein app developers would have to take extra measures to deal with the discrepancy. We will also discuss those measures in this post and try to address some commonly asked questions.<\/p>\n

Workflow #1: Handling attachments created on Couchbase Lite<\/h2>\n

Let\u2019s take a look at how you can create JSON documents with binary data attachments with Couchbase Lite and sync them over to the server-side. This is the flow that we will be describing:<\/p>\n

\"\"<\/figure>\n

Create binary data attachments on Couchbase Lite<\/h3>\n

Developers must use the blob API<\/a> for creating blob data. A document can be associated with one or more attachments or blobs. Here is a code snippet that shows the usage of this API in swift. Refer to the developer documentation for equivalent code snippets for other platforms.<\/p>\n

\r\n let mutableDoc = MutableDocument.init(id: \"user::priya\")\r\n if let profileImage = UIImage(named: \"profile.jpg\"), let imageData = UIImageJPEGRepresentation(profileImage, 0.75) {\r\n   let blob = Blob(contentType: \"image\/jpeg\", data: imageData)\r\n   mutableDoc.setBlob(blob, forKey: \"image\")\r\n }\r\n \/\/ \u2026 Add other properties\r\n mutableDoc.setString(\"user\", forKey: \"type\")\r\n mutableDoc.setString(\"Priya\", forKey: \"name\")\r\n mutableDoc.setString(\"priya.rajagopal@couchbase.com, forKey: \"email\")\r\n do {\r\n   try? db.saveDocument(mutableDoc)\r\n }\r\n catch {\r\n   print(\"Error in saving document : (error)\")\r\n }\r\n<\/code><\/pre>\n
Internal Representation<\/h3>\n
When the document is created in Couchbase Lite, internally, it looks something like this:<\/p>\n
{\r\n  \"email\": \"priya.rajagopal@couchbase.com\",\r\n  \"image\": {\r\n   \"length\": 3888349,\r\n   \"digest\": \"sha1\u20134xlj1AKFgLdzcD7a1pVChrVTJIc=\",\r\n   \"content_type\": \"image\/jpeg\",\r\n   \"@type\": \"blob\"\r\n  },\r\n \"type\": \"user\",\r\n \"name\": \"Priya\",\r\n \"id\": \"user::priya\",\r\n \"rev\": \"1\u20131c8502034001b333cc469fe8c4c39e112eedf8a3\"\r\n}<\/code><\/pre>\n
Notice the \u201c@type\u201d: \u201cblob\u201d <\/em>type entry created for the image type data.<\/p>\n
Note<\/strong> that there are several system-level metadata such as _id<\/em> that are included in the document. For brevity, not all of it is shown in the example. Applications must never make any assumptions about the format and availability of system-level metadata and for that reason, apps must never access those attributes directly. Always use the metadata retrieval options such as meta().id<\/em>.<\/p>\n
Syncing attachments to Sync Gateway<\/h3>\n
Sync Gateway is backward compatible with Couchbase Mobile 1.x. This implies that the Sync Gateway needs to be capable of processing binary data using the 1.x _ attachments<\/em> style representation as well as the 2.x blob<\/em> type. That also implies that when the Couchbase Lite 2.x client pushes up data to the Sync Gateway it needs to send it in a format that is compatible with 1.x clients.<\/p>\n
For that reason, when Couchbase Lite syncs the document with the Sync Gateway, it adds the _<\/em><\/strong>attachments\u00a0<\/em>entry into the document. So the document when pushed up would look something like the example below. The list of attachments associated with the document is specified within the _attachments<\/em> object.<\/p>\n
{\r\n  \"attachments\": {\r\n  \"blob\/image\": {\r\n   \"content_type\": \"image\/jpeg\",\r\n   \"digest\": \"sha1\u20134xlj1AKFgLdzcD7a1pVChrVTJIc=\",\r\n   \"length\": 3888349,\r\n   \"revpos\": 1,\r\n   \"stub\": true\r\n   }\r\n  },\r\n  \"email\": \"priya.rajagopal@couchbase.com\",\r\n  \"image\": {\r\n  \"@type\": \"blob\",\r\n  \"content_type\": \"image\/jpeg\",\r\n  \"digest\": \"sha1\u20134xlj1AKFgLdzcD7a1pVChrVTJIc=\",\r\n  \"length\": 3888349\r\n  },\r\n  \"name\": \"Priya\",\r\n  \"type\": \"user\",\r\n  \"id\": \"user::priya\",\r\n  \"rev\": \"1\u20131c8502034001b333cc469fe8c4c39e112eedf8a3\",\r\n}<\/code><\/em><\/pre>\n
Note<\/strong> that there are several system-level metadata such as _id<\/em> that is included in the document. For brevity, not all of it is shown in the example. Applications must never make any assumptions about the format and availability of system-level metadata and for that reason, apps must never access those attributes directly. Always use the metadata retrieval options such as meta().id<\/em>.<\/p>\n
Retrieval of attachment on Sync Gateway<\/h3>\n
The attachment data must be retrieved through the Sync Gateway _attachments<\/em> REST<\/a> endpoint. At the time of writing of this post, attachments cannot be directly managed using the Couchbase Server SDKs.<\/p>\n
Here is a sample curl command to retrieve the attachment(s) associated with a document using the _attachments<\/em> REST endpoint. You would replace the authorization header with suitable credentials corresponding to the user configured in your system. Also, notice the name of the attachment, “blob_<\/em>%2Fimage” is the URL encoded version of “blob<\/em>\/image”_.<\/p>\n
curl -X GET \r\n 'https:\/\/sync-gateway-url:4984\/dbname\/user::priya\/blob_%2Fimage' \r\n -H 'Accept: application\/json' \r\n -H 'Authorization: Basic ZGVtbzpwYXNzd29yZA=='<\/code><\/pre>\n
Updating attachments on Sync Gateway<\/h3>\n
The attachment data must be updated through the Sync Gateway _attachments<\/em> REST<\/a> endpoint. At the time of writing of this post, mobile attachments cannot be managed using Couchbase Server SDKs.<\/p>\n
Here is a sample curl command to update an attachment associated with a document using the _attachments<\/em> REST endpoint. You would replace the authorization header with suitable credentials corresponding to the user configured in your setup. Also, notice that the \u201crev\u201d<\/em> parameter must be provided. This parameter corresponds to the revision of the document that is to be updated. You can retrieve the revId using the GET document REST<\/a>.<\/p>\n
 curl -i -X PUT 'https:\/\/sync-gateway-url:4984\/dbname\/user::priya\/blob_%2Fimage?rev=12-fa2bf00dab7e811eb562a502429ec633' \r\n -H 'Accept: application\/json' \r\n -H 'Authorization: Basic ZGVtbzE6cGFzc3dvcmQ=' \r\n -H 'Content-Type: image\/png' \r\n -H 'cache-control: no-cache' \r\n \u2013data-binary \"@layered.png\"<\/code><\/pre>\n
But\u2026wait..a mismatch!<\/h3>\n
Now, after you update the attachment, if you retrieve the document using the GET document REST<\/a>,<\/p>\n
 curl -X GET \r\n 'https:\/\/sync-gateway-url:4984\/dbname\/user::priya\/blob_%2Fimage' \r\n -H 'Accept: application\/json' \r\n -H 'Authorization: Basic ZGVtbzE6cGFzc3dvcmQ=' \r\n -H 'Content-Type: application\/json' \r\n -H 'cache-control: no-cache'<\/code><\/pre>\n
The corresponding response would look something like this:<\/p>\n
{\r\n  \"attachments\": {\r\n  \"blob\/image\": {\r\n   \"content_type\": \"image\/png\",\r\n   \"digest\": \"sha1-XeeXt0I014+1qpH+s2S2AUSg4II=\",\r\n   \"length\": 3444504,\r\n   \"revpos\": 16,\r\n   \"stub\": true\r\n   }\r\n  },\r\n  \"id\": \"user::demo1\",\r\n  \"rev\": \"13-e61d1a9e63069030212007a8b5eddde9\",\r\n  \"address\": \"\",\r\n  \"email\": \"demo1\",\r\n  \"image\": {\r\n   \"@type\": \"blob\",\r\n   \"content_type\": \"image\/jpeg\",\r\n   \"digest\": \"sha1\u20134xlj1AKFgLdzcD7a1pVChrVTJIc=\",\r\n   \"length\": 3888349\r\n  },\r\n  \"name\": \"Priya Rajagopal\",\r\n  \"type\": \"user\"\r\n}<\/code><\/pre>\n
You will notice that the _attachment<\/em> and blob<\/em> entry do not match. While the _attachment<\/em> entry points to the latest image, the blob<\/em> entry still describes the old image. But that is OK<\/em>!<\/p>\n
The reason for this discrepancy is because the Sync Gateway only deals with 1.x style attachments.<\/p>\n
So how does this still work?<\/p>\n
This works because within the context of the Sync Gateway which deals with 1.x style attachments, only the attachment entry is honored.<\/p>\n
But what about Couchbase Lite? What happens when the updated document is synced down by the Couchbase Lite 2.x client?<\/p>\n
When the document is replicated by Couchbase Lite 2.x client, Couchbase Lite looks for the presence of _attachments<\/em> and blobs<\/em> within the document and implements appropriate logic to identify that this was a 2.x style document that was created by a 2.x client but was subsequently updated by a 1.x client (such as the Sync Gateway REST API). It, therefore, treats the _attachments<\/em> entry as the \u201creal\u201d attachment and unifies the corresponding blob entry.<\/p>\n
From a developer\u2019s perspective, all this is handled automatically. So you don\u2019t really have to worry about any of the details. As a developer, you would have to know how to retrieve the updated attachment from within Couchbase Lite enabled app.<\/p>\n
Retrieval of updated attachment on Couchbase Lite<\/h3>\n
When the updated attachment is synced over to your Couchbase Lite app, \u00a0use the Blob API<\/a> to retrieve the data. Here is a code snippet that shows the usage of this API in swift. Refer to the developer documentation for equivalent code snippets for other platforms.<\/p>\n
\r\n if let doc = db.document(withID: \"user::priya\") {\r\n     let blobValue = doc.blob(forKey:\"image\")?.content\r\n     \/\/ use the blobValue\r\n }\r\n<\/code><\/pre>\n
Now let\u2019s look at the reverse flow.<\/p>\n
Workflow #2: Handling of attachments created on Sync Gateway<\/h2>\n
Let\u2019s take a look at how you can create JSON documents with binary data attachments on Sync Gateway and sync it over to the Couchbase Lite side.
\nThis is the flow that we will be describing –<\/p>\n
\"\"<\/figure>\n
Create binary data attachments on Server<\/h3>\n
In order to attach binary data on the Couchbase Server side that can be synced over to the clients via the Sync Gateway, you would have to use the Sync Gateway attachments REST endpoint<\/a>. At the time of writing of this post, mobile compatible attachments cannot be directly created using the Couchbase Server SDKs. The attachment data that is created through the Sync Gateway REST endpoint is persisted in the Couchbase Server bucket and synced over to Couchbase Lite clients subject to the access control policies configured on the Sync Gateway.<\/p>\n
In order to do this, first create a document (or retrieve a previously created document) and then create the attachment for the document. Alternatively, you could create a multi-part document with both JSON and binary data. But that could be tedious as you would need to also generate the relevant attachment metadata. So the steps outlined below is my preferred option<\/p>\n