Missing attachment

I have some image stored as attachments, but in one document 1 of the 3 is somehow not working. The attachment lump from the doc is below. When I hit the url for 2 of them (‘low-res’ and ‘thumbnail’) the images are shown in my browser as expected. The 3rd (‘original’) comes back as {error: "not_found", reason: "missing"}. Any tips on how I can debug this?

“_attachments”: {
“low-res”: {
“digest”: “sha1-w3zGYxPNdMqlbQEd8mSL8Aiex40=”,
“length”: 47959,
“stub”: true,
“content_type”: “image/jpeg”,
“revpos”: 3
},
“thumbnail”: {
“digest”: “sha1-UF9t5vZ7uUlC2XC4NZiRsOdO0s8=”,
“length”: 8081,
“stub”: true,
“content_type”: “image/jpeg”,
“revpos”: 2
},
“original”: {
“digest”: “sha1-5FovQJ5EGDY68Nh7gNAwFWxs23Q=”,
“length”: 1260363,
“stub”: true,
“content_type”: “image/jpeg”,
“revpos”: 4
}
}

Look into the application’s data directory, in Application Support/CouchbaseLite/name_of_database.cblite2/Attachments/. You’ll see a bunch of files named *.blob where “*” is a hex SHA-1 digest.

Unfortunately the _attachments.digest property uses a (more compact) base64 encoding instead of hex; but it can be converted to hex like this:

$ echo '5FovQJ5EGDY68Nh7gNAwFWxs23Q=' | base64 --decode - | hexdump -e '20/1 "%01x"'
e45a2f409e4418363af0d87b80d030156c6cdb74

So check whether there’s a file e45a2f409e4418363af0d87b80d030156c6cdb74.blob.

I don’t have access to the users phone. This document is one that got replicated to the sync gateway, which I do have access to. The data in my post above has come from the sync-gateway.

Hm. If the doc originated on the user’s device, i.e. the attachment was originally created there, it’s especially weird that it wouldn’t be found.

On SG, you can try using the REST API (same as in CBL) to find the attachment to see if it got uploaded.

Sorry, was being clear. That output is from the sync-gateway rest api. The doc says there’s an attachment called ‘original’, but when I try to load it I just get some json back saying ‘not found’. The other attachments in the save document load fine.

@nick-couchbase did you get this resolved?

You can persist data on sync gateway and examine it, if you have a test arrangement where the performance hit won’t matter.

No, its a mystery. As far as I can tell the server has an attachment with a non-zero size, but when you try to access it you just get an error.

So when you run something like curl http://gateway:4984/dbname/docid/original, you get a 404 with response body {error: "not_found", reason: "missing"}?

You can use the server admin console to look at the contents of the bucket. The attachment’s key will be _sync:att: plus the digest of the attachment (either base64 or hex, can’t remember which.)

Thanks jens. Yes, that right:

curl -u user:pass http://localhost:4984/sync_gateway/7559529-065b-4357-9609-cb80dc0552eb/original
{"error":"not_found","reason":"missing"}

Not sure I’m using the admin console correctly. I went to Data Buckets > Documents (in my bucket) … then what? I tried entering _sync:att:sha1-UF9t5vZ7uUlC2XC4NZiRsOdO0s8= where it says “document id”, but just got a blank screen and a warning “Warning: Editing of binary document is not allowed”

Any other idea on what could cause this? It seems to happen from time to time when the user replicates an image from CBL

Does the attachment document associated with the missing entries exist when you attempt to view it in the Admin Console? (i.e. do you also get the “Warning: Editing of binary document is not allowed” when attempting to view _sync:att:sha1-5FovQJ5EGDY68Nh7gNAwFWxs23Q= in the original scenario?

Yes, the document is there. I could even read the other attachments that were on it. Just one of them was displaying this weird behaviour. Since then I’ve seen it happen a few more times.

The error definitely looks like Sync Gateway is failing to retrieve the attachment - I see the same behaviour if I manually delete the attachment in the bucket.

Given the relative sizes of the attachments, I wonder if a timeout is happening between Sync Gateway and Couchbase Server during retrieval of the larger attachment. I would expect this type of timeout to show up in the Sync Gateway logs - is there anything related to the doc/attachement in the SG logs when the request fails?

I’ve refreshed the db an lost the document now. I’ll keep an eye out for it happening again and check. The file isn’t particular big, its just a small image.

Ok - I’ll watch for a subsequent update. Based on your initial post, it’s showing the length of ‘original’ as about 1.26 MB, which isn’t huge, but is quite a bit larger than the other two attachments on the document (48K, 8K).

That’s true. There are several others in the system though, large ones. This is the only one I spotted that wouldn’t loads. And it wouldn’t load consistently. Shame I lost it.

Hi Nick,

I am experiencing the same issue as you described for missing attachements in synchronized Couchbase documents. That’s why I am reopening this old thread.
Attachment content size is present in the Couchbase document but it’s impossible to retrieve the attachment through its URL.
I have approximatif 20% of all my documents that are in such situation.
So I would like to know if you 're still experiencing this issue or if it was solved and how.
I’m running Couchbase Server 4.1,. Sync gateway 1.2 ans CB Lite 1.2.

Thanks,
Olivier

Couchbase Lite for which platform? (iOS/Mac, Android, .NET…)

And to confirm: the documents & attachments were created on a client device and then replicated to Sync Gateway? Yet you get a 404 when you try to GET the attachment via the Sync Gateway REST API? Is the attachment still accessible on the device that created it?

I don’t believe I’m seeing this problem right now. Using CBL 1.3 and SG 1.3

Hi Jens,

Yes the attachments and the CB document were created on an Android device
and replicated through the Sync gateway.
Once successfully replicated, we do have a 404 when trying to retrieve the
attachment via the Sync gateway api, although the CB document can be
retrieved , with a reference to the attachment, giving it’s content length.
Unfortunately the document and its attachments are not present anymore as
we physically delete the CB Lite DB after every successfull replication, as
we don’t need these documents anymore on the device to savez space (by the
way what would be the alternative to do that without physically deleting
the whole CB Lite DB?)
In your previous answers, you explained that the attachment could actually
be on the CB server but that could be an issue with the Sync gateway rest
api. Was it confirmed? Can you reexplain how to do that query and calculate
the attachment id?

Thanks!