Sync Gateway delete question

Mobile Sync Gateway
1

So I created a JSON file of data that I wanted to load into my Couchbase server via the sync gateway using the public REST API online function (https://developer.couchbase.com/documentation/mobile/1.4/references/sync-gateway/rest-api/index.html#/database - bulk docs). There were about 30 documents in the JSON file. It loaded ok and I could see it sync down to the mobile version once I fired up the app. So far so good. Then I realized I made a mistake and loaded a prior version of the JSON file. Without thinking I just ran a delete statement in the Couchbase console instead of running the deletes through the sync gateway. My questions are:

  1. Cleaning up the records from the mobile CBL. Should I write one-time code within my app to delete all those records? Would it confuse the gateway if I did that? I suppose I could just delete the entire app, but I’m thinking long term and would like to better understand how to use Couchbase.

  2. What would have been the best way to delete those records? I was able to get all the document id’s of the ones I wanted to delete. Perhaps I can create a JSON file of all those documents that I want to delete and do a similar bulk delete. I didn’t see a bulk delete though. I could have done them one by one, but doing that through the REST API would be very tedious.

Thanks,
David

2

The best way to think about this is to think about each document’s canonical history. Let’s say you had two documents you imported, foo and bar (because I’m not all that inventive with names). Each of these then had a first revision: 1- followed by a hash of the body. These then got sync’d over to Couchbase Lite.

By deleting the documents directly in Couchbase Server, you essentially performed a purge. You didn’t touch the canonical history of those documents at all, you just made Sync Gateway forget about them. This has a few interesting ramifications:

  • If you tried to push from Couchbase Lite to Sync Gateway, they’ll get pushed back (Couchbase Lite says “Hey have you heard of foo/1-abc” and Sync Gateway checks and says “Nope, never heard of it in my life, please send it over.”)

  • If you re-imported the incorrect documents into Sync Gateway again they would hash the same; so if you then tried to get Couchbase Lite to pull from Sync Gateway, the conversation would be more along the lines of
    SG: "Hey, have you heard of foo/1-abc?"
    CBL: "Yup, way ahead of you, already got that."
    If you then did a normal delete on the Sync Gateway side, that would create a new revision 2-def, so the next replication would go:
    SG: "Have you heard of foo/2-def? It’s a child of 1-abc."
    CBL: "Nope, haven’t seen it, send it over please."
    I might be getting a little over the top with this example, but you get the idea ;).

  • If you imported the newer, correct docs with the same key, they’d hash differently, so you might have foo/1-xyz. That’s fine from Sync Gateway’s perspective, because it’s the only foo it knows about, but when that hits Couchbase Lite it becomes a conflict (Priya’s Blog Post is a good read on this). Essentially though, this wouldn’t solve the problem, even if it might look like it to start with.

Depending on the situation, there are different ways to go about it. In development, it might be easiest to wipe everything and start again; in production, you probably wouldn’t have that option so something along the lines of re-importing and correctly deleting might be better.

Hope that helps, let me know if you have any questions :slight_smile:

1 Like
3

Hey JFlath,

Thanks for the in-depth explanation and it pretty much makes sense, although I’m still way more versed in the SQL Server/Oracle world so the transition is taking time.

So I reloaded the original JSON document and confirmed there were no issues with my app firing up (vis-a-vis with the sync gateway). I conveniently had a debug routine where it spit out all the document id’s and a type property to the debug window. In that loop I just added a doc?.delete() statement. The next time I fired up the app it ran through all the deletes. I ran the app again and saw that there were no documents of that type. To confirm, I went over to the Couchbase database and ran a N1QL query and also saw that there were no docs. Problem solved, this time.

How would you have done multiple document deletes? I wish the public REST API had a multiple delete function too.

Thanks!
David

EDIT: Well, it mostly corrected the situation. So when I reloaded the JSON file to the sync_gateway, I see it added the documents to the Couchbase server, but for some reason they are not being brought down to the mobile app. Thinking that I just confused the heck out of the sync gateway, I went ahead and just deleted my app and reinstalled it (I’m testing using a simulator). Upon opening up the application, it is not bringing down the documents to the local lite database. I would have expected that pushing them through the sync_gateway would have queued them up for syncing. Going to try a few things, perhaps in a different sequence.

EDIT (again): So I then tried to reload the app on my actual iPhone and I saw it sync down to the iPhone. I then retried the app through the simulator and I saw it sync down to the simulator. Going to retrace some of my steps to see if I can understand this a little bit better.