Approach to deletion of large number of documents
Hey all, I'm evaluating Couchbase and I've got a question about deletion. Is there a way to efficiently delete an entire collection of documents with a value?
I've searched but can't seem to find an answer. Most of our app fits the CB model to a T, but this one issue may be a showstopper.
Say we have 10000+ small documents 'foo' associated with another document key 'bar'. If bar is deleted we need to delete all related 'foo' documents. It seems the recommended way to do this is to query for all the related 'foo' document keys and delete them one at a time synchronously. That just seems like the wrong approach with the number of associated documents we could have, but I can't see any other ways to do it via the api (Java).
Another approach possibly is to embed all the smaller 'foo' documents in the larger 'bar' document so only 1 delete happens. But, then I'm concerned about sub-reads and reading the entire 'bar' document, which can be very large, just to read a small subset of the document.
TIA, if anyone can comment on the above issue it would be much appreciated.
Here's a trick for you if you can tolerate a couple seconds of a deleted document still being available.
Do a setMulti on the documents to be deleted with an expiry of, say, 3 seconds. (or 1 second :)
The document won't be able to be retrieved after the expiry time and couchbase will purge the docs in its hourly run.
Hello,
If you cannot embed all the documents in a parent you have to delete all the documents one by one.
You can create views to help your to get the keys of children documents.
The delete is extremly fast since it is done in memory on the server.
When you say that is a showstopper I do not understand why. Can you elaborate?
Regards
Tug
@tgrall