Remove corrupted documents

Community
1

Hi,

I’m using Couchbase 6.0.1 and having issue related to my some documents inside bucket
I;m using OCP with gluster storage, and yesterday it restarted unexpectedy

I’ve found some of my documents changed into binary with type base64 ( I assume this is corrupted documents)

Since I need to delete and resubmit the data, is there any fastest way to detect this corrupted documents?
Or is there any way to recover my documents?

Thanks

2

Would
WHERE meta().type <> "json"
or
WHERE meta().type = "base64"
help?

3

This means I need to create primary index, right?
I have 90mio docs and it would be very slow I think, let me try

4

There is also a way to index metadata

5

Option 1: See if you can create view and only emit the document key if it is binary document. Then use those keys delete.
https://docs.couchbase.com/server/current/learn/views/views-query-samples.html

Option 2: If 6.0.1 supports eventing service . Use that delete binary documents. Checkout @jon.strabala response for https://stackoverflow.com/questions/65312247/add-field-to-existing-documents-over-million-records/65365278#65365278

Option 3: N1QL . Checkout

   CREATE INDEX ix1 ON mybucket(IS_BINARY(self) ) WHERE IS_BINARY(self) = true;
   DELETE FROM mybucket AS b WHRE  IS_BINARY(b) = true;
6

but for type is not covered for this.
I’ve tried below index creation:

CREATE INDEX idx_meta_type ON myBucket ( META().type ) WITH { “defer_build”:true};

Below is the error message:

“msg”: “Expression not indexable: (meta().type) - at )”,

Is there any way to recover my doc instead?

7

I’ve found the solution using map reduce.
Now I can get doc id list to be fixed

Thanks