Hi,
We are using Couchbase Elastic Connector for moving data to the elastic index. Right now we are doing document filtering based on document ID, Wanted to check if there is any way we can segregate documents based on a document field.
https://docs.couchbase.com/elasticsearch-connector/current/configuration.html#document-matching-rules
Thanks
Hi Ashish,
It is not currently possible to define types based on document fields. This issue is tracked as CBES-146. It has been in the backlog for a long time, because it’s difficult to handle correctly (see this related forum topic).
If you’re using Couchbase 7 and the field you want to look at indicates the type of the document, a possible solution is to store the different types of documents in different Couchbase collections, and defined the type based on the collection name. (Migrating to using collections can be a lot of work, though.)
An alternative workaround would be to use an Elasticsearch ingest pipeline to inspect the document field and modify the destination index. If you do this, you’ll probably want to configure the type definition to use ignoreDeletes=true, since the connector won’t know which index to delete the documents from.
Thanks,
David
1 Like