Couchbase Elastic Connector filter on different field than ID

Hi Ashish,

It is not currently possible to define types based on document fields. This issue is tracked as CBES-146. It has been in the backlog for a long time, because it’s difficult to handle correctly (see this related forum topic).

If you’re using Couchbase 7 and the field you want to look at indicates the type of the document, a possible solution is to store the different types of documents in different Couchbase collections, and defined the type based on the collection name. (Migrating to using collections can be a lot of work, though.)

An alternative workaround would be to use an Elasticsearch ingest pipeline to inspect the document field and modify the destination index. If you do this, you’ll probably want to configure the type definition to use ignoreDeletes=true, since the connector won’t know which index to delete the documents from.

Thanks,
David

1 Like