Feature request: Elasticsearch Connector should be able to filter documents

Elasticsearch Connector
9

Hi David,

I have applied your suggestions on the filtering deployment and now seeing the following messages:

19:22:44.367 [nioEventLoopGroup-5-2] INFO  c.c.c.d.DcpHelper - do not pass but also not released. releasing
19:22:44.367 [nioEventLoopGroup-5-2] INFO  c.c.c.d.DcpHelper - do not pass but also not released. releasing
19:22:44.367 [nioEventLoopGroup-5-1] INFO  c.c.c.d.DcpHelper - do not pass but also not released. releasing
19:22:44.367 [nioEventLoopGroup-5-2] INFO  c.c.c.d.DcpHelper - do not pass but also not released. releasing
19:22:44.367 [nioEventLoopGroup-5-1] INFO  c.c.c.d.DcpHelper - do not pass but also not released. releasing

I know that the first 50% of the Couchbase history should be filtered out based on the documents and the filters I now applied. Let’s see how far it can go. After the first 50%, documents should start appearing near the sink.

In the meantime, I noticed that you mentioned CheckpointService as dependent on the sink receiving any messages. Could this potentially lead to a refactor of CheckpointService to make sure that events not hitting the sink to get counted for?

Thanks,
Zoltán