Kafka Source connector's filter parallelism

Sorry, I misunderstood your requirements earlier. And I probably still don’t understand very well :slight_smile: Rather than get myself into trouble by giving bad design advice, I’ll try to stick to just clarifying how the connector behaves.

If you’re using persistence polling, the latest version of a document is the one with the highest dcp seqno. It’s also the most recent message in the Kafka topic, unless the stream was temporarily rewound due to a connector restart.

If you’re looking for a per-document counter that’s incremented whenever the document changes, that revSeqNo field we discussed in the other thread might be exactly what you want. I’m not sure if it’s reset to zero if a document is deleted and recreated; it should be easy enough to find out by experimenting, although we’re getting deep into unsupported territory.