We have a requirement around producers writting documents to couchbase, kafka connectors reading couchbase documents and delivering to kafka.
We would like the documents to be deleted once message delivered successful.
Can kafka connector delete or can it update ttl of couchbase bucket.
The Couchbase source connector does not know how to delete the document after sending it, but maybe you could have a separate Kafka consumer read from the topic and delete whatever documents it sees. This could even be a Couchbase sink connector, with a Single Message Transform that sets every record value to null; by default the Couchbase sink interprets a null value as an instruction to remove the document.
Alternatively, if you don’t need the messages to be deleted right away, perhaps you could set a maxTTL on the source bucket with an expiry period long enough to accommodate your worst-case disaster recovery window.
Thanks,
David
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.