Set document expiration from a field of Kafka consumed event

Kafka Connector
1

Hi, I have a flow from Kafka to Couchbase with Sink Connector.

I don’t want to give a global couchbase.document.expiration option to connector.

I want to set this field from the Kafka event field which will be ttl. So each document will be created with that ttl value for expiration.

How can I achieve this? I just want to pass the Kafka event field value to couchbase.document.expiration. So each document can have different expiration according to its ttl value of the Kafka event.

2

@david.nault do you have any opinion for this?

3

If the expiry times are specific to the type of document, one solution would be to send each type of document to a different collection (by publishing them to different topics, and configuring the sink connector’s “topic to collection” map). Then set a max TTL on each collection (you can set the max TTL when creating a collection).

Otherwise, the only solution I can think of is to write a custom SinkHandler that does what you want.

Thanks,
David