I have a need to write few millions of records to a kafka topic using the source connector. Is there an option available in Source Connector to throttle reading/writing the documents from Couchbase and ingest to kafka topic?
You want to know how to make the Kafka connector slower? That’s a refreshing change
Serious answer: There’s nothing built into the connector to support that. Kafka supports quotas, but I’m not sure whether its possible to apply a quota to a connector.
One thing you could try would be to write a Single Message Transform that introduces a delay if it detects messages are flowing too quickly.
cc: @kotak25krishna