Hi, I have one connector which has below configuration fields.
"topics": "topic.a, topic.b",
You can see that
topic.b is listened and written to
However I don’t want to store data of two different topics into a single bucket. I want to store
topic.a data to bucket
topic.b data to bucket
b. Is there a way to do this with one connector?
Hi, and welcome to the forum.
A single instance of the sink connector can write to only one bucket.
I filed KAFKAC-300 to track this feature request.
In the meantime: if you’re using Couchbase 7 or later, you could write data from different Kafka topics to different collections in the same bucket (see the
couchbase.topic.to.collection connector config property). Then, if you’re using Couchbase Enterprise Edition, you could use the Eventing service to copy the data from one of the collections to a different bucket. At least, I think that’s something Eventing can do. What do you think, @jon.strabala ?
Would using Couchbase 7 and storing the documents in different collections solve your issue, or do you definitely want to store them in separate buckets?
^^ If this suffices it will be the best practice.
Principal Product Manager - Server