I have a Kafka 3 node cluster running on a single machine with Kafka and Zookeeper running in Docker instances. I worked through the Kafka connector quick start tutorial with the JDBC connector and MySQL. For the tutorial I created a directory of /var/lib/kafka/jars to which I copied the JDBC connector jar. I specified this location as a mounting point in the Kafka connect docker instance as: -v /var/lib/kafka/jars:/etc/kafka-connect/jars. This worked fine for the JDBC connector. I am not sure where I should place the Kafka Couchbase connector jars and configuration file(s) and how I should specify the mount point for them.
you should put the couchbase connector jars right there in /var/lib/kafka/jars. Configuration should be similar to JDBC. Also you can use REST endpoint it to setup connector:
curl http://localhost:8083/connectors -X POST -H "Content-Type: application/json" --data '
{"name": "cb",
"config": {
"connector.class":"com.couchbase.connect.kafka.CouchbaseSourceConnector",
"tasks.max":"1",
"topic.name":"test",
"connection.cluster_address":"172.18.0.1",
"connection.bucket":"default",
"connection.password":"password",
"connection.username":"Administrator",
}}'Hi @kim.turner
I have a Kafka 3 node cluster running on kubernetes. Also i have 3 running instances of zookeeper in kubernetes only. I have to install Kafka couchbase connector on it using kafka only, and not through confluent platform.
Have u done this? I need help regarding this. And if we copy jars and config, will data from my bucket will be fetched by all the 3 nodes. I mean will there be data duplication on my kafka topic in this case.
Thanks in advance!