Source Connector fails to open DCP Connection

I am trying to run the Kafak Source Connector. When I start it using the command from the quickstart documentation, I can see from logs that the bootstrapping of the Couchbase cluster nodes is successful, but then it trying to open a connection to CB on port 11210 and fails with below message and just repeats this failure continuously for all four nodes. I have verified that there is no firewall rule preventing the connection and the port is accessible from the server where I am running the Kafka Connector and they are on the same domain.

What else can cause this error? Do you have any troubleshooting suggestions?

INFO Connect attempt to <host>/ failed
Could not open DCP Connection: Failed in the OPEN step, response status is 0x0024 (No access / insufficient permissions)

Hi Atkinsr,

I would suggest double-checking the bucket name and user credentials.

Start by using the Administrator user. Then try a user with more limited access; make sure this user has the DCP Reader role for the bucket.


Thank you David! It was the DCP Reader role that was not enabled for the user. After fixing that, the Connector starts successfully and is polling every 30 secconds per config.

However, I don’t see any messages being sent after I add new documents to the bucket. I don’t see any errors in the Connector logs, but inserting into the bucket does not trigger a message.

What’s the value of your couchbase.persistence_polling_interval setting? Some versions of the connector shipped with a quickstart config that has an inappropriate value. You probably want something like 100ms for this, or 0 to disable persistence polling.

How are you watching for Kafka messages? Are you sure you’re watching the same topic the connector is publishing to?

Thanks for the response. I’m sure I am missing something simple again. The polling interval is set at 30 seconds as noted in my last post and I can see a log ouput from the connector every 30 seconds that looks like this:

[2020-07-29 17:49:16,029] INFO WorkerSourceTask{id=test-couchbase-source-1} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:424)
[2020-07-29 17:49:16,029] INFO WorkerSourceTask{id=test-couchbase-source-1} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:441)

The default config uses a topic named “test-default”. This topic never gets created in kafka (I would expect it to be created once there is a message generated). However, I thought maybe the problem was that I need to pre-create the topic, so I did that. I created a new topic called “test-topic” and re-configured the Connector to publish to that topic. I also sent a few test messages to the topic to ensure it was working. I ran a consumer from the command line as follows in order to monitor the messages: --bootstrap-server localhost:9092 --topic test-topic --from-beginning

My test messages were output, but the Connector never published anything to this topic. I inserted at least a half dozen new documents into the bucket that is the same one the Connector is configured on, but nothing.

The log messages about flushing 0 outstanding messages for offset commit are for saving the connector’s progress, and are unrelated to persistence polling. (See the description of in the Kafka Connect config documentation.)

I would try disabling persistence polling by setting couchbase.persistence_polling_interval to 0. The results will give us more information.

Incidentally, is the source bucket an “ephemeral” bucket?

The source bucket IS an ephermeral bucket, does that matter?

It is working now. I saw the comment in the config file about ephermeral buckets which require setting polling to 0. Once I did that, the messages began streaming…lots of them :slight_smile:

Thank you for your help David.

1 Like

Just to summarize the solution to this topic:

  1. I needed to change user permissions to enable the “Data DCP Reader” role for the ‘connection.username’ specified in the source config properties file.

  2. I needed to set the value of ‘couchbase.persistence_polling_interval’ in the source config file to 0 because the bucket specified in ‘connection.bucket’ was ephemeral as noted in the comment in the config file: If the source is an ephemeral bucket (which never persists documents) this value must be set to 0 to disable the persistence check.