Hi @david.nault,
kafka connect stopped streaming with this in logs :
[2018-09-03 12:51:23,943] INFO Starting to Stream for 35 partitions (com.couchbase.client.dcp.Client:115)
[2018-09-03 12:51:23,946] WARN Received rollback for vbucket 342 to seqno 185925 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,947] WARN Received rollback for vbucket 343 to seqno 185814 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,947] WARN Received rollback for vbucket 344 to seqno 185224 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,948] WARN Received rollback for vbucket 345 to seqno 184133 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,948] WARN Received rollback for vbucket 346 to seqno 181795 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,948] WARN Received rollback for vbucket 347 to seqno 186086 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,949] WARN Received rollback for vbucket 348 to seqno 185535 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,949] WARN Received rollback for vbucket 349 to seqno 185968 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,950] WARN Received rollback for vbucket 350 to seqno 184557 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,950] WARN Received rollback for vbucket 351 to seqno 181226 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,951] WARN Received rollback for vbucket 352 to seqno 186283 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,951] WARN Received rollback for vbucket 353 to seqno 185233 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,951] WARN Received rollback for vbucket 354 to seqno 183598 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,952] WARN Received rollback for vbucket 355 to seqno 185833 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,952] WARN Received rollback for vbucket 356 to seqno 185027 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,953] WARN Received rollback for vbucket 357 to seqno 185709 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,953] WARN Received rollback for vbucket 358 to seqno 184611 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,953] WARN Received rollback for vbucket 359 to seqno 214112 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,954] WARN Received rollback for vbucket 360 to seqno 213769 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,954] WARN Received rollback for vbucket 361 to seqno 212166 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,955] WARN Received rollback for vbucket 362 to seqno 181507 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,955] WARN Received rollback for vbucket 363 to seqno 182636 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,955] WARN Received rollback for vbucket 364 to seqno 181643 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,956] WARN Received rollback for vbucket 365 to seqno 185530 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,956] WARN Received rollback for vbucket 366 to seqno 184469 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,956] WARN Received rollback for vbucket 367 to seqno 182833 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,957] WARN Received rollback for vbucket 368 to seqno 184627 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,957] WARN Received rollback for vbucket 369 to seqno 182510 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,958] WARN Received rollback for vbucket 370 to seqno 185868 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,958] WARN Received rollback for vbucket 393 to seqno 1300 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,959] WARN Received rollback for vbucket 394 to seqno 542 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,959] WARN Received rollback for vbucket 395 to seqno 8869 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,959] WARN Received rollback for vbucket 396 to seqno 0 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,960] WARN Received rollback for vbucket 397 to seqno 0 (com.couchbase.client.dcp.Client:160)
[2018-09-03 12:51:23,960] WARN Received rollback for vbucket 398 to seqno 0 (com.couchbase.client.dcp.Client:160)
So this happened when we tried to increase connector tasks. The procedure I followed was:
- Pause the connector and ensure all tasks are in paused state
- Updated connector configuration with increased tasks.max
- Resume the connector
I went through the thread and issue regarding Connector stops streaming after rollback. We don’t mind if we stream the events again (duplicate in kafka). Looks like you have fixed this in recent release. But unfortunately we can’t use later versions as our couchbase cluster is old(we discussed this before).
I am hoping you can shed some light on scenarios where rollback will happen.