Advise to reprocess a specific list of documents

Kafka Connector
7

HI @yeikel,

Is your goal to re-trigger processing the latest version of the document in Couchbase? If so, perhaps you could use a Couchbase SDK (for the language of your choice!) to “tickle” the document you want to reprocess.

Here’s what that might look like with the Couchbase Java SDK:

/**
 * Modifies an extended attribute (XATTR) on the document.
 * This is sufficient to trigger a DCP mutation event.
 * 
 * @throws DocumentNotFoundException if document does not exist
 */
public static void tickle(
  Collection collection,
  String documentId
) {
  collection.mutateIn(
    documentId,
    List.of(MutateInSpec.increment("tickle", 1).xattr()),
    MutateInOptions.mutateInOptions()
      .preserveExpiry(true)
  );
}

CAVEAT: If the document was removed, there is no way to reprocess it using this strategy.

Out of curiosity, what requirements guided your decision to use the Couchbase Kafka connector instead of the Couchbase Elasticsearch connector?

Thanks,
David

1 Like