{"id":12471,"date":"2021-11-15T10:34:56","date_gmt":"2021-11-15T18:34:56","guid":{"rendered":"https:\/\/www.couchbase.com\/blog\/?p=12471"},"modified":"2025-06-13T22:39:19","modified_gmt":"2025-06-14T05:39:19","slug":"couchbase-eventing-handling-errors-and-retries","status":"publish","type":"post","link":"https:\/\/www.couchbase.com\/blog\/couchbase-eventing-handling-errors-and-retries\/","title":{"rendered":"Couchbase Eventing Handling Errors and Retries"},"content":{"rendered":"

The <\/span>Couchbase Eventing Service<\/span><\/a> allows you to promptly act on mutations (or changes) to your data. All actions in Eventing<\/a> are accomplished by executing a lambda, a small piece of business logic written in JavaScript.<\/span><\/p>\n

Common use cases include data enrichment, document archiving, and integration with external REST services. See more details <\/span>here<\/span><\/a>.<\/span><\/p>\n

In the following blog, we will outline how errors during the event listener execution can be handled. By using a retry mechanism we ensure that the anticipated action is performed, even if the event listener fails during execution.<\/span><\/p>\n

Sample application<\/i><\/b><\/p>\n

As an example, we implement part of an e-commerce application that stores the customer orders in a Couchbase collection. As soon as the status of an order is changed to <\/span>paid<\/b>, we want to send an order confirmation to the customer.<\/span><\/p>\n

To accomplish this we integrate a Couchbase event listener with an external <\/span>email service<\/span><\/i>. The Couchbase event listener will pick up any changes to the order document, verify that the order was paid and then call out to the email service to trigger the confirmation message.<\/span><\/p>\n

The email service is a standalone microservice providing a REST endpoint. We use the cURL support built directly into Couchbase Eventing to call the microservice from the event listener.<\/span><\/p>\n

\"\"<\/p>\n

Conceptual flow: As the orders are updated in Couchbase, events are triggered and picked up by an event listener. The event listener then calls out to the external Email Service.<\/span><\/i><\/p>\n

\u00a0<\/span>Email service returning an error<\/b><\/p>\n

The described scenario works very well if the <\/span>email service<\/span><\/i> is operational. However, what happens if the <\/span>email service<\/span><\/i> is returning an error? Requests from the event listener to the email service will fail and therefore no confirmation message is sent to the customer. Since at the time of failure the Couchbase document change event has already been processed, no new event for the same document is triggered unless there is another change to it. In order to ensure that the confirmation is sent we need to handle the error and implement a retry mechanism. By doing this we can work around any temporary issue of the external service and at the same time guarantee that the confirmation is sent.<\/span><\/p>\n

\u00a0<\/span>There are different ways to approach this. In my example below I choose to create a new collection called \u2018retry\u2019, that will store references to the documents for which the event listener execution failed.<\/span><\/p>\n

\"\"<\/p>\n

\u00a0<\/span>The event listener will pick up changes to the order documents (step #1) and then call the Email Service (step #2). If the call to the email service succeeds the event listener then updates the confirmation message status in the order document (step #3). However, in case of failure a retry document is created and put into the \u2018retry\u2019 collection (step #3*).<\/span><\/p>\n

Keeping a reference to the documents allows us to identify all failed updates and enables us to rerun them later. This could be either by manual intervention by an operator or automatic retry using Couchbase eventing timers.<\/span><\/p>\n

\"\"<\/p>\n

    \n
  1. We initiate the retry process by adding a document with a specified document id into the retry collection. A recurring timer is created given a provided timer interval.<\/span><\/li>\n
  2. On timer execution all documents older than a small time quanta in the <\/span>retry<\/span><\/i> collection are updated. By adding an attribute like <\/span>fireRetry = true<\/span><\/i> to the retry documents we trigger another update event that is picked up by the event listener to execute the retry mechanism. This gives us a recursive mutation that lights up all the documents in the retry collection in parallel. The retry function is now executed using all available worker threads in parallel.<\/span><\/li>\n
  3. A document update event is triggered for each retry document individually.<\/span><\/li>\n
  4. The corresponding order document is retrieved from the inbound collection<\/span><\/li>\n
  5. Now the email service is called.<\/span><\/li>\n
  6. If the call to the email service succeeds the event listener then updates the confirmation message status in the order document and removes the retry document\u00a0<\/span><\/li>\n
  7. In case of failure the retry document is updated and put into the \u2018retry\u2019 collection.<\/span><\/li>\n<\/ol>\n

    Code Review<\/b><\/p>\n

    \u00a0<\/span>Now that we have established the conceptual design, let\u2019s have a look at the sample implementation:<\/span><\/p>\n

    Prerequisites<\/b>:<\/span><\/p>\n