With the release of Couchbase 6.6 the Eventing Service has major improvements in functionality.
We’ve introduced new Eventing Timers that can be cancelled using the cancelTimer() function, or by creating a new timer with the same reference identifier as an existing timer. Recurring timers are also fully supported, and can easily be used to create repetitive logic by using a timer callback to create fresh timers. Timer scheduling enables timers to be created for days, weeks, or years in the future with no adverse performance impact. The OnDelete Handler now indicates if a document was deleted or expired using the new “options” parameter. Key Eventing statistics in the UI are now co-located with each Functions lifecycle control.
Together these enhancements simplify the effort and code required to make robust business logic.
Prerequisites
In this article we will be introducing key Eventing improvements added to the latest GA release, i.e. Couchbase version 6.6.0, and for each item we provide a working basic example. However, please understand that none of the Eventing Functions provided in this article will work “as-is” in earlier versions of the Couchbase server without significant changes and complex work arounds.
If you are not familiar with Couchbase or the Eventing service please walk through GET STARTED and one Eventing example specifically refer to the following:
- Setup a working Couchbase 6.6.0 server as per the directions in Start Here!
- Understand both basic Eventing concepts and how to deploy a basic Eventing Function as per the directions in the Document Archival example.
Eventing Timers can now be Cancelled
With the addition of the cancelTimer() function, or by creating a new Eventing Timer with the same reference identifier as an existing timer active Timers that have yet to fire can be canceled. This enhancement simplifies the code required to make robust business logic.
Developers no longer are forced to add fields and logic and do additional checking to ensure a Timer being fired is not “stale” and superseded by a newer Timer.
Example:
- Create a “source” bucket and a “metadata” bucket for Eventing.
- Deploy this function (code below).
- Create a document in the source bucket with:
1KEY user_scoreboard::1 and DATA {"type": "user_scoreboard", "id": 1} - Inspect the logs after a minute.
- Delete the document with key “user_scoreboard::1”.
- Inspect the logs after a minute.
- Undeploy this function.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
function UserInactivityCallback(context) { log('user was inactive for 10 minutes', context.docId); /* * take some actions ... */ } function OnUpdate(doc, meta) { if (doc.type != 'user_scoreboard') return; // Create a timestamp 600 seconds from now var tenMinutesFromNow = new Date(); // Get current time & add 600 sec. to it. tenMinutesFromNow.setSeconds(tenMinutesFromNow.getSeconds() + 600); // Every mutation to the user's scoreboard will result in a new Timer but // the old timers will be canceled as they have the same identifier, meta.id. createTimer(UserInactivityCallback, tenMinutesFromNow, meta.id, { "docId": meta.id }); } function OnDelete(meta, options) { if (!(meta.id.startsWith("user_scoreboard:"))) return; log('user scoreboard was deleted for', meta.id); // remove the timer as there is no related document anymore. cancelTimer(UserInactivityCallback,meta.id); /* * take some actions ... */ } |
Recurring Eventing Timers are now fully supported
Recurring Timers are fully supported, i.e. a function that is invoked by a timer callback can reliably create fresh Timers. This update allows a single Eventing Function to reliably implement complex recurring events creating new timers from within the callback of another timer.
Previously a co-Function was required to reliably create a recurring series of Timers (left image), the 6.6 release simplifies the code needed to implement recurring (or scheduled) business logic (right image).
Example:
- Create a “source” bucket and a “metadata” bucket for Eventing.
- Make an Bucket alias in the Function’s setting as “src_bkt” to the source bucket in mode read+write.
- Deploy this function (code below).
- Create a document in the source bucket with:
1KEY recurring_timer::1 and DATA {"type": "recurring_timer", "id": 1, "active": true} - Inspect the logs after a minute.
- Inspect the logs after several minutes.
- Alter the document with key “recurring_timer::1” and change the field “active” to false as follows:
-
1KEY recurring_timer::1 and DATA {"type": "recurring_timer", "id": 1, "active": false}
- Inspect the logs after a minute.
- Undeploy this function.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
function CreateRecurringTimer(context) { log('From CreateRecurringTimer: creating timer', context.mode, context.id); // Create a timestamp 30 seconds from now var thirtySecFromNow = new Date(); // Get current time & add 30 sec. to it. thirtySecFromNow.setSeconds(thirtySecFromNow.getSeconds() + 30); // Create a document to use as out for our context createTimer(RecurringTimerCallback, thirtySecFromNow, context.id, context); } function RecurringTimerCallback(context) { log('From RecurringTimerCallback: timer fired', context); /* * do any sort of recurring work here, just update a date_stamp in a document * written back to the source bucket (but with the ID prefixed whit "cur_") */ src_bkt["cur_" + context.id] = { "last_update": new Date() }; // rearm the timer CreateRecurringTimer({ "id": context.id, "mode": "via_callback" }) } function OnUpdate(doc, meta) { // You would typically filter to mutations of interest if (doc.type !== 'recurring_timer') return; if (doc.active === false) { if (cancelTimer(RecurringTimerCallback, meta.id)) { log('From OnUpdate: canceled active Timer, doc.active', doc.active, meta.id); } else { log('From OnUpdate: no active Timer to cancel, doc.active', doc.active, meta.id); } } else { log('From OnUpdate: create/overwrite doc.active', doc.active, meta.id); CreateRecurringTimer({ "id": meta.id, "mode": "via_onupdate" }); } } |
Timers can be created days/weeks/years in the future
One timer or a million timers can be created with no adverse performance impact on an otherwise idle Eventing system. This ability opens up use cases for long term notifications and customer re-engagement programs.
In the 6.5.X releases creating a few thousand Timers in the future (as in one hour+) in an otherwise idle system resulted in a growing number of metadata bucket operations which impacted performance and could eventually block mutations for a the given Eventing Function.[1]
Note on a cancelTimer() operation or overwriting an existing Timer by reference there will be a temporary documents in the Eventing “metadata” bucket, eventually these documents will be purged.[2]
Example:
Demonstrate the creation of a lot of Eventing Timers (in this case 50,000) and schedule them well into the future (by 96 hours) and then cancel the entire set (or allowed them all to fire the callback routine TimerCallback).
- Create a “source” bucket and a “metadata” bucket for Eventing.
- Deploy this function (code below).
- Create a document in the source bucket with any DATA as we only look at.
1KEY spawn_50k_timers::1 - Inspect the logs after a minute or two.
- Delete the document with key “spawn_50k_timers::1”.
- Inspect the logs after a minute or two.
- Undeploy this function.
To see the 50K Eventing Timers being created and firing (instead of being canceled):
- Edit the function change delayMinutes to 1 (you do not want to wait 4 days).
- Deploy the modified function (code below).
- Create a document in the source bucket with any DATA as we only look at the KEY.
1KEY spawn_50k_timers::1 - Inspect the logs after two minutes.
- Delete the document with key “spawn_50k_timers::1”.
- Inspect the logs after a minute or two.
- Undeploy this function.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
function TimerCallback(context) { if (context.timer_num == 1 || context.timer_num == 50000) log('TimerCallback: fired timer #', context.timer_num); /* * take some actions ... */ } function OnUpdate(doc, meta) { if (meta.id != 'spawn_50k_timers::1') return; // Create a timestamp 96 hours from now (4 days) var delayMinutes = 60*96; var futureTime = new Date(); futureTime.setMinutes(futureTime.getMinutes() + delayMinutes); // make 50K timers for (var i=1; i<= 50000; i++) { var timer_id = "tmr_" + i; if (i == 1 || i == 50000) log('OnUpdate: create timer # ', i, timer_id); try { createTimer(TimerCallback, futureTime, timer_id, { "docId": meta.id, "timer_num": i}); } catch (e) { log('OnUpdate: create #', i, " error ", e); } } } function OnDelete(meta, options) { if (meta.id != 'spawn_50k_timers::1') return; /* * Remove the 50K timer as there is no related control document anymore. * * Note on a cancel or overwrite there will be documents which will be * cleaned up at a) initial firing schedule of the Timers, b) undeployment * of the function, or c) a lazy cleaning process by the Eventing Service. */ for (var i=1; i<= 50000; i++) { var timer_id = "tmr_" + i; if (i == 1 || i == 50000) log('OnDelete: clear timer # ', i, timer_id); try { cancelTimer(TimerCallback, timer_id); } catch (e) { log('OnDelete: clear timer #', i, " error ", e); } } } |
Eventing Statistics are now co-located with Lifecycle Controls
Four (4) key Eventing statistics in the UI are now co-located with each Functions Lifecycle control. These enhancements simplify the effort and code and diagnose Eventing Functions to quickly make robust business logic.
The developer or administrator can immediately determine that a Function is misbehaving without having to navigate away from the Function’s lifecycle controls.
Example:
- Create a “source” bucket and a “metadata” bucket for Eventing.
- Make a few document any KEY and any DATA in the source bucket.
- Deploy the modified function (code below) it contains a syntax error, forgetting to put var in front of a variable.
|
1 2 3 4 5 |
function OnUpdate(doc, meta) { log('docId', meta.id); // this is an error missing 'var' a = 2; } |
- When the Eventing Function is deployed there is an immediate feedback that something went wrong to a developer.
- The user responds to the feed back, i.e. “red” failures and can inspect the Function for the source of the error.
- Undeploy this Eventing Function.
- Fix the error in this case make line 4: “var a=2;”.
- Deploy this Eventing Function again.
- Once the code is fixed the developer can see the correct behavior and progress as success counter now increments.
OnDelete Handler now indicates Deletion or Expiration
The OnDelete Handler now indicates if a document was deleted or expired via a new parameter “options”. This often requested capability allows different logic to be executed depending on the type of removal.
Example:
- Create a “source” bucket and a “metadata” bucket for Eventing
- Deploy this function (code below)
- Create a document in the source bucket with any DATA as we only look at the KEY
1KEY doc_to_delete::1 - Inspect the logs after a few seconds
- Delete the document with key “doc_to_delete::1”
- Inspect the logs after a minute
- Undeploy this function
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
function OnDelete(meta, options) { if (options.expired) { log("doc expired:",meta.id); /* * take some action ... */ } else { log("doc deleted:",meta.id); /* * take some other action ... */ } } |
Explore Couchbase Server 6.6 resources
References
- Couchbase Eventing documentation:
https://docs.couchbase.com/server/current/eventing/eventing-overview.html - Couchbase Eventing examples:
https://docs.couchbase.com/server/current/eventing/eventing-examples.html - Couchbase Server 6.6 What’s New:
https://docs.couchbase.com/server/6.6/introduction/whats-new.html - Couchbase blogs on Eventing:
https://www.couchbase.com/blog/tag/eventing/
We would love to hear from you on how you liked the 6.6 features and how it’ll benefit your business going forward. Please share your feedback via the comments or in the Couchbase forum.
Footnotes
[1] The severity is governed by: a) The number of vBuckets holding an active timer. Therefore if there are only a few timers in the future the issue may not be noticeable or materialize. and b) Whether an Eventing timer has fired recently on a vBucket (which clears the issue for the given vBucket on a per function basis). Therefore 6.5 systems with lots of near term timer activity will not experience this issue even if timers are scheduled far into the future. This is not an issue in 6.6.
[2] On a cancelTimer() operation or overwriting an existing Timer by reference there will be a temporary documents in the Eventing “metadata” bucket which will be cleaned up at a) at the initial firing schedule of the canceled (or overwritten) Timers, or b) an undeployment of the function. This behavior is unlike a Timer firing at the scheduled time in which all associated Eventing metadata is immediately cleaned up form the Eventing “metadata” bucket upon a Timer firing.
