Hi @Ashish_Mishra,
Think of Eventing a a “post trigger” basically when either of the entry points OnUpdate or OnDelete is invoke the body of the document is not known.
Now there are several workarounds (basically you need to somehow store the needed state in KV). Below I will give two examples however I am not sure if the first one will work for you, but I am positive the second one will provide a solution for you.
EXAMPLE #1:
Here I assume all you care about is externalId. If you change your KEY structures in an existing deployment (this can be much more elegant), i.e. just use KEYs like anydoc:124:98765 - note the suffix is externalId
Eventing Function name: MultiKeyVersion
// A bucket binding to the source bucket aliased as 'src_bkt'
// in r+w mode is required.
function OnUpdate(doc, meta) {
// only work on KEYS that match the prefix anydoc: have :#:#
var array = meta.id.split(/(anydoc:.*:)(\d+)/);
if (array[1] === undefined || array[2] === undefined) return;
var pfx = array[1];
var old_externalId = parseInt(array[2]);
var cur_externalId = doc.externalId;
if (old_externalId === cur_externalId) {
// no change to the ID leave the doc alone
log(meta.id,"externalId did not change",old_externalId);
return;
} else {
// Here we essentially rename the document
delete src_bkt[meta.id];
src_bkt[pfx + cur_externalId] = doc;
log(meta.id,"externalId changed from",old_externalId,"to",
cur_externalId,"new meta.id",pfx + cur_externalId);
}
}
Initial data in source bucket (add it before or after you Deploy the above Eventing Function)
KEY: anydoc:124:98765
{
"id": 124,
"externalId": 98765,
"more": {
"megabytes of uninteresting data": "just emulating ...."
}
}
The log(…) message for the mutation will be emitted to the Application log
2021-06-06T08:34:29.999-07:00 [INFO] "anydoc:124:98765" "externalId did not change" 98765
Now let’s update the externalId in the document change it from 98765 to 77777 change nothing else and of course leave the Eventing Function deployed
KEY: anydoc:124:98765
{
"id": 124,
"externalId": 77777,
"more": {
"megabytes of uninteresting data": "just emulating ...."
}
}
The log(…) message for the mutation will be emitted to the Application log, note the input mutation with KEY anydoc:124:98765 was copied to a new document anydoc:124:77777 (and anydoc:124:98765 was deleted)
2021-06-06T08:42:13.734-07:00 [INFO] "anydoc:124:98765" "externalId changed from" 98765 "to" 77777 "new meta.id" "anydoc:124:77777"
Of course you design might not tolerate the KEY names updating as such I also give you Example #2
EXAMPLE #2:
Here I assume you can not or do not want to change your KEY structures in an existing production deployment as such I will give you a a bit more “heavy” general purpose pure Eventing Function (without concern to resources or data volume). Tip delete your test documents form Example #1 above.
Eventing Function name: ArchiveKeyData
// A bucket binding to an archive bucket aliases as 'arc_bkt'
// in r+w mode is required. If you only want to use the source
// bucket for the archive you would add a ".arc" suffix to meta.id
// when accessing the exposed JavaScript map 'arc_bkt'
function OnUpdate(doc, meta) {
// only work on KEYS that match the prefix
if (!meta.id.startsWith("anydoc:")) return;
var previous_key_info = arc_bkt[meta.id]
var key_info = extractKeyDataToArchive(doc);
if (!previous_key_info) {
// The archive of key information did not exist seed the first
// deployment with feed boundary Everything fill in everything
arc_bkt[meta.id] = key_info;
log(meta.id,"had no prior key_info archived");
// new doc was added (or initial run)
// *** Add any needed code here ...
return;
}
// Determine subset from extractKeyDataToArchive has changed
var changed = false;
if (crc64(previous_key_info) != crc64(key_info)) {
changed = true;
// since the key_info change update the archive version
arc_bkt[meta.id] = key_info;
}
// *** Add any needed code here ...
// FYI in your use changed is true only on a externalId change
log(meta.id,"key_info changed:",changed,"current", key_info,
"previous",previous_key_info);
}
function extractKeyDataToArchive (doc) {
// *** Adjust the data you want to achive
// Note "return doc;"" would archive the entire document
return { "externalId": doc.externalId };
}
Initial data in source bucket (add it before or after you Deploy the above Eventing Function)
KEY: anydoc:124
{
"id": 124,
"externalId": 98765,
"more": {
"megabytes of uninteresting data": "just emulating ...."
}
}
Created data in archive bucket (alias of arc_bkt) after the Function is deployment - this operated on all documents in the source bucket since I didn’t add a filter for a KEY prefix or a doc.type
KEY: anydoc:124
{
"externalId": 98765
}
The log(…) message for the mutation will be emitted to the Application log
2021-06-06T07:46:14.934-07:00 [INFO] "anydoc:124" "had no prior key_info archived"
Now let’s update the externalId change it from 98765 to 77777 change nothing else (in the source bucket) and of course leave the Eventing Function deployed
{
"id": 124,
"externalId": 77777,
"more": {
"megabytes of uninteresting data": "just emulating ...."
}
}
The archive bucket will now have
KEY: anydoc:124
{
"externalId": 77777
}
The log(…) message for the mutation will be emitted to the Application log note the Eventing Function had access to the prior key_info as per the log message and it knows the key_data (in your case externalId) has changed.
2021-06-06T07:49:28.652-07:00 [INFO] "anydoc:124" "key_info changed:" true "current" {"externalId":77777} "previous" {"externalId":98765}
Now let’s update anything OTHER THAN externalId (in the source bucket) and of course leave the Eventing Function deployed - I added another field
{
"id": 124,
"externalId": 77777,
"more": {
"megabytes of uninteresting data": "just emulating ...."
},
"more2": {
"additional megabytes of uninteresting data": "just emulating ...."
}
}
The archive bucket will be unchanged in fact it will not have been updated at all
KEY: anydoc:124
{
"externalId": 77777
}
The log(…) message for the mutation will be emitted to the Application log note the Eventing Function had access to the prior key_info as per the log message and it knows the key_data (in your case externalId) HAS NOT changed.
2021-06-06T07:52:50.051-07:00 [INFO] "anydoc:124" "key_info changed:" false "current" {"externalId":77777} "previous" {"externalId":77777}
Best
Jon Strabala
Principal Product Manager - Server