Debounce workflow (instead of throttling)

Help
1

I’ve got a webhook triggered from a CMS platform that kicks off a static site build. The problem is that in some cases there can be a flurry of events - (update / delete / copy / published, etc). Is it possible to configure a workflow to wait a few seconds until the activity had stopped?

2

@sergeystoma there’s not built-in debouncing functionality, but take a look at this example workflow for a workaround. You can likely improve on the workflow and modify it for your use case!

First, Copy the workflow. Then visit your workflow’s Settings tab, just above the code. Set your Throttling and Concurrency settings like so:

Screen Shot 2021-05-03 at 4.38.46 PM
Screen Shot 2021-05-03 at 4.38.46 PM734×390 21.2 KB

This ensures the workflow runs serially, so that we can read / write a consistent value of $checkpoint, where we store the last execution time.

The check_last_execution_and_exit_early step in the workflow essentially checks the last time it executed the workflow and, if that value is less than 60 seconds, it ends the workflow early. You’re essentially receiving a large number of events, processing the first (since the previous build would have occurred more than 60 seconds ago), and throttling each subsequent one. Since subsequent events all happen one after the another, we’ll end the workflow early for each of those.

You’ll still be charged a workflow invocation for each event, but the $end() logic ensures that the core logic of your workflow — anything below this step — isn’t run.

Let me know if that works OK for you.

3

Note that you can store any JSON-serializable data in $checkpoint (see docs here), so one improvement could be to store the last execution timestamp per build, based on some unique build ID that arrives in the incoming payload:

$checkpoint[event.body.build_id] = startTs

Even this solution has its shortcomings, but you can see how it can be improved.

4

Hmm, but that is essentially still throttling, right? Since it would process the first event and ignore the rest within the time interval. Instead, I’m looking to wait until a period of time had elapsed after the last event, something like:

// javascript pseudo-code...

function workflow() {
    if ($checkpoint.timer) {
        clearTimeout($checkpoint.timer);
    }

    $checkpoint.timer = setTimeout(() => {
        // Do the processing 15 seconds after the last event.
    }, 15000);
}

// Flood of events...
workflow();
workflow();
workflow();
workflow();
workflow();
5

@sergeystoma that’s right, this solution doesn’t implement proper debouncing. We do provide a task scheduler, but support for cancelling tasks is still a work in progress. Once that’s done, you should be able to debounce events in exactly the manner you describe. Feel free to follow that issue for updates.