If a workflow errors, is there anything I can do to only run from the failed step; instead of from the start of the workflow?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

David McKay : If a workflow errors, is there anything I can do to only run from the failed step; instead of from the start of the workflow?

2

Dylan Sather (Pipedream) : does the step depend on step exports from prior steps? If not, could you toggle the prior steps off and replay the event that threw the error manually in this case?

It’s a little more work, but you can make your steps idempotent to ensure they aren’t impacted by duplicate runs. See https://pipedream-users.slack.com/archives/CN1BUB92B/p1599930213173200 .

3

David McKay : I’m working on making them idempotent, it’s just a lot of extra API calls to check if external requests were completed or not

4

Dylan Sather (Pipedream) : could you use this.$checkpoint to store the API response of the completed call, keyed on an event ID from event?

For example:

if (this.$checkpoint && this.$checkpoint[event.body.id]) {
  return this.$checkpoint[event.body.id]
}

// else make API call
this.resp = await myAPI()

// then store its pertinent data in this.$checkpoint
this.$checkpoint[event.body.id] = this.resp
5

David McKay : Hmm. That’s a good idea. Lemme try!

6

David McKay : It would be cool if we could mark the step as not idempotent and the NodeJS code was automatically wrapped with some memoisation

7

Dylan Sather (Pipedream) : would you want it to return the step exports from the previous run?

What identifies a run in your workflow? Some ID, or is every run guaranteed to be unique?

8

David McKay : Composite key of the values of any event., param. Or step. In the step.

9

David McKay : If the inputs to the step change, it can be rerun?

10

David McKay : Thinking aloud mostly, I’ve not given this a lot of thought yet

11

David McKay : Yeah, the this. bindings and return value should be propegated

12

David McKay : In fact, this would be cool for when conditionals/loops/branches are a Pipedream primitive

13

Dylan Sather (Pipedream) : I hear you. I can definitely see the value.