Source exceptions should trigger general error workflow (or otherwise give option to send error)

Feature Requests
1

When an uncaught exception is thrown in a source, I don’t see a way for it to trigger an error workflow (such as sending an email with the exception). This leads to a silently failing source that does not emit events.

I see that workflows offer a default setting to forward to a general error workflow:
image

Something similar for sources would be useful.

2

@eaptdudfield yes, this is a known gap and something we hope to address seamlessly in the future (e.g. with a UI toggle just like for workflows).

Currently, we expose a stream of events on the $errors channel for every source. These events have roughly the same shape as errors emitted by workflows, so you can forward these errors to your global error workflow (or a different workflow) like so:

  1. Find the workflow ID of your global error workflow. Or you can create a new workflow to handle source errors. Note its workflow ID.
  2. Create a subscription to forward errors from all sources (dc_*) to the workflow, like so:
curl -X POST "https://api.pipedream.com/v1/subscriptions?emitter_id=dc_*&event_name=\$errors&listener_id=<your workflow ID>" -H 'Authorization: Bearer XXX'
  1. Create an auto-subscription to forward errors for all future sources to the workflow, as well.

Let me know if that helps.

1 Like