Handling errors

Two types of errors are raised in Pipedream workflows:

Workflow errors — Errors in the workflow execution environment, like Timeouts or Out of Memory errors. Often, you can change your workflow's configuration to fix them. You can find more details on these errors in our troubleshooting guide.
Step errors — Errors raised by individual code or action steps. These can be syntax errors, errors raised by the Node or Python runtime, errors with input data, and more. Pipedream will surface details about the error and the stack trace, and you can even debug these errors with AI.

Both types of errors will trigger error notifications, can be handled by custom error handlers, and will show up in the REST API.

Auto-retry

You can automatically retry events that yield an error. This can help for transient errors that occur when making API requests, like when a service is down or your request times out.

Apply conditional logic

Many errors result from the data you're processing. You might only receive certain data from a webhook under certain conditions, or have malformed data in the payload that causes an error.

You can apply conditional logic in code, or using the If / Else operator, handling these conditions accordingly.

Error notifications

By default, Pipedream sends an email when a workflow throws an unhandled error. But you can

Send error notifications to Slack
Handle errors from one workflow in a specific way
Fetch errors asynchronously using the REST API, instead of handling the event in real-time

These docs describe the default error behavior, and how to handle custom use cases like these.

Before you jump into the examples below, remember that all Pipedream workflows are just code. You can always use the built-in error handling logic native to your programming language, for example: using JavaScript's try / catch statement. In the future, Pipedream plans to support this kind of error-handling for built-in actions, as well.

Default system emails

Any time your workflow throws an unhandled error, you'll receive an email like this:

Example error email

This email includes a link to the error so you can see the data and logs associated with the run. When you inspect the data in the Pipedream UI, you'll see details on the error below the step that threw the error, e.g. the full stack trace.

Duplicate errors do not trigger duplicate emails

High-volume errors can lead to lots of notifications, so Pipedream only sends at most one email, per error, per workflow, per 24 hour period.

For example, if your workflow throws a TypeError, we'll send you an email, but if it continues to throw that same TypeError, we won't email you about the duplicate errors until the next day. If a different workflow throws a TypeError, you will receive an email about that.

Test mode vs. live mode

When you're editing and testing your workflow, any unhandled errors will not raise errors as emails, nor are they forwarded to error listeners. Error notifications are only sent when a deployed workflow encounters an error on a live event.

Debug with AI

You can debug errors in code or action steps with AI by pressing the Debug with AI button at the bottom of any error.

Data we send with errors

When you debug an error with AI, Pipedream sends the following information to OpenAI:

The error code, message, and stack trace
The step's code
The input added to the step configuration. This does not contain the event data that triggered your workflow, just the static input entered in the step configuration, like the URL of an HTTP request, or the names of step exports.

We explicitly do not send the event data that triggered the error, or any other information about your account or workflow.

Handle errors with custom logic

Pipedream exposes a global stream of all errors, raised from all workflows. You can subscribe to this stream, triggering a workflow on every event. This lets you handle errors in a custom way. Instead of sending all errors to email, you can send them to Slack, Discord, AWS, or any other service, and handle them in any custom way.

To do this:

Create a new workflow.
Add a new trigger. Search for the Pipedream app.
Select the custom source Workspace $error events.
Generate an error in a live version of any workflow (errors raised while you're testing your workflow do not send errors to the $errors stream). You should see this error trigger the workflow in step #1. From there, you can build any logic you want to handle errors across workflows.

Duplicate errors do trigger duplicate error events on custom workflows

Unlike the default system emails, duplicate errors are sent to any workflow listeners.

Poll the REST API for workflow errors

Pipedream provides a REST API endpoint to list the most recent 100 workflow errors for any given workflow. For example, to list the errors from workflow p_abc123, run:

curl 'https://api.pipedream.com/v1/workflows/p_abc123/$errors/event_summaries?expand=event' \
  -H 'Authorization: Bearer <api_key>'

By including the expand=event query string param, Pipedream will return the full error data, along with the original event that triggered your workflow:

{
  "page_info": {
    "total_count": 100,
    "start_cursor": "1606370816223-0",
    "end_cursor": "1606370816223-0",
    "count": 1
  },
  "data": [
    {
      "id": "1606370816223-0",
      "indexed_at_ms": 1606370816223,
      "event": {
        "original_event": {
          "name": "Luke",
          "title": "Jedi"
        },
        "original_context": {
          "id": "1kodJIW7jVnKfvB2yp1OoPrtbFk",
          "ts": "2020-11-26T06:06:44.652Z",
          "workflow_id": "p_abc123",
          "deployment_id": "d_abc123",
          "source_type": "SDK",
          "verified": false,
          "owner_id": "u_abc123",
          "platform_version": "3.1.20"
        },
        "error": {
          "code": "InternalFailure",
          "cellId": "c_abc123",
          "ts": "2020-11-26T06:06:56.077Z",
          "stack": "    at Request.extractError ..."
        }
      },
      "metadata": {
        "emitter_id": "p_abc123",
        "emit_id": "1kodKnAdWGeJyhqYbqyW6lEXVAo",
        "name": "$errors"
      }
    }
  ]
}

By listing these errors, you may be able to replay them against your workflow programmatically. For example, if your workflow is triggered by HTTP requests, you can send an HTTP request with the data found in event.original_event (see the example above) for every event that errored.

Inspect events Concurrency and Throttling