Troubleshooting Common Issues

This doc describes some common solutions for fixing issues with pipedream.com or with a specific workflow.

A feature isn’t working on pipedream.com

If you’re seeing an issue with pipedream.com (for example, the site won’t load, or you think you’ve found a bug), try each of the following steps, checking to see if they fix the problem:

1. Hard refresh pipedream.com in your browser.

2. Log out of your pipedream.com account, and log back in.

3. Disable your browser extensions or use features like Chrome Guest mode to browse pipedream.com without any existing extensions / cookies / cache.

If you’re still seeing the issue after trying these steps, please report a bug.

How do I contact Pipedream Support?

Start by filling out the request form at https://pipedream.com/support, providing detailed information about your issue.

How do I share my workflow with Support?

First, navigate to your Project Settings and share your project with Pipedream Support.

When filling out the request form at https://pipedream.com/support, please provide detailed information along with the URL from your browser’s address bar, which should look something like:

https://pipedream.com/@yourworkspace/projects/proj_abc123/test-workflow-p_abc123/inspect

Workflows

Where do I find my workflow’s ID?

Open https://pipedream.com and visit your workflow. Copy the URL that appears in your browser’s address bar. For example:

https://pipedream.com/@yourworkspace/projects/proj_abc123/test-workflow-p_abc123/inspect

Your workflow’s ID is the value that starts with p_. In this example: p_abc123.

My workflow isn’t working

If you’re encountering a specific issue in a workflow, try the following steps, checking to see if they fix the problem:

1. Make a trivial change to your workflow, and Deploy your workflow again.

2. Try searching the community or the pipedream GitHub repo to see if anyone else has solved the same issue.

If you’re still seeing the issue after trying these steps, please reach out in the community.

Error in workflow

If you see a generic Error in workflow when invoking a webhook-triggered workflow, follow these steps to resolve the issue:

1. Check if your trigger is configured to return a custom HTTP response.
2. Confirm that your workflow is returning an HTTP response in every situation, e.g., by using $.respond().

How do I invoke another workflow?

We provide a Trigger Workflow action in the Helper Functions app. See more here.

Another option is to make an HTTP request to a Pipedream HTTP webhook trigger.

Triggers

Why is my trigger not saving?

If your trigger continuously spins without saving, it might be processing too much data at once or taking longer than expected. This issue often occurs with polling database-related triggers (e.g., PostgreSQL, Snowflake, Notion Databases). To resolve it, try reducing the volume of data fetched, e.g., by limiting the number of rows returned in your query.

Why is my trigger not emitting events?

First, look at your trigger logs and check for any errors there. Verify that it has been running, whether on a new webhook event or at the configured polling interval.

If your trigger operates on a large amount of data at once, it may fail to log the execution and you won’t see any new events and logs. Try polling more frequently or limiting the number of records being fetched in the API request or Database query.

Webhook-based instant sources

These sources will get triggered immediately. But because events come in in real-time, most will not automatically fetch historical events upon creation. To surface test events in your workflow while building, you’ll need to generate an eligible event in the selected app.

For example, if you’ve configured the “Message Updates (Instant) Telegram source, you’ll need to send a message in the Telegram account you’ve selected in order for an event to appear.

Sources for apps like Telegram and Google Sheets use webhooks and get triggered immediately.

Timer-based polling sources

These sources will fetch new events on a regular interval, based on a schedule you specify in the trigger configuration.

In most cases, Pipedream will automatically fetch recent historical events to help enable easier workflow development. Sources for apps like Twitter and Spotify require we poll their endpoints in order to fetch new events.

Where do I find my event source’s ID?

Open https://pipedream.com/sources and click on your event source. Copy the URL that appears in your browser’s address bar. For example:

https://pipedream.com/sources/dc_abc123

Your source’s ID is the value that starts with dc_. In this example: dc_abc123.

Where can I find the trigger logs?

Find your source, then click on the logs or visit this URL:

https://pipedream.com/sources/dc_abc123/logs

Why is my trigger paused?

Pipedream automatically disables sources with a 100% error rate in the past 5 days for accounts on the Free plan.

To troubleshoot, you can look at the errors in the source logs, and may need to reconnect your account and re-enable the source for it to run again. If the issue persists, please reach out in the community.

Warnings

Pipedream displays warnings below steps in certain conditions. These warnings do not stop the execution of your workflow, but can signal an issue you should be aware of.

Code was still running when the step ended

This error occurs when Promises or asynchronous code is not properly finished before the next step begins execution.

See the Asynchronous section of the Node.js documentation for more details.

Undeployed changes — You have made changes to this workflow. Deploy the latest version from the editor

On workflows that are not synced with GitHub, you may notice the following warning at the top of your workflow:

Undeployed changes — You have made changes to this workflow. Deploy the latest version from the editor

This means that you’ve made some changes to your workflow that you haven’t yet deployed. To see a diff of what’s changed, we recommend enabling GitHub sync, where you’ll get a full commit history of changes made to your workflows, synced to your own GitHub repo.

Errors

Limit Exceeded Errors

Pipedream sets limits on runtime, memory, and other execution-related properties. If you exceed these limits, you’ll receive one of the errors below. See the limits doc for details on specific limits.

Quota Exceeded

On the Free tier, Pipedream imposes a limit on the daily credits across all workflows and sources. If you hit this limit, you’ll see a Quota Exceeded error.

Paid plans have no credit limit. Upgrade here.

Runtime Quota Exceeded

You do not use credits testing workflows, but workspaces on the Free plan are limited to 30 minutes of test runtime per day. If you exceed this limit when testing in the builder, you’ll see a Runtime Quota Exceeded error.

Timeout

Event sources and workflows have a default time limit on a given execution. If your code exceeds that limit, you may encounter a Timeout error.

To address timeouts, you’ll either need to:

1. Figure out why your code is running for longer than expected. It’s important to note that timeouts are not an issue with Pipedream — they are specific to your workflow. Often, you’re making a request to a third party API that doesn’t respond in the time you expect, or you’re processing a large amount of data in your workflow, and it doesn’t complete before you hit the execution limit.
2. If it’s expected that your code is taking a long time to run, you can raise the execution limit of a workflow in your workflow’s settings. If you need to change the execution limit for an event source, please reach out to our team.

Out of Memory

Pipedream limits the default memory available to workflows and event sources. If you exceed this memory, you’ll see an Out of Memory error. You can raise the memory of your workflow in your workflow’s Settings.

This can happen for two main reasons:

1. Even for small files or objects, avoid loading the entire content into memory (e.g., by saving it in a variable). Always stream files to/from disk to prevent potential memory leaks, using a technique like this.
2. When you have many steps in your Pipedream workflow. When your workflow runs, Pipedream runs a separate process for each step in your workflow. That incurs some memory overhead. Typically this happens when you have more than 8-10 steps. When you see an OOM error on a workflow with many steps, try increasing the memory.

Rate Limit Exceeded

Pipedream limits the number of events that can be processed by a given interface (e.g. HTTP endpoints) during a given interval. This limit is most commonly reached for HTTP interfaces - see the QPS limits documentation for more information on that limit.

This limit can be raised for HTTP endpoints. Reach out to our team to request an increase.

Request Entity Too Large

By default, Pipedream limits the size of incoming HTTP payloads. If you exceed this limit, you’ll see a Request Entity Too Large error.

Pipedream supports two different ways to bypass this limit. Both of these interfaces support uploading data up to 5TB, though you may encounter other platform limits.

• You can send large HTTP payloads by passing the pipedream_upload_body=1 query string or an x-pd-upload-body: 1 HTTP header in your HTTP request. Read more here.
• You can upload multiple large files, like images and videos, using the large file upload interface.

Function Payload Limit Exceeded

The total size of console.log() statements, step exports, and the original event data sent to workflows and sources cannot exceed a combined size of 6MB. If you produce logs or step exports larger than this - for example, passing around large API responses, CSVs, or other data - you may encounter a Function Payload Limit Exceeded in your workflow.

Often, this occurs when you pass large data between steps using step exports. You can avoid this error by writing that data to the /tmp directory in one step, and reading the data into another step, which avoids the use of step exports and should keep you under the payload limit.

Pipedream also compresses the function payload from your workflow, which can yield roughly a 2x-3x increase in payload size (somewhere between 12MB and 18MB), depending on the data.

JSON Nested Property Limit Exceeded

Working with nested JavaScript objects that have more than 256 nested objects will trigger a JSON Nested Property Limit Exceeded error.

Often, objects with this many nested objects result from a programming error that explodes the object in an unexpected way. Please confirm the code you’re using to convert data into an object is correctly parsing the object.

Event Queue Full

Workflows have a maximum event queue size when using concurrency and throttling controls. If the number of unprocessed events exceeds the maximum queue size, you may encounter an Event Queue Full error.

Paid plans can increase their queue size up to 10,000 for a given workflow.

Credit Budget Exceeded

Credit Budgets are configurable limits on your credit usage at the account or workspace level.

If you’re receiving this warning on a source or workflow, this means your allocated Credit Budget has been reached for the defined period.

You can increase this limit at any time in the billing area of your settings.

Pipedream Internal Error

A Pipedream Internal Error is thrown whenever there’s an exception during the building or executing of a workflow that’s outside the scope of the code for the individual components (steps or actions).

There are a few known ways this can be caused and how to solve them.

Out of date actions or sources

Pipedream components are updated continously. But when new versions of actions and sources are published to the Pipedream Component Registry, your workflows are not updated by default.

An Update prompt is shown in the in the top right of the action if the component has a new version available.

Sources do not feature an update button at this time, to receive the latest version, you’ll need to create a new source, then attach it to your workflow.

New package versions issues

If an NPM or PyPI package throws an error during either the building of the workflow or during it’s execution, it may cause a Pipedream Internal Error.

By default, Pipedream automatically updates NPM and PyPI packages to the latest version available. This is designed to make sure your workflows receive the latest package updates automatically.

However, if a new package version includes bugs, or changes it’s export signature, then this may cause a Pipedream Internal Error.

You can potentially fix this issue by downgrading packages by pinning in your Node.js or Python code steps to the last known working version.

Alternatively, if the error is due to a major release that changes the import signature of a package, then modifying your code to match the signature may help.

Packages consuming all available storage

A Pipedream Internal Error could be the result of NPM or PyPI packages using the entireity of the workflow’s storage capacity.

The lodash library for example will import the entire package if individual modules are imported with this type of signature:

// This style of import will cause the entire lodash package to be installed, not just the pick module
import { pick } from "lodash"

Instead, use the specific package that exports the pick module alone:

// This style imports only the pick module, since the lodash.pick package only contains this module
import pick from "lodash.pick"

Is there a way to replay workflow events programmatically?

Not via the API, but you can bulk select and replay failed events using the Event History.

How do I store and retrieve data across workflow executions?

If you operate your own database or data store, you can connect to it directly in Pipedream.

Pipedream also operates a built-in key-value store that you can use to get and set data across workflow executions and different workflows.

How do I delay the execution of a workflow?

Use Pipedream’s built-in Delay actions to delay a workflow at any step.

How can my workflow run faster?

Here are a few things that can help your workflow execute faster:

1. Increase memory: Increase your workflow memory to at least 512 MB. Raising the memory limit will proportionally increase CPU resources, leading to improved performance and reduced latency.

2. Return static HTTP responses: If your workflow is triggered by an HTTP source, return a static HTTP response directly from the trigger configuration. This ensures the HTTP response is sent to the caller immediately, before the rest of the workflow steps are executed.

3. Simplify your workflow: Reduce the number of steps and segments in your workflow, combining multiple steps into one, if possible. This lowers the overhead involved in managing step execution and exports.

4. Activate warm workers: Use warm workers to reduce the startup time of workflows. Set as many warm workers as you want for high-volume traffic.

How can I save common functions as steps?

You can create your own custom triggers and actions (“components”) on Pipedream using the Component API. These components are private to your account and can be used in any workflow.

You can also publish common functions in your own package on a public registry like npm or PyPI.

Is Puppeteer supported in Pipedream?

Yes, see our Puppeteer docs for more detail.

Is Playwright supported in Pipedream?

Yes, see our Playwright docs for more detail.