Concurrency and Throttling
Pipedream makes it easy to manage the concurrency and rate at which events trigger your workflow code using execution controls.
You can avoid race conditions like this by limiting workflow concurrency to a single “worker”. What this means is that only one event will be processed at a time, and the next event will not start processing until the first is complete (unprocesssed events will maintained in a queue and processed by the workflow in order). The following diagram illustrates how the events in the last diagram would executed if concurrency was limited to a single worker.
While the first example resulted in only two rows of data in Google Sheets, this time data for all three events are recorded to three separate rows.
The maximum number of events Pipedream will queue per workflow depends on your account type.
Overview
Workflows listen for events and execute as soon as they are triggered. While this behavior is expected for many use cases, there can be unintended consequences.Concurrency
Without restricting concurrency, events can be processed in parallel and there is no guarantee that they will execute in the order in which they were received. This can cause race conditions. For example, if two workflow events try to add data to Google Sheets simultaneously, they may both attempt to write data to the same row. As a result, one event can overwrite data from another event. The diagram below illustrates this example — bothEvent 1 and Event 2 attempt to write data to Google Sheets concurrently — as a result, they will both write to the same row and the data for one event will be overwritten and lost (and no error will be thrown).
Throttling
If your workflow integrates with any APIs, then you may need to limit the rate at which your workflow executes to avoid hitting rate limits from your API provider. Since event-driven workflows are stateless, you can’t manage the rate of execution from within your workflow code. Pipedream’s execution controls solve this problem by allowing you to control the maximum number of times a workflow may be invoked over a specific period of time (e.g., up to 1 event every second).Pipedream controls the frequency of workflow invocations over a specified time interval using fixed window throttling. For example, if the execution rate limit is set at 1 execution every 5 seconds, this means that your workflow will be invoked no more than once within a fixed 5-second time box. This doesn’t mean that executions will occur 5 seconds apart.
Usage
Events emitted from a source to a workflow are placed in a queue, and Pipedream triggers your workflow with events from the queue based on your concurrency and throttling settings. These settings may be customized per workflow (so the same events may be processed at different rates by different workflows).- Up to 100 events will be queued per workflow for the workspaces on the free tier.
- Workflows owned by paid plans may have custom limits. If you need a larger queue size, see here.
Where Do I Manage Concurrency and Throttling?
Concurrency and throttling can be managed in the Execution Controls section of your Workflow Settings. Event queues are currently supported for any workflow that is triggered by an event source. Event queues are not currently supported for native workflow triggers (native HTTP, cron, SDK and email).Managing Event Concurrency
Concurrency controls define how many events can be executed in parallel. To enforce serialized, in-order execution, limit concurrency to1 worker. This guarantees that each event will only be processed once the execution for the previous event is complete.
To execute events in parallel, increase the number of workers (the number of workers defines the maximum number of concurrent events that may be processed), or disable concurrency controls for unlimited parallelization.
Throttling Workflow Execution
To throttle workflow execution, enable it in your workflow settings and configure the limit and interval. The limit defines how many events (from0-10000) to process in a given time period.
The interval defines the time period over which the limit will be enforced. You may specify the time period as a number of seconds, minutes or hours (ranging from 1-10000)
Applying Concurrency and Throttling Together
The conditions for both concurrency and throttling must be met in order for a new event to trigger a workflow execution. Here are some examples:| Concurrency | Throttling | Result |
|---|---|---|
| Off | Off | Events will trigger your workflow as soon as they are received. Events may execute in parallel. |
| 1 Worker | Off | Events will trigger your workflow in a serialized pattern (a maximum of 1 event will be processed at a time). As soon as one event finishes processing, the next event in the queue will be processed. |
| 1 Worker | 1 Event per Second | Events will trigger your workflow in a serialized pattern at a maximum rate of 1 event per second. If an event takes less than one second to finish processing, the next event in the queue will not being processing until 1 second from the start of the most recently processed event. If an event takes longer than one second to process, the next event in the queue will begin processing immediately. |
| 1 Worker | 10 Events per Minute | Events will trigger your workflow in a serialized pattern at a maximum rate of 10 events per minute. If an event takes less than one minute to finish processing, the next event in the queue immediately begin processing. If 10 events been processed in less than one minute, the remaining events will be queued until 1 minute from the start of the initial event. |
| 5 Workers | Off | Up to 5 events will trigger your workflow in parallel as soon as they are received. If more events arrive while 5 events are being processed, they will be queued and executed in order as soon as an event completes processing. |
Pausing Workflow Execution
To stop the queue from invoking your workflow, throttle workflow execution and set the limit to0.
Increasing the queue size for a workflow
By default, your workflow can hold up to events in its queue at once. Any events that arrive once the queue is full will be dropped, and you’ll see an Event Queue Full error. For example, if you serialize the execution of your workflow by setting a concurrency of1, but receive 200 events from your workflow’s event source at once, the workflow’s queue can only hold the first 100 events. The last 100 events will be dropped.
Users on paid tiers can increase their queue size up to events for a given workflow, just below the Concurrency section of your Execution Controls settings: