Running workflows for your end users
Just like you can build and run internal workflows for your team, you can run workflows for your end users, too.
Whether you're building well-defined integrations or more-autonomous AI agents, workflows provide a powerful set of tools for running code or pre-defined actions on behalf of your users. Pipedream's UI makes it easy to build, test, and debug workflows.
What are workflows?
Workflows are sequences of steps triggered by an event, like an HTTP request, or new rows in a Google sheet.
You can use pre-built actions or custom Node.js, Python, Golang, or Bash code in workflows and connect to any of our 2,400 integrated apps.
Workflows also have built-in:
Read the quickstart to learn more.
How to run workflows for your end users
Enable the feature flag
Go to https://pipedream.com/settings/alpha and click the checkbox, Allow use Connect in builder.
Create an OAuth client
This step is optional but strongly recommended. To securely run workflows for end users, you should create a Pipedream OAuth client. Pipedream uses OAuth to authenticate requests to the Pipedream API and workflows.
Create a workflow
Create a new workflow or open an existing one.
Add an HTTP trigger, configure OAuth
- Add an HTTP trigger to your workflow.
- Configure OAuth authorization on the trigger. Again, this step is optional but strongly recommended.
Configure accounts to use your end users' auth
When you configure pre-built actions or custom code that connects to third-party APIs, you can link accounts in one of two ways:
- Use your own account: If you're connecting to an API that uses your own API key or developer account — for example, a workflow that connects to the OpenAI API or a PostgreSQL database — click the Connect account button to link your own, static account.
- Use your end users' auth: If you're building a workflow that connects to your end users' accounts — for example, a workflow that sends a message with your user's Slack account — you can select the option to Use end user's auth via Connect:
When you trigger the workflow, Pipedream will look up the corresponding account for the end user whose user ID you provide when invoking the workflow.
Connect a test account
To run an end-to-end test as an end user, you need to have users and connected accounts in your project. If you already have a development account linked, you can skip this step.
If you don't, the fastest way to do this is on the Users tab in your Pipedream project:
- You'll see there's a button to Connect account
- Go through the flow and make sure to create the account in development mode
- Note the external user ID of the account you just connected, you'll need it in the next step
Generate a test request
Test events are critical for developing workflows effectively. Without a test event, you won't be able to test your workflow end to end in the builder, see the shape of the event data that triggers the workflow, and the lookup to use your end user's auth won't work.
To generate a test event, click Send Test Event in the trigger, and fill in the event data. This will trigger the workflow and allow you to test the workflow end to end in the builder.
Make sure to include these headers in your test request:
x-pd-environment: development
x-pd-external-user-id: {your_external_user_id}
Deploy your workflow
When you're done with the workflow, click Deploy at the top right.
Invoke the workflow
If you're using TypeScript or a JavaScript runtime, install the Pipedream SDK. Pipedream also provides an HTTP API for invoking workflows (see example below).
npm i @pipedream/sdk
To invoke workflows, you'll need:
- The OAuth client ID and secret from your OAuth client in step 2
- Your Project ID
- Your workflow's HTTP endpoint URL
- The external user ID of the user you'd like to run the workflow for
- The Connect environment tied to the user's account
Then invoke the workflow like so:
import { createBackendClient, HTTPAuthType } from "@pipedream/sdk/server";
// These secrets should be saved securely and passed to your environment
const pd = createBackendClient({
environment: "development", // change to production if running for a test production account, or in production
credentials: {
clientId: "{oauth_client_id}",
clientSecret: "{oauth_client_secret}",
},
projectId: "{your_project_id}"
});
await pd.invokeWorkflowForExternalUser(
"{your_endpoint_url}", // pass the endpoint ID or full URL here
"{your_external_user_id}" // The end user's ID in your system
{
method: "POST",
body: {
message: "Hello World"
}
},
HTTPAuthType.OAuth // Will automatically send the Authorization header with a fresh token
)
Step configuration
When configuring a workflow that's using your end user's auth instead of your own, you'll need to define most configuration fields manually in each step.
For example, normally when you connect your own Google Sheets account directly in the builder, you can dynamically list all of the available sheets from a dropdown.
However, when running workflows on behalf of your end users, that UI configuration doesn't work, since the Google Sheets account to use is determined at the time of workflow execution. So instead, you'll need to configure these fields manually.
- Either make sure to pass all required configuration data when invoking the workflow, or add a step to your workflow that retrieve it from your database, etc. For example:
curl -X POST https://{your-endpoint-url} \
-H "Content-Type: application/json" \
-H "Authorization: Bearer {access_token}" \
-H 'X-PD-External-User-ID: {your_external_user_id}' \
-H 'X-PD-Environment: development' \ # 'development' or 'production'
-d '{
"slackChannel": "#general",
"messageText": "Hello, world!",
"gitRepo": "AcmeOrg/acme-repo",
"issueTitle": "Test Issue"
}' \
- Then in the Slack and GitHub steps, you'd reference those fields directly:
We plan to improve this interface in the future, and potentially allow developers to store end user metadata and configuration data alongside the connected account for your end users, so you won't need to pass the data at runtime.
Errors
No external user ID passed, but one or more steps require it
- One or more steps in the workflow are configured to Use end user's auth via Connect, but no external user ID was passed when invoking the workflow.
- Refer to the docs to make sure you're passing external user ID correctly when invoking the workflow.
No matching external user ID
- There was an external user ID passed, but it didn't match any users in the project.
- Double-check that the external user ID that you passed when invoking the workflow matches one either in the UI or via the API.
Required account not found for external user ID
- The external user ID was passed when invoking the workflow, but the user doesn't have a connected account for one or more of the apps that are configured to use it in this workflow execution.
- You can check which connected accounts are available for that user in the UI or via the API.
Running workflows for your users in production requires a higher tier plan
- Anyone is able to run workflows for your end users in
development
. The Business plan is required to run on behalf ofproduction
users. - Schedule a call with our sales team and learn more about pricing here.