Prefer to write quick scripts in Bash? We've got you covered. You can run any Bash in a Pipedream step within your workflows.

Within a Bash step, you can share data between steps and access environment variables. But you can't connect accounts, return HTTP responses, or take advantage of other features available in the Node.js environment at this time.

Adding a Bash code step

  1. Click the + icon to add a new step
  2. Click Custom Code
  3. In the new step, select the bash runtime in language dropdown

Logging and debugging

When it comes to debugging Bash scripts, echo is your friend.

Your echo statements will print their output in the workflow step results:

MESSAGE='Hello world'
# The message will now be available in the "Result > Logs" area in the workflow step

Available binaries

Bash steps come with many common and useful binaries preinstalled and available in $PATH for you to use out of the box. These binaries include but aren't limited to:

  • curl for making HTTP requests
  • jq for manipulating and viewing JSON data
  • git for interacting with remote repositories

If you need a package pre-installed in your Bash steps, just ask us. Otherwise, you can use the /tmp directory to download and install software from source.

Sharing data between steps

A step can accept data from other steps in the same workflow, or pass data downstream to others.

Using data from another step

In Bash steps, data from the initial workflow trigger and other steps are available in the $PIPEDREAM_STEPS environment variable.

In this example, we'll pretend this data is coming into our HTTP trigger via a POST request.

  "id": 1,
  "name": "Bulbasaur",
  "type": "plant"

In our Bash script, we can access this data via the $PIPEDREAM_STEPS file. Specifically, this data from the POST request into our workflow is available in the trigger object.

cat $PIPEDREAM_STEPS | jq .trigger.event
# Results in { id: 1, name: "Bulbasaur", type: "plant" }

The period (.) in front the trigger.event in the example is not a typo. This is to define the starting point for jq to traverse down the JSON in the HTTP response.

Sending data downstream to other steps

To share data for future steps to use downstream, append it to the $PIPEDREAM_EXPORTS file.

# Retrieve the data from an API and store it in a variable
DATA=`curl --silent`
# Write data to $PIPEDREAM_EXPORTS to share with steps downstream

Not all data types can be stored in the $PIPEDREAM_EXPORTS data shared between workflow steps.

You can only export JSON-serializable data from Bash steps.

Using environment variables

You can leverage any environment variables defined in your Pipedream account in a bash step. This is useful for keeping your secrets out of code as well as keeping them flexible to swap API keys without having to update each step individually.

To access them, just append the $ in front of the environment variable name.


Or an even more useful example, using the stored environment variable to make an authenticated API request.

curl --silent -X POST -h "Authorization: Bearer $TWITTER_API_KEY"

Making a GET request

You can use curl to perform GET requests.

# Get the current weather in San Francisco
WEATHER=`curl --silent\ Francisco\?format=3`
# Produces:
# San Francisco: 🌫  +48°F

Use the --silent flag with curl to suppress extra extra diagnostic information that curl produces when making requests.

This enables you to only worry about the body of the response so you can visualize it with tools like echo or jq.

Making a POST request

curl --silent -X POST -d 'name=Bulbasaur&id=1'
# To store the API response in a variable, interpolate the response into a string and store it in variable
RESPONSE=`curl --silent -X POST -d 'name=Bulbasaur&id=1'`
# Now the response is stored as a variable

Using API key authentication

Some APIs require you to authenticate with a secret API key.

curl has an -h flag where you can pass your API key as a token.

For example, here's how to retrieve mentions from the Twitter API:

# Define the "Authorization" header to include your Twitter API key
curl --silent -X POST -h "Authorization: Bearer $(<your api key here>)"

Raising exceptions

You may need to stop your step immediately. You can use the normal exit function available in Bash to quit the step prematurely.

echo "Exiting now!" 1>&2
exit 1

Using exit to quit a Bash step early won't stop the execution of the rest of the workflow.

Exiting a Bash step will only apply that particular step in the workflow.

This will exit the step and output the error message to stderr which will appear in the results of the step in the workflow.

File storage

If you need to download and store files, you can write them to the /tmp directory.

Writing a file to /tmp

Download a file to /tmp using curl

# Download the current weather in Cleveland in PNG format
curl --silent --output /tmp/weather.png
# Output the contents of /tmp to confirm the file is there
ls /tmp

The /tmp directory does not have unlimited storage. Please refer to the disk limits for details.