npm install
or package.json
requiredtype: action
(see the docs on the type
property). Sources require no type
property be set. Components without a type
are considered sources.
this.$emit
, which trigger linked workflows. Any features associated with emitting events (e.g., dedupe strategies) can only be used with sources. Actions return data using return
or $.export
, which is made available to future steps of the associated workflow.
$
variable, passed as a parameter to the run
method. This variable exposes functions that allow you to send data to destinations, export data from the action, return HTTP responses, and more.
pd dev
. Actions currently cannot (please follow this issue to be notified of updates).
pd deploy
to deploy sources to your account. You use pd publish
to publish actions, making them available for use in workflows.
components/
directory of the PipedreamHQ/pipedream
repo.
Property | Type | Required? | Description |
---|---|---|---|
name | string | required | The name of the component, a string which identifies components deployed to users’ accounts. This name will show up in the Pipedream UI, in CLI output (for example, from pd list commands), etc. It will also be converted to a unique slug on deploy to reference a specific component instance (it will be auto-incremented if not unique within a user account). |
key | string | recommended | The key uniquely identifies a component within a namespace. The default namespace for components is your account. When publishing components to the Pipedream registry, the key must be unique across registry components and should follow the pattern: app_name_slug -slugified-component-name |
type | string | required | When publishing an action, type: "action" is required. When publishing a source, use type: "source" . |
version | string | required | The component version. There are no constraints on the version, but semantic versioning is required for any components published to the Pipedream registry. |
description | string | recommended | The description will appear in the Pipedream UI to aid in discovery and to contextualize instantiated components |
props | object | optional | Props are custom attributes you can register on a component. When a value is passed to a prop attribute, it becomes a property on that component instance. You can reference these properties in component code using this (e.g., this.propName ). |
methods | object | optional | Define component methods for the component instance. They can be referenced via this (e.g., this.methodName() ). |
hooks | object | optional (sources only) | Hooks are functions that are executed when specific component lifecycle events occur. |
dedupe | string | optional (sources only) | You may specify a dedupe strategy to be applied to emitted events |
run | method | required | Each time a component is invoked (for example, via HTTP request), its run method is called. The event that triggered the component is passed to run , so that you can access it within the method. Events are emitted using this.$emit() . |
this
(e.g., this.propName
).
Prop Type | Description |
---|---|
User Input | Enable components to accept input on deploy |
Interface | Attaches a Pipedream interface to your component (e.g., an HTTP interface or timer) |
Service | Attaches a Pipedream service to your component (e.g., a key-value database to maintain state) |
App | Enables managed auth for a component |
Data Store | Provides access to a Pipedream data store |
HTTP Request | Enables components to execute HTTP requests based on user input |
Alert | Renders an informational alert in the prop form to help users configure the source or action |
Property | Type | Required? | Description |
---|---|---|---|
type | string | required | Value must be set to a valid PropType (see below). Suffix with [] (e.g. string[] ) to denote array of that type (if supported). |
label | string | optional | A friendly label to show to user for this prop. If a label is not provided, the propName is displayed to the user. |
description | string | optional | Displayed near the prop input. Typically used to contextualize the prop or provide instructions to help users input the correct value. Markdown is supported. |
options | string[] or object[] or method | optional | Provide an array to display options to a user in a drop down menu. [] Basic usage Array of strings. E.g., ['option 1', 'option 2'] object[] Define Label and Value [{ label: 'Label 1', value: 'label1'}, { label: 'Label 2', value: 'label2'}] method Dynamic Options You can generate options dynamically (e.g., based on real-time API requests with pagination). See configuration details below. |
useQuery | boolean | optional | Use in conjunction with Dynamic Options. If set to true , the prop accepts a real-time query that can be used by the options method to obtain results according to that query. |
optional | boolean | optional | Set to true to make this prop optional. Defaults to false . |
propDefinition | [] | optional | Re-use a prop defined in an app file. When you include a prop definition, the prop will inherit values for all the properties listed here. However, you can override those values by redefining them for a given prop instance. See propDefinitions below for usage. |
default | string | optional | Define a default value if the field is not completed. Can only be defined for optional fields (required fields require explicit user input). |
secret | boolean | optional | If set to true , this field will hide your input in the browser like a password field, and its value will be encrypted in Pipedream’s database. The value will be decrypted when the component is run in the execution environment. Defaults to false . Only allowed for string props. |
min | integer | optional | Minimum allowed integer value. Only allowed for integer props.. |
max | integer | optional | Maximum allowed integer value . Only allowed for integer props. |
disabled | boolean | optional | Set to true to disable usage of this prop. Defaults to false . |
hidden | boolean | optional | Set to true to hide this field. Defaults to false . |
Prop Type | Array Supported | Supported in Sources? | Supported in Actions? | Custom properties |
---|---|---|---|---|
app | ✓ | ✓ | See App Props below | |
boolean | ✓ | ✓ | ✓ | |
integer | ✓ | ✓ | ✓ | - min (integer ): Minimum allowed integer value. - max (integer ): Maximum allowed integer value. |
string | ✓ | ✓ | ✓ | - secret (boolean ): Whether to treat the value as a secret. |
object | ✓ | ✓ | ||
any | ✓ | |||
$.interface.http | ✓ | |||
$.interface.timer | ✓ | |||
$.service.db | ✓ | |||
data_store | ✓ | |||
http_request | ✓ | |||
alert | ✓ | ✓ | See Alert Prop below |
Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.myPropName | Returns the configured value of the prop | run() hooks methods | n/a (input props may only be modified on component deploy or update via UI, CLI or API) |
Property | Type | Required? | Description |
---|---|---|---|
options() | method | optional | Typically returns an array of values matching the prop type (e.g., string ) or an array of object that define the label and value for each option. The page and prevContext input parameter names are reserved for pagination (see below). When using prevContext for pagination, it must return an object with an options array and a context object with a nextPageToken key. E.g., { options, context: { nextPageToken }, } |
page | integer | optional | Returns a 0 indexed page number. Use with APIs that accept a numeric page number for pagination. |
prevContext | string | optional | Returns a string representing the context for the previous options execution. Use with APIs that accept a token representing the last record for pagination. |
query | string | optional | Returns a string with the user input if the prop has the useQuery property set to true . Use with APIs that return items based on a query or search parameter. |
Property | Type | Required? | Description |
---|---|---|---|
propDefinition | array | optional | An array of options that define a reference to a propDefinitions within the propDefinitions for an app |
app | object | required | An app object |
propDefinitionName | string | required | The name of a specific propDefinition defined in the corresponding app object |
inputValues | object | optional | Values to pass into the prop definition. To reference values from previous props, use an arrow function. E.g.,: c => ({ variableName: c.previousPropName }) See these docs for more information. |
propDefinitions
.
Trello board example
Trello board and lists props
lists
prop, notice how opts.board
references the board. You can pass opts
to the prop’s options
method when you reference propDefinitions
in specific components:
configuredProps
contains the props the user previously configured (the board). This allows the lists
prop to use it in the options
method.
Google Sheets Additional props example - header columns loading as props
reloadProps
field set to true
:
additionalProps
component method to render props:
previousPropDefs
are the full set of props (props merged with the previous additionalProps
). When the function is executed, this
is bound similar to when the run
function is called, where you can access the values of the props as currently configured, and call any methods
. The return value of additionalProps
will replace any previous call, and that return value will be merged with props to define the final set of props.
Following is an example that demonstrates how to use additionalProps
to dynamically change a prop’s disabled
and hidden
properties:
app
boolean
integer
string
object
any
$.interface.http
$.interface.timer
data_store
http_request
Interface Type | Description |
---|---|
Timer | Invoke your source on an interval or based on a cron expression |
HTTP | Invoke your source on HTTP requests |
$.interface.timer
:
Definition
Property | Type | Required? | Description |
---|---|---|---|
type | string | required | Must be set to $.interface.timer |
default | object | optional | Define a default interval { intervalSeconds: 60, }, Define a default cron expression { cron: "0 0 * * *", }, |
Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.myPropName | Returns the type of interface configured (e.g., { type: '$.interface.timer' } ) | run() hooks methods | n/a (interface props may only be modified on component deploy or update via UI, CLI or API) |
event | Returns an object with the execution timestamp and interface configuration (e.g., { "timestamp": 1593937896, "interval_seconds": 3600 } ) | run(event) | n/a (interface props may only be modified on source deploy or update via UI, CLI or API) |
$.interface.timer
and has default defined as a cron expression.
$.interface.timer
and has a default
interval defined.
$.interface.http
:
Property | Type | Required? | Description |
---|---|---|---|
type | string | required | Must be set to $.interface.http |
respond | method | required | The HTTP interface exposes a respond() method that lets your component issue HTTP responses to the client. |
Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.myPropName | Returns an object with the unique endpoint URL generated by Pipedream (e.g., { endpoint: 'https://abcde.m.pipedream.net' } ) | run() hooks methods | n/a (interface props may only be modified on source deploy or update via UI, CLI or API) |
event | Returns an object representing the HTTP request (e.g., { method: 'POST', path: '/', query: {}, headers: {}, bodyRaw: '', body: {}, } ) | run(event) | The shape of event corresponds with the the HTTP request you make to the endpoint generated by Pipedream for this interface |
this.myPropName.respond() | Returns an HTTP response to the client (e.g., this.http.respond({status: 200}) ). | n/a | run() |
respond()
method that lets your source issue HTTP responses. You may run this.http.respond()
to respond to the client from the run()
method of a source. In this case you should also pass the customResponse: true
parameter to the prop.
Property | Type | Required? | Description |
---|---|---|---|
status | integer | required | An integer representing the HTTP status code. Return 200 to indicate success. Standard status codes range from 100 - 599 |
headers | object | optional | Return custom key-value pairs in the HTTP response |
body | string object buffer | optional | Return a custom body in the HTTP response. This can be any string, object, or Buffer. |
run()
method of your source:
$.interface.http
and returns { 'msg': 'hello world!' }
in the HTTP response. On deploy, Pipedream will generate a unique URL for this source:
Service | Description |
---|---|
DB | Provides access to a simple, component-specific key-value store to maintain state across executions. |
Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.myPropName.get('key') | Method to get a previously set value for a key. Returns undefined if a key does not exist. | run() hooks methods | Use the set() method to write values |
this.myPropName.set('key', value) | Method to set a value for a key. Values must be JSON-serializable data. | Use the get() method to read values | run() hooks methods |
components/
directory of the pipedream GitHub repo for example app files.
Definition
Property | Type | Required? | Description |
---|---|---|---|
type | string | required | Value must be app |
app | string | required | Value must be set to the name slug for an app registered on Pipedream. App files are programmatically generated for all integrated apps on Pipedream. To find your app’s slug, visit the components directory of the Pipedream GitHub repo, find the app file (the file that ends with .app.mjs ), and find the app property at the root of that module. If you don’t see an app listed, please open an issue here. |
propDefinitions | object | optional | An object that contains objects with predefined user input props. See the section on User Input Props above to learn about the shapes that can be defined and how to reference in components using the propDefinition property |
methods | object | optional | Define app-specific methods. Methods can be referenced within the app object context via this (e.g., this.methodName() ) and within a component via this.myAppPropName (e.g., this.myAppPropName.methodName() ). |
Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.$auth | Provides access to OAuth tokens and API keys for Pipedream managed auth | App Object: methods | n/a |
this.myAppPropName.$auth | Provides access to OAuth tokens and API keys for Pipedream managed auth | Parent Component: run() hooks methods | n/a |
this.methodName() | Execute a common method defined for an app within the app definition (e.g., from another method) | App Object: methods | n/a |
this.myAppPropName.methodName() | Execute a common method defined for an app from a component that includes the app as a prop | Parent Component: run() hooks methods | n/a |
$auth
keys supported for each app will be published in the near future.Code | Description | Read Scope | Write Scope |
---|---|---|---|
this.myPropName.execute() | Execute an HTTP request as configured | n/a | run() methods |
Property | Type | Required? | Description |
---|---|---|---|
type | string | required | Set to alert |
alertType | string | required | Determines the color and UI presentation of the alert prop. Can be one of info , neutral , warning , error . |
content | string | required | Determines the text that is rendered in the alert. Both plain text and markdown are supported. |
pipedream
repo for an example implementation.
Info alert prop in GitHub source
{{steps.nodejs.$return_value}}
, well below the limit. The value of these expressions is evaluated at runtime, and are subject to different limits.
methods
property of your component. You have access to these functions within the run
method, or within other methods.
Methods can be accessed using this.<method-name>
. For example, a random
method:
Property | Type | Required? | Description |
---|---|---|---|
deploy | method | optional | Executed each time a component is deployed |
activate | method | optional | Executed each time a component is deployed or updated |
deactivate | method | optional | Executed each time a component is deactivated |
IMPORTANT: To use a dedupe strategy, you must emit anid
as part of the event metadata (dedupe strategies are applied to the submittedid
)
Strategy | Description |
---|---|
unique | Pipedream maintains a cache of 100 emitted id values. Events with id values that are not in the cache are emitted, and the id value is added to the cache. After 100 events, id values are purged from the cache based on the order received (first in, first out). A common use case for this strategy is an RSS feed which typically does not exceed 100 items |
greatest | Pipedream caches the largest id value (must be numeric). Only events with larger id values are emitted, and the cache is updated to match the new, largest value.. |
last | Pipedream caches the ID associated with the last emitted event. When new events are emitted, only events after the matching id value will be emitted as events. If no id values match, then all events will be emitted. |
run
method is called. Sources are invoked by their interface (for example, via HTTP request). Actions are run when their parent workflow is triggered.
You can reference this
within the run
method. this
refers to the component, and provides access to props, methods, and more.
run
, so that you can access it within the method:
this.$emit()
is a method in scope for the run
method of a source
Property | Type | Required? | Description |
---|---|---|---|
event | JSON serializable data | optional | The data to emit as the event |
id | string or number | Required if a dedupe strategy is applied | A value to uniquely identify this event. Common id values may be a 3rd party ID, a timestamp, or a data hash |
name | string | optional | The name of the “channel” you’d like to emit the event to. By default, events are emitted to the default channel. If you set a different channel here, listening sources or workflows can subscribe to events on this channel, running the source or workflow only on events emitted to that channel. |
summary | string | optional | Define a summary to customize the data displayed in the events list to help differentiate events at a glance |
ts | integer | optional | Accepts an epoch timestamp in milliseconds. If you submit a timestamp, events will automatically be ordered and emitted from oldest to newest. If using the last dedupe strategy, the value cached as the last event for an execution will correspond to the event with the newest timestamp. |
run
method in the Logs section of the Pipedream source UI, or using the pd logs
CLI command:
run
method emits events using this.$emit
, you can access the events in the EVENTS section of the Pipedream UI for the component, or using the pd events
CLI command:
$
variable that gives you access to special functions, outlined below:
return
keyword, or 2) use $.export
to return a named export from a step.
return
Use return
to return data from an action:
steps.[STEP NAME].$return_value
. For example, if you ran the code above in a step named nodejs
, you’d reference the returned data using steps.nodejs.$return_value
.
$.export
You can also use $.export
to return named exports from an action. $.export
takes the name of the export as the first argument, and the value to export as the second argument:
steps.[STEP NAME].[EXPORT NAME]
.
$.respond
$.respond
lets you issue HTTP responses from your workflow. See the full $.respond
docs for more information.
return $.flow.exit
return $.flow.exit
terminates the entire workflow. It accepts a single argument: a string that tells the workflow why the workflow terminated, which is displayed in the Pipedream UI.
$.summary
$.summary
is used to surface brief, user-friendly summaries about what happened when an action step succeeds. For example, when adding items to a Spotify playlist:
Spotify example with $summary
$.send
$.send
allows you to send data to Pipedream destinations.
$.send.http
See the HTTP destination docs.
$.send.email
See the Email destination docs.
$.send.s3
See the S3 destination docs.
$.send.emit
See the Emit destination docs.
$.send.sse
See the SSE destination docs.
$.context
$.context
exposes the same properties as steps.trigger.context
, and more. Action authors can use it to get context about the calling workflow and the execution.
All properties from steps.trigger.context
are exposed, as well as:
Property | Description |
---|---|
deadline | An epoch millisecond timestamp marking the point when the workflow is configured to timeout. |
JIT | Stands for “just in time” (environment). true if the user is testing the step, false if the step is running in production. |
run | An object containing metadata about the current run number. See the docs on $.flow.rerun for more detail. |
secret
props to reference sensitive data.
In actions, you’ll see a list of your environment variables in the object explorer when selecting a variable to pass to a step:
package.json
or npm install
required.
require
string, for example: require("axios@0.19.2")
. Moreover, you can pass the same version specifiers that npm and other tools allow to specify allowed semantic version upgrades. For example:
require("axios@~0.20.0")
require("axios@^0.20.0")
pd dev
command. pd dev
deploys a local file, attaches it to a component, and automatically updates the component on each local save. To deploy a new component with pd dev
, run:
pd deploy
command.
pd publish
command.
deploy
deploy()
hook is automatically invoked by Pipedream when a source is deployed. A common use case for the deploy hook is to create webhook subscriptions when the source is deployed, but you can run any Node.js code within the deploy
hook. To learn more about the deploy()
hook, refer to the API documentation.
activate
activate()
hook is automatically invoked by Pipedream when a source is deployed or updated. For example, this hook will be run when users update component props, so you can run code here that handles those changes. To learn more about defining a custom activate()
hook, refer to the API documentation.
deactivate
deactivate()
hook is automatically invoked by Pipedream when a source is updated or deleted. A common use case for the deactivate hook is to automatically delete a webhook subscription when a component is deleted, but you can run any Node.js code within the deactivate
hook. To learn more about the deactivate()
hook, refer to the API documentation.
activate()
hook.
deactivate()
hook and then deletes the deployed component instance.
deploy()
and activate()
hooks. A unique deployed component ID is generated for the component.
You can deploy a component via the CLI, UI or API.
deactivate()
hook, updates the code and props for a deployed component, and then invokes the optional activate()
hook. The deployed component ID is not changed by an update operation.
deactivate()
hook and deletes the component instance.
Pipedream Components Event Lifecycle Diagram
run()
method of the component is executed. Standard output and errors are surfaced in the Logs tab.
this.$emit()
. If you define a dedupe strategy for a source, Pipedream automatically dedupes the events you emit.
TIP: if you want to use a dedupe strategy, be sure to pass an id
for each event. Pipedream uses this value for deduping purposes.
pd events
command to retrieve the last 10 events via the CLI: