Find the details of the source you’d like to create
To create an event source using Pipedream’s REST API, you’ll need two things:- The
key
that identifies the component by name - The
props
- input data - required to create the source
key
by reviewing the code for the source, in Pipedream’s Github repo.
In the components/
directory, you’ll see a list of apps. Navigate to the app-specific directory for your source, then visit the sources/
directory in that dir to find your source. For example, to create an RSS source, visit the components/rss/sources/new-item-in-feed/new-item-in-feed.js
source.
The key
is a globally unique identifier for the source. You’ll see the key
for this source near the top of the file:
/components/registry/{key}
endpoint of Pipedream’s REST API:
configurable_props
section that lists the input you’ll need to provide to create the source:
rss
“app” prop. The other two props — url
and timer
— are inputs that you can control:
url
: the URL to the RSS feedtimer
(optional): the frequency at which you’d like to poll the RSS feed for new items. By default, this source will poll for new items every 15 minutes.
Creating the source
To create an RSS event source, make an HTTP POST request to the/v1/sources
endpoint of Pipedream’s REST API, passing the url
you’d like to poll and the frequency at which you’d like to run the source in the timer
object. In this example, we’ll run the source once every 60 seconds.
200 OK
response from the API with the following payload:
Fetching new events
The RSS source polls your feed URL for items at the specified frequency. It emits new items as events of the following shape:SSE
You can subscribe to new events in real time by listening to the SSE stream tied to this source. Take theid
from the API response above —dc_abc123
in our example —and make a request to this endpoint:
REST API
You can also fetch items in batch using the REST API. If you don’t need to act on items in real time, and just need to fetch new items from the feed on a regular interval, you can fetch events like so:/event_summaries
endpoint for more details on the parameters it accepts. For example, you can pass a limit
param to return only N
results per page, and paginate over results using the before
and after
cursors described in the pagination docs.