unique
and greatest
strategieshello world!
component using the Pipedream CLI and invoke it manually$.service.db
to maintain state across executions$.interface.timer
to invoke a component on a schedule$.interface.http
to invoke code on HTTP requestsrss-parser
npm package to retrieve an RSS feed and emit each itemunique
deduping strategy so we only emit new items from the RSS feedgreatest
deduping strategy to only emit new issuespd dev
command. This command will deploy your code in “development mode”. What that means is that the CLI will attach to the deployed component and watch your local file for changes —when you save changes to your local file, your component will automatically be updated on Pipedream (the alternative is to pd deploy
and run pd update
for each change).
If your pd dev
session is terminated and you need to re-attach to a deployed component, run the following command.
components/sources/my-source.mjs
, then pass the fully qualified path to pd dev
:
dc
argument to update it with new source code:
{ message: "hello world!" }
on each execution.
.js
file (e.g., source.js
) and run the following CLI command:
https://pipedream.com/sources/dc_v3uXKz
in the sample output above) to view your source in Pipedream’s UI.
Then click RUN NOW to invoke your source. Your event will appear in real-time, and you can select it to inspect the emitted data.
source
db
service to track the number of times the component is invoked.
First, we’ll assign $.service.db
to a prop so we can reference it in our code via this
.
run()
method to:
count
key (using the get()
method of $.service.db
)count
and save the updated value to $.service.db
using the set()
methodsource
timer
interface and we’ll set the default execution interval to 15 minutes by adding the following code to props:
source
intervalSeconds
within the component’s code will not change the schedule of the running instance of the component. You can also set one value as the default intervalSeconds
in the component’s code, but run
timer
interface with an http
interface.
run()
method to use event
to both echo back the request body in the HTTP response and emit it as the event payload.
{ message: "hello world!" }
as the request body. { message: "hello world!" }
will be emitted as the event similar to the earlier examples, and it will also be echoed back in the HTTP response.
source
rss-parser
package at https://www.npmjs.com/package/rss-parser. To use most npm packages on Pipedream, just import
them — there is no package.json
or npm install
required.rss-parser
npm package.
run()
method to:
https://lorem-rss.herokuapp.com/feed
(it’s important you use this feed — a new item is added every minute, so it will help us test deduplication)title
as the summary for each RSS item. To do that, we add a metadata object to this.$emit()
.
Add summary to emit metadata…
unique
strategy. This strategy caches the last 100 event id
values, and only emits events with id
values that are not contained in that cache.
To dedupe with the unique
strategy, we need to first declare it:
id
value in the metadata for this.$emit()
for Pipedream to use for deduping:
axios
so we can make a request to the Github REST API:
app
property is the name slug for the app in Pipedream. This is not currently discoverable, but it will be in the near future. For the time being, if you want to know how to reference an app, please reach out on our public Slack.run()
method to fetch issues from Github using axios
and emit them. Notice that we’re passing the oauth_access_token
in the authorization header by referencing the app prop this.github.$auth.oauth_access_token
. Again, it’s important that you stick with the pddemo/demo
repo shown in the below example so you can test the next dedupe strategy.
pd dev
—follow the CLI prompts to select a connected account for Github (or connect a new one). Then load the Pipedream UI, and click RUN NOW. Your component should emit 30 issues.
unique
strategy. The limitation of the unique strategy is that it will only maintain uniqueness for 100 items. Since Github issues have increasing numeric IDs, we can use the greatest
strategy to filter for new issues.
To use this strategy, we first have to declare it.
this.$emit()
. We can also add a summary and a timestamp (based on the date/time when the issue was created). Note: when you add a timestamp, Pipedream will automatically emit events from oldest to newest.