This topic was automatically generated from Slack. You can find the original thread here.
Dylan Sather (Pipedream) : re: Reddit, go ahead and start on the New Link in a Subreddit source. That source should accept the subreddit name as a prop, similar to the action.
In the components directory of your fork, create a new reddit directory.
This is a simple example of the structure of a directory for an app. First, you’ll need to create an “app file” - reddit.app.js - which contains basic metadata about the app and shared props and methods. Here, the app name should be reddit (app: "reddit"). The propDefinitions section should contain props that are shared across multiple sources. Adding them in the app file allows you to reuse the props across sources. So when you add the subreddit prop, I’d recommend keeping it here. The methods section contains shared methods that can also be used across sources, as well.
A note on auth. In workflows, the user’s connected account details for Reddit are accessible at auths.reddit. In sources, that auth data is exposed at this.$auth. So to reference the OAuth access token, for example, you’d use this.$auth.oauth_access_token.
When you create a specific source, create a sources subdirectory within the reddit dir, then another subdir for the source name, then finally the source file itself. For example, this first source should live at components/reddit/sources/new-link-in-subreddit/new-link-in-subreddit.js.
The structure of the source itself is defined by the component API. Here’s one example source, but you’ll find many more in the components dir under the relevant apps, so I’d recommend taking a look there for examples.
Reddit doesn’t support webhooks, so you’ll need to poll the API endpoint on a schedule, looking for new items added since the last new post that was returned from the API. Luckily, this endpoint supports an after parameter that should allow you to fetch new items since a specific item ID. A lot of our sources operate in the same way: 1) pull first set of items, 2) store the last item / last time the source ran in this.db (see docs), 3) the next time the source runs, pull the last item ID from this.db and fetch new items added since then. Take a look at similar sources here, here, and here.
Let me know if that helps get you started, or if you have any other questions.
SERW2039 : Thanks so much for this get started guide, . It totally helps me to get started. I am on the reddit source. About 7), I noticed the need for polling the API for new items. Yeah, I may get back with other questions.
SERW2039 : So I review all the documents, and all info is sinking in. Have one question regarding relationship between the app file and the source file. I notice many of the API requests needed by the source are defined as method in the app file. Does it means that most of the times the app file will have code accessing the app’s API to get data, then then it is called, consumed in the source file to emit events? I.e. the app file could have a method to pull a list, and then the source file could include logic to filter out the newest, or the newest and an arbitrary criteria (i.e. list item with a lenght meeting a minimal num of characters). The app file also has some common methods mostly defined in all apps, such as _getBaseUrl, or _getHeaders
Dylan Sather (Pipedream) : Yes typically we keep most of the API requests in the app file. Then the logic for how that API request is used, and how we handle the associated records, will typically be contained within the source itself.
SERW2039 : I also have a gap when it comes to the app file. How is that going to be deployed? Will that happen when I get a Pull Request accepted/merged on master with my app file (I.e reddit.app.js) and it is built by the PD’s CI/CD process? And then I will have my new-link-in-subreddit.js file be deployed with pd deploy or is it going to be deployed as well bia a probable PR? (and hence be “official”, that is, offitially available as a source for others PD users, with the github example from the component.md file being source that used the existing github.app.js and a “custom” source that is available only for the user pd deploying??
Dylan Sather (Pipedream) : Yes all great questions. When you have a PR ready I’ll show you more. For now, you’ll want to use pd dev to develop like you did in the quickstart. When we merge your PR to master, a Github action publishes the app file + component to our registry, which makes it available to all users.
SERW2039 : Hey Dylan, I don’t have a PR ready yet. But I have two blockers. One is that in a source using the requested new link source, the $auth object is null when deployed and thus reddit api calls fail, but in another source using reddit, the $auth is defined and reddit api work. the other blocker is i defined a reddit.app.js with a simpler reddit api call in the source file, when i do pd dev, it fails saying of a 500 error (my guess is that the reddit.app.js needs to be deployed throug the PR)
Dylan Sather (Pipedream) : I need to turn the Vercel build off, that runs for our docs/ directory but is triggered on every PR (regardless of whether it touches docs). You can ignore the Vercel stuff for now
Dylan Sather (Pipedream) : I added a couple of comments based on issues I observed that I think may have caused the issues you were seeing. Let me know if that helps