This topic was automatically generated from Slack. You can find the original thread here.
I am creating a custom GitHub source with props that should be set using values passed during instantiate via api. What are the prop settings that I should use. Right now, when I develop the GitHub source, it asks me prop values and always just uses these prop values, instead of the ones that I am passing via instantiate via api
To create a custom GitHub source with props that can be set using values passed during instantiation via the API, you need to define the props in your component and ensure they are settable via the API. Here’s how you can define the props and use them in your component: 1. Define the Props: Define the props in your component with the appropriate types and labels. Ensure that these props can accept values passed during instantiation. 2. Use the Props in the Component: Access these props within the run method of your component. Here’s an example of how you can define a GitHub source component with props that can be set via the API: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { github: { type: "app", app: "github", }, repo: { type: "string", label: "Repository", description: "The GitHub repository to monitor", }, eventType: { type: "string", label: "Event Type", description: "The type of GitHub event to listen for", }, }, async run({ steps, $ }) { const repo = this.repo; const eventType = this.eventType; // Example API request using the props const response = await axios($, { url: `https://api.github.com/repos/${repo}/events`, headers: { Authorization: `Bearer ${this.github.$auth.oauth_access_token}`, "X-GitHub-Api-Version": `2022-11-28`, }, }); // Filter events by the specified event type const filteredEvents = response.filter(event => event.type === eventType); return filteredEvents; }, }); ### Explanation: 1. Props Definition: - github: This prop is used to connect the GitHub app. - repo: This prop accepts the repository name as a string. - eventType: This prop accepts the type of GitHub event to listen for as a string. 2. Using Props in the run Method: - The repo and eventType props are accessed using this.repo and this.eventType. - An API request is made to fetch events from the specified repository. - The events are filtered based
Can I use this source now as a trigger in a template, and then use instantiate via api to dynamically initialise these repo and eventType props?
Yes, you can use a source as a trigger in a template and then instantiate it via the API to dynamically initialize the repo and eventType props. Here’s a step-by-step guide on how to achieve this: 1. Create the Source Component: First, you need to create a source component that accepts repo and eventType as props. 2. Deploy the Source Component: Deploy the source component using the Pipedream CLI or UI. 3. Use the Source as a Trigger in a Template: When creating a workflow template, you can select the deployed source as the trigger. 4. Instantiate the Source via API: Use the Pipedream API to instantiate the source with the desired repo and eventType props dynamically. ### Example Source Component Here’s an example of a source component that accepts repo and eventType as props: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { repo: { type: "string", label: "Repository", }, eventType: { type: "string", label: "Event Type", }, }, async run({ steps, $ }) { // Your logic to handle the event const response = await axios($, { url: `https://api.github.com/repos/${this.repo}/events`, params: { event_type: this.eventType, }, }); return response; }, }); ### Deploy the Source Component You can deploy the source component using the Pipedream CLI: sh pd deploy path/to/your/source/component ### Use the Source as a Trigger in a Template In the Pipedream UI, create a new workflow and select the deployed source as the trigger. Configure the repo and eventType props as needed. ### Instantiate the Source via API You can use the Pipedream API to instantiate the source with the desired repo and eventType props dynamically. Here’s an example of how to do this: ```sh curl -X POST https://api.pipedream.com/v1/sources \ -H “Authorization: Bearer <YOUR_API_KEY>” \ -H “Content-Type: application/json” \ -d '{ “component_id”: “<YOUR_COMPONENT_ID>”, "
I was trying to use this custom source as part of the multi-step workflow. Then, when I use instantiate via api, I was trying to change the prop values to initialise the custom source component. Is that possible?
Yes, it is possible to change the prop values when you instantiate a custom source via the API. Here’s a step-by-step guide on how to achieve this: 1. Define Your Custom Source Component: Ensure your custom source component is defined with props. For example: import { axios } from "@pipedream/platform" export default defineComponent({ props: { myProp: { type: "string", label: "My Prop", default: "default value", }, }, async run({ steps, $ }) { console.log(`Prop value: ${this.myProp}`); // Your source logic here }, }); 2. Deploy the Source via API: Use the Pipedream API to deploy your source. You can use the /v1/sources endpoint to create a new source instance and pass the prop values you want to initialize. Here’s an example using curl: bash curl -X POST https://api.pipedream.com/v1/sources \ -H "Authorization: Bearer <YOUR_API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "component": "<YOUR_COMPONENT_PATH>", "name": "My Custom Source", "props": { "myProp": "new value" } }' Replace <YOUR_API_KEY> with your Pipedream API key and <YOUR_COMPONENT_PATH> with the path to your custom source component. 3. Verify the Source: Once the source is created, you can verify that the prop values have been set correctly by checking the source details via the API or the Pipedream UI. ### Example API Request to Instantiate a Source with Custom Props bash curl -X POST https://api.pipedream.com/v1/sources \ -H "Authorization: Bearer <YOUR_API_KEY>" \ -H "Content-Type: application/json" \ -d '{ "component": "my-custom-source", "name": "My Custom Source", "props": { "myProp": "new value" } }' This will create a new instance of your custom source with the myProp value set to "new value". For more detailed information on using components and props in Pipedream,
Do you see any issues with this code: // To use any npm package on Pipedream, just import it
import axios from “axios”
export default defineComponent({
async run({ steps, $ }) {
const { data } = await axios({
method: “POST”,
url: “https://api.pipedream.com/v1/workflows?template_id=tch_4RfVGb”,
headers: {
“Authorization”: Bearer ${process.env.VELORA_PIPEDREAM_API_KEY}
},
data: {
“org_id”: “o_PwIP1O1”,
“project_id”: “proj_GzsL0b2”,
“steps”: [
{
“namespace”: “insert_github_issues_raw”,
“props”: {
“supabase”: {
“subdomain”: “{{process.env.VELORA_SUPABASE_SUBDOMAIN}}”,
“service_key”: “{{process.env.VELORA_SUPABASE_SERVICE_KEY}}”
},
“table”: “github_issues_raw”,
“data”: {
“workspace_id”: “{{steps.trigger.event.velora_workspace_id}}”,
“workflow_id”: “{{steps.trigger.event.velora_workflow_id}}”,
“source_type”: “github_issue”,
“github_repo_full_name”: “{{steps.trigger.event.issue.repo_full_name}}”,
“github_issue_title”: “{{steps.trigger.event.issue.title}}”,
“github_issue_body”: “{{steps.trigger.event.issue.body}}”,
“github_issue_url”: “{{steps.trigger.event.issue.html_url}}”,
“github_comments_url”: “{{steps.trigger.event.issue.comments_url}}”,
“github_issue_logged_by”: “{{steps.trigger.event.issue.user.login}}”,
“github_issue_upvote_count”: “{{steps.trigger.event.issue.reactions[+1]}}”,
“github_issue_downvote_count”: “{{steps.trigger.event.issue.reactions[-1]}}”,
“github_issue_logged_at”: “{{steps.trigger.event.issue.created_at}}”,
“github_issue_word_count”: “{{steps.issue_word_count.$return_value}}”
}
}
}
],
“triggers”: [
{
“props”: {
“github”: {
“authProvisionId”: “”
},
“timer”: {
“intervalSeconds”: 900,
“timezone”: “UTC”
},
“repo_full_name”: steps.trigger.event.body.repo_full_name,
“nango_connection_id”: steps.trigger.event.body.nango_connection_id,
“nango_integration_id”: steps.trigger.event.body.nango_integration_id,
“velora_workflow_id”: steps.trigger.event.body.velora_workflow_id,
“velora_workspace_id”: steps.trigger.event.body.velora_workspace_id
}
}
],
“settings”: {
“name”: ${steps.trigger.event.body.velora_workflow_id} - Github issues to product insights,
“auto_deploy”: true
}
}
});
return data;
},
});
You should visit Pricing - Pipedream for the latest pricing, but I’ll do my best to answer. Here are a few issues and suggestions for your code: 1. Importing axios: - You should import axios from @pipedream/platform instead of axios directly. This ensures that the Pipedream environment handles the HTTP requests correctly. 2. Using Environment Variables: - In Pipedream, environment variables should be accessed through this or steps rather than process.env. 3. Authorization Header: - Ensure that the Authorization header is correctly formatted and that the API key is correctly accessed. 4. Template Strings: - Ensure that template strings are correctly formatted and that the variables are correctly accessed. Here is the corrected code: ``` import { axios } from “@pipedream/platform”; export default defineComponent({ props: { supabase: { type: “app”, app: “supabase”, }, github: { type: “app”, app: “github”, }, }, async run({ steps, $ }) { const { data } = await axios($, { method: “POST”, url: “https://api.pipedream.com/v1/workflows?template_id=tch_4RfVGb”, headers: { “Authorization”: Bearer ${this.github.$auth.oauth_access_token} }, data: { “org_id”: “o_PwIP1O1”, “project_id”: “proj_GzsL0b2”, “steps”: [ { “namespace”: “insert_github_issues_raw”, “props”: { “supabase”: { “subdomain”: this.supabase.$auth.subdomain, “service_key”: this.supabase.$auth.service_key }, “table”: “github_issues_raw”, “data”: { “workspace_id”: steps.trigger.event.velora_workspace_id, “workflow_id”: steps.trigger.event.velora_workflow_id, “source_type”: “github_issue”, “github_repo_full_name”: steps.trigger.event.issue.repo_full_name, “github_issue_title”: steps.trigger.event.issue.title, “github_issue_body”: steps.trigger.event.issue.body
How come you are using github access token to call pipedream api
It seems like there might be some confusion regarding the use of the GitHub access token and the Pipedream API. Let me clarify: 1. GitHub Access Token: This is used to authenticate requests to the GitHub API. When you connect your GitHub account to Pipedream, Pipedream uses this token to make authorized requests to GitHub on your behalf. 2. Pipedream API: This is a separate API that allows you to interact with Pipedream itself, such as managing workflows, components, and other resources. To authenticate requests to the Pipedream API, you typically use a Pipedream API key, not a GitHub access token. If you have any specific use case or example where you see the GitHub access token being used to call the Pipedream API, please provide more details or context so I can assist you better.
In the code you suggested above, you are making call to pipedream api and using github token.
async run({ steps, $ }) {
const { data } = await axios($, {
method: "POST",
url: "https://api.pipedream.com/v1/workflows?template_id=tch_4RfVGb",
headers: {
"Authorization": `Bearer ${this.github.$auth.oauth_access_token}`
},
It looks like you want to make a POST request to the Pipedream API using the GitHub OAuth token. However, the GitHub OAuth token should not be used for authenticating requests to the Pipedream API. Instead, you should use a Pipedream API key for that purpose. Here’s an example of how you can modify the code to use a Pipedream API key: 1. Add a new prop for the Pipedream API key. 2. Use this API key in the Authorization header when making the request to the Pipedream API. Here is the updated code: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { github: { type: "app", app: "github", }, pipedreamApiKey: { type: "string", label: "Pipedream API Key", }, }, async run({ steps, $ }) { return await axios($, { method: "POST", url: "https://api.pipedream.com/v1/workflows?template_id=tch_4RfVGb", headers: { "Authorization": `Bearer ${this.pipedreamApiKey}`, }, }); }, }); In this code: - github prop is used to connect your GitHub account. - pipedreamApiKey prop is used to store your Pipedream API key. - The Authorization header uses the Pipedream API key instead of the GitHub OAuth token. You can find your Pipedream API key in your Pipedream account settings. Make sure to replace <YOUR_PIPEDREAM_API_KEY> with your actual API key when configuring the component.
what is right way to access pipedream env variables?
To access environment variables in Pipedream, you can use the process.env object in your Node.js code. Environment variables allow you to securely store secrets or other configuration values that you can access within your Pipedream workflows. Here’s an example of how to access an environment variable in a Pipedream component: export default defineComponent({ async run({ steps, $ }) { // Access the environment variable using process.env const mySecret = process.env.MY_SECRET; // Use the environment variable in your code console.log(`The secret value is: ${mySecret}`); return mySecret; } }); ### Steps to Set Environment Variables in Pipedream: 1. Navigate to the Environment Variables Section: - Go to your Pipedream dashboard. - Click on the “Environment Variables” tab. 2. Add a New Environment Variable: - Click on “Add Variable”. - Enter the name and value of your environment variable. - If the variable is sensitive (like an API key), mark it as a secret to hide its value from the frontend. 3. Access the Variable in Your Code: - Use process.env.VARIABLE_NAME to access the variable in your Pipedream component code. ### Types of Environment Variables: - Project Variables: Available within a specific project. - Workspace Variables: Available across all projects in a workspace. For more detailed information, you can refer to the Pipedream Environment Variables Documentation. If you have any more questions or need further assistance, feel free to ask!
I have a custom source component with props as trigger to my workflow. How do I dynamically deploy this source using API, along with rest of the workflow steps?
To dynamically deploy a custom source component using the API, along with the rest of the workflow steps, you can follow these steps: 1. Deploy the Source Component: Use the Pipedream API to deploy your custom source component. You will need to define the props for your source and make an API call to create the source. 2. Create the Workflow: After deploying the source, you can create a workflow that uses this source as a trigger and add the necessary actions. Here is an example of how you can achieve this using the Pipedream API: ### Step 1: Deploy the Source Component First, you need to deploy your custom source component. You can use the Pipedream API to create a new source. Here is an example using axios in Node.js: import { axios } from "@pipedream/platform"; async function deploySource() { const response = await axios($, { method: "POST", url: "https://api.pipedream.com/v1/sources", headers: { Authorization: `Bearer ${process.env.PIPEDREAM_API_KEY}`, }, data: { component: "path/to/your/source/component.mjs", // Path to your source component props: { // Define your props here prop1: "value1", prop2: "value2", }, }, }); return response.data; } ### Step 2: Create the Workflow Once the source is deployed, you can create a workflow that uses this source as a trigger. Here is an example: async function createWorkflow(sourceId) { const response = await axios($, { method: "POST", url: "https://api.pipedream.com/v1/workflows", headers: { Authorization: `Bearer ${process.env.PIPEDREAM_API_KEY}`, }, data: { name: "My Workflow", steps: [ { type: "trigger", name: "my_trigger", source_id: sourceId, }, { type: "action", name: "my_action", // Define your action step here }, ], }, }); return response.data; } ### Putting It All Together You can combine the above steps into a single function to deploy the source and create the workflow: ```
In this example, you have used path to component.mjs file to deploy the component. Instead of that, is it possible to send the component config code, directly in the post call?
Yes, it is possible to send the component configuration code directly in the POST call instead of using the path to a component.mjs file. You can do this by including the component_code parameter in your request. Here’s how you can structure your POST request: on { "component_code": "Your component code here" } For example, if you’re using axios to make the POST request, it would look something like this: import axios from "axios"; const componentCode = ` import { axios } from "@pipedream/platform"; export default defineComponent({ props: { openai: { type: "app", app: "openai", } }, async run({steps, $}) { return await axios($, { url: "https://api.openai.com/v1/models", headers: { Authorization: \`Bearer \${this.openai.$auth.api_key}\`, }, }); }, }); `; [axios.post](http://axios.post)("https://api.pipedream.com/v1/components", { component_code: componentCode, }, { headers: { Authorization: `Bearer YOUR_API_KEY`, }, }) .then(response => { console.log(response.data); }) .catch(error => { console.error(error); }); In this example, replace YOUR_API_KEY with your actual Pipedream API key and componentCode with your actual component code. For more details, you can refer to the Pipedream REST API documentation.
shouldn’t this be a source deploy call?
It seems like you’re asking about deploying a source in Pipedream. To deploy a source via CLI, you can use the pd deploy command. Here is an example: bash pd deploy <filename> For instance: ```
bash pd deploy my-source.js