This topic was automatically generated from Slack. You can find the original thread here.
Hi , is there a way for me to add my own custom integration with 3p easily. These are private apps which I want to keep for myself. Also, does pipedream offer web browser automation to secure websites via headless browsers (I can provide the credentials to login)
Absolutely, you can add your own custom integrations with private third-party (3p) apps on Pipedream, and you have a couple of flexible options: — ## 1. Custom Integrations for Private Apps You can build private, custom integrations in Pipedream using code steps (Node.js or Python) within your workflows. This allows you to connect to any API or service, even if there isn’t a pre-built Pipedream integration. Your code and credentials remain private to your account/workspace. - Node.js Example: Use the axios library (imported from @pipedream/platform) to make HTTP requests to your private app’s API. - Python Example: Use standard Python libraries to connect to APIs. - Environment Variables: Store your credentials securely as environment variables or use Pipedream’s built-in secrets management. You can also build and use private components (custom triggers or actions) if you want to reuse your integration logic across workflows. These components can remain private and are not published to the public Pipedream registry. For more details, see the Supported Apps documentation. — ## 2. Web Browser Automation (Headless Browsers) Currently, Pipedream does not natively support running headless browsers (like Puppeteer or Playwright) for web browser automation, especially for interacting with secure websites. Pipedream workflows run in a serverless environment that does not support launching full browser instances due to resource and security constraints. Alternatives: - If you need browser automation, you might consider running a headless browser on another service (like AWS Lambda with custom layers, or a dedicated VM), and trigger that automation from Pipedream via an HTTP request. - For many APIs, direct HTTP requests (with credentials) are sufficient and more efficient than browser automation. — ## Summary - Custom/private integrations: Fully supported via code steps or private components. - Web browser automation: Not natively supported on Pipedream; consider external services for this need. If you have more specific requirements or run into any issues, feel free to ask! If you need more advanced browser automation directly within Pipedream, you can request this feature or integration here. Let me know if you want a code example for your private integration!
understood. My use case would suffice with these two integrations: custom 3p integrations and headless web browser trigger via my own deployment. Would pipedream be able to trigger these custom integrations/triggers if I give it a workflow generation description in the assistant chat?
Yes, Pipedream can trigger your custom integrations and external headless browser automations as part of a workflow, based on a workflow description you provide in the assistant chat. How this works in practice: - Custom 3rd-party integrations: You can use Pipedream code steps (Node.js or Python) to connect to any API or service, including your private apps. Just describe the integration you want, and the assistant can help generate the code step or workflow structure for you. - Headless browser automation via your own deployment: If you deploy your headless browser automation (e.g., Puppeteer, Playwright) on your own infrastructure (like a server, AWS Lambda, etc.), you can expose it via an HTTP endpoint (API). Pipedream can trigger this endpoint using an HTTP request action or a custom code step. Workflow generation via assistant chat: If you describe your workflow in the assistant chat (for example: “When I receive a webhook, call my private API and then trigger my headless browser automation”), the assistant can generate a Pipedream workflow outline or code steps for you. You can then copy these into your Pipedream workflow. Example structure: 1. Trigger: HTTP/Webhook, Scheduler, or any app-based trigger. 2. Action: Custom code step to call your private API. 3. Action: HTTP request or code step to trigger your headless browser automation endpoint. Sample code step to trigger your external automation: ```
import { axios } from “@pipedream/platform”; export default defineComponent({ props: { automationUrl: { type: “string”, label: “Automation Endpoint URL”, description: “The URL of your headless browser automation endpoint”, }, payload: { type: “object”, label: “Payload”, description: “Payload to send to your automation”, optional: true, }, }, async run({ steps, $ }) { return await axios($, { method: “POST”, url: this.automationUrl, data: this.payload, }); }, });
how do I setup pipedream in my local
To set up Pipedream locally, you’ll use the Pipedream CLI. Here are the steps: 1. Create a Pipedream account Sign up for a free account at https://pipedream.com using your Google or GitHub account. 2. Install the Pipedream CLI Download and install the CLI by following the instructions at Pipedream CLI Install. 3. Link your Pipedream account to the CLI After installing the CLI, open your terminal and run: ```
bash pd login