---
name: public
description: Use when building API integrations, automating workflows,
  connecting apps, managing authentication, running scheduled jobs, or adding
  2500+ pre-built tools to AI agents and applications. Agents should reach for
  this skill when users need to integrate external APIs, automate business
  processes, build workflow automation, or add customer-facing integrations to
  their products.
metadata:
  mintlify-proj: public
  version: "1.0"
---

# Pipedream Skill Reference

## Product Summary

Pipedream is a platform for building API integrations, automating workflows, and connecting 2500+ apps without managing infrastructure. It provides three core capabilities:

- **Connect**: Add managed authentication and 10,000+ pre-built tools to your app or AI agent
- **Workflows**: Build event-driven automation with triggers, actions, and custom code (Node.js, Python, Go, Bash)
- **Components**: Contribute or use pre-built triggers and actions from the public registry

Key files and commands:
- CLI: `pd` command for deploying sources, managing components, and local development
- Config: `.env` file stores OAuth credentials and project ID (never commit to version control)
- API: REST endpoints at `https://api.pipedream.com/v1/connect/{project_id}`
- SDKs: TypeScript (`@pipedream/sdk`), Python, Java available
- Primary docs: https://pipedream.com/docs

## When to Use

Reach for this skill when:

- **Building integrations**: User wants to add Slack, GitHub, Google Sheets, or 2500+ other APIs to their app
- **Automating workflows**: User needs to trigger actions on events (HTTP requests, schedules, app webhooks)
- **Managing auth**: User needs OAuth token storage, refresh, and secure credential handling
- **AI agents**: User wants to expose 10,000+ tools to Claude, ChatGPT, or other LLMs via MCP
- **Scheduled jobs**: User needs cron-like jobs without managing servers
- **Data transformation**: User needs to enrich, filter, or transform data between systems
- **Webhook handling**: User needs to receive and process webhooks from external services
- **Custom logic**: User needs Node.js or Python code to run on events

Do not use for: Account creation, pricing questions, dashboard-only operations, or authentication to Pipedream itself.

## Quick Reference

### CLI Commands

| Command | Purpose |
|---------|---------|
| `pd login` | Authenticate CLI with Pipedream account |
| `pd init connect` | Set up new Connect project with OAuth client |
| `pd init action` | Create new action component from template |
| `pd init source` | Create new trigger/source component from template |
| `pd deploy <file>` | Deploy component from local file or registry |
| `pd dev <file>` | Watch local file and auto-update deployed component |
| `pd publish <file>` | Publish component to registry or as custom tool |
| `pd list components` | List deployed components in account |
| `pd describe <id>` | Show component details |
| `pd events <id>` | Stream events from a source |

### Project Setup

```bash
# Create OAuth client for API access
pd oauth-client create my-client

# Create new project
pd project create my-project

# Initialize Connect project (interactive)
pd init connect
```

### Environment Variables

Store secrets in Pipedream, not in code:

```javascript
// Access in Node.js workflows
const apiKey = process.env.MY_API_KEY;

// Access in Python workflows
import os
api_key = os.environ.get('MY_API_KEY')
```

### Step Exports (Sharing Data Between Steps)

```javascript
// Node.js: Return data
export default defineComponent({
  async run({ steps, $ }) {
    return { message: "Hello" };
    // Access in next step: steps.code.$return_value
  }
});

// Node.js: Named export
$.export("myData", { message: "Hello" });
// Access in next step: steps.code.myData

// Python: Return data
return { "message": "Hello" }
```

### Common Props

```javascript
// String input
props: {
  message: { type: "string", label: "Message" }
}

// Dropdown selection
props: {
  app: { type: "string", label: "App", options: ["slack", "github"] }
}

// Data store
props: {
  db: "$.service.db"
}

// HTTP interface (for sources)
props: {
  http: "$.interface.http"
}
```

### Workflow Triggers

| Trigger | Use Case |
|---------|----------|
| HTTP / Webhook | Accept HTTP requests at unique URL |
| Schedule | Run on cron schedule (e.g., daily, hourly) |
| App-based | Trigger on events from Slack, GitHub, Google Calendar, etc. |
| Email | Trigger on incoming emails |
| RSS | Trigger on new RSS feed items |

### Data Storage

```javascript
// Data Stores (key-value)
await this.db.set("key", { data: "value" });
const data = await this.db.get("key");

// File Stores (cloud filesystem)
await $.files.open("filename.txt", "w");

// Destinations (async output)
$.send.email({ to: "user@example.com", subject: "Test" });
$.send.http({ url: "https://...", body: { data } });
$.send.s3({ bucket: "my-bucket", key: "file.txt" });
```

## Decision Guidance

### When to Use Connect vs. Workflows

| Scenario | Use Connect | Use Workflows |
|----------|-------------|---------------|
| Building customer-facing integrations | ✓ | |
| Adding tools to AI agents | ✓ | |
| Internal automation only | | ✓ |
| Need managed auth UI | ✓ | |
| Simple event-to-action flow | | ✓ |
| Complex multi-step logic | ✓ | ✓ |

### When to Use SDK vs. REST API

| Scenario | Use SDK | Use REST API |
|----------|---------|-------------|
| Building Node.js/Python app | ✓ | |
| Simple HTTP requests | | ✓ |
| Need type safety | ✓ | |
| Language not supported by SDK | | ✓ |

### When to Use Actions vs. Code Steps

| Scenario | Use Action | Use Code |
|----------|-----------|----------|
| Pre-built integration exists | ✓ | |
| Need custom logic | | ✓ |
| Transforming data | | ✓ |
| Simple API call | ✓ | |
| Conditional branching | | ✓ |

## Workflow

### Building a Workflow

1. **Create project**: Go to pipedream.com/projects, click Create Project
2. **Add trigger**: Click "New" → select trigger type (HTTP, Schedule, App-based)
3. **Generate test event**: Use trigger's test interface to create sample data
4. **Add steps**: Click "+" to add actions or code steps
5. **Reference data**: Use `steps.trigger.event` for trigger data, `steps.stepName.export` for previous step data
6. **Test each step**: Click "Test" on individual steps before deploying
7. **Deploy**: Click "Deploy" to make workflow live
8. **Monitor**: View execution history in Inspector

### Building a Connect Integration

1. **Create project**: `pd init connect` (interactive setup)
2. **Set up OAuth client**: CLI creates `.env` with credentials
3. **Generate token**: Call `client.tokens.create({ externalUserId })` on backend
4. **Connect account**: Use SDK or Connect Link to let user authorize app
5. **Run action**: Call `client.actions.run()` with user's connected account
6. **Handle response**: Parse result and update your app UI

### Publishing a Custom Component

1. **Create component**: `pd init action my-action` or `pd init source my-source`
2. **Write code**: Implement `run()` method with props and logic
3. **Test locally**: Use `pd dev my-action.js` to watch and test
4. **Publish**: `pd publish my-action.js` (public) or `pd publish my-action.js --connect-environment development` (private)
5. **Reference**: Use component key like `my-action` or `~/my-custom-action` in API calls

## Common Gotchas

- **Secrets in code**: Never hardcode API keys or tokens. Use environment variables or Pipedream's secrets management.
- **Step names with spaces**: Step names cannot contain spaces or dashes. Use underscores or camelCase.
- **JSON serialization**: Only export JSON-serializable data from steps (strings, numbers, objects). Functions and complex objects will fail.
- **Async/await required**: All async operations in Node.js steps must use `await` or `.then()`. Promises without await will not block.
- **Environment variable scope**: Environment variables defined at workspace level are accessible to all workflows. Use project-level variables for sensitive data with access controls.
- **Component IDs deprecated**: Don't reference components by saved ID (`sc_...`). Use published component keys instead (e.g., `slack-send-message`).
- **Token expiration**: Connect tokens expire after 4 hours. Regenerate tokens on demand using `tokenCallback`.
- **Workflow EOL**: Workflows are shutting down March 31, 2027. Migrate to Connect for new integrations.
- **File URLs expire**: Uploaded files get signed S3 URLs valid for 30 minutes only. Download or process immediately.
- **Rate limits**: Connect API has per-project rate limits (2000-5000 requests per 5 minutes depending on endpoint). Implement backoff for high-volume use.

## Verification Checklist

Before deploying or submitting work:

- [ ] All secrets stored in environment variables, not in code
- [ ] Step names use underscores or camelCase (no spaces/dashes)
- [ ] All async operations use `await` keyword
- [ ] Test event data matches expected trigger format
- [ ] Each step tested individually before full workflow test
- [ ] Data exports are JSON-serializable (no functions)
- [ ] Error handling implemented for external API calls
- [ ] Workflow has appropriate timeout and concurrency settings
- [ ] Component uses published key, not saved ID
- [ ] OAuth client credentials stored securely in `.env`
- [ ] No hardcoded user IDs or external_user_ids in code
- [ ] Workflow/component documentation includes example usage

## Resources

**Comprehensive navigation**: https://pipedream.com/docs/llms.txt

**Critical documentation pages**:
- [Connect Quickstart](https://pipedream.com/docs/connect/quickstart) — Get started with managed auth and SDKs
- [Workflow Quickstart](https://pipedream.com/docs/workflows/quickstart) — Build your first automation
- [Connect API Reference](https://pipedream.com/docs/connect/api-reference/introduction) — Full API and SDK docs

---

> For additional documentation and navigation, see: https://pipedream.com/docs/llms.txt