CONNECT APP

Build with Databricks

Databricks is the lakehouse company, helping data teams solve the world’s toughest problems.

Data Analytics

API key

Visit Databricks ↗ Start building

MCP

Give your agent Databricks tools

Every Databricks action is exposed as an MCP tool on Pipedream's remote server. Point a client at it with your end user's ID and Connect resolves that user's Databricks account for each tool call — you store no tokens.

Pipedream Connect

// accessToken: mint a short-lived token with the Connect SDK — see the MCP guide
const transport = new StreamableHTTPClientTransport(
  new URL("https://remote.mcp.pipedream.net/v3"),
  {
    requestInit: {
      headers: {
        Authorization: `Bearer ${accessToken}`,
        "x-pd-project-id": "{project_id}",
        "x-pd-environment": "production",
        "x-pd-external-user-id": "{external_user_id}", // any stable ID for this user in your system
        "x-pd-app-slug": "databricks",
      },
    },
  },
)

const mcp = new Client({ name: "my-agent", version: "1.0.0" })
await mcp.connect(transport)

const { tools } = await mcp.listTools()

// e.g. run Cancel All Runs:
const result = await mcp.callTool({
  name: "databricks-cancel-all-runs",
  arguments: {
    jobId: "Job",
    allQueuedRuns: true,
  },
})

# access_token: mint a short-lived token with the Connect SDK — see the MCP guide
headers = {
    "Authorization": f"Bearer {access_token}",
    "x-pd-project-id": "{project_id}",
    "x-pd-environment": "production",
    "x-pd-external-user-id": "{external_user_id}",  # any stable ID for this user in your system
    "x-pd-app-slug": "databricks",
}

async with streamablehttp_client("https://remote.mcp.pipedream.net/v3", headers=headers) as (read, write, _):
    async with ClientSession(read, write) as session:
        await session.initialize()
        tools = await session.list_tools()

        # e.g. run Cancel All Runs:
        result = await session.call_tool("databricks-cancel-all-runs", {
            "jobId": "Job",
            "allQueuedRuns": True,
        })

MCP guide →

API PROXY

Call the Databricks API directly

For an endpoint with no pre-built tool, the Connect proxy forwards your request to the Databricks API with the connected user's credentials attached. You store no tokens and write no refresh logic.

Pipedream Connect

const resp = await pd.proxy.get({
  externalUserId: "{external_user_id}", // any stable ID for this user in your system
  accountId: "apn_xxxxxxx",
  url: "https://api.example.com/v1/me",
})

// Any allowed Databricks endpoint works here. Pipedream attaches the
// connected account's credentials to the outgoing request.

# The path segment is the target URL, URL-safe base64 encoded:
# https://api.example.com/v1/me

curl "https://api.pipedream.com/v1/connect/{project_id}/proxy/aHR0cHM6Ly9hcGkuZXhhbXBsZS5jb20vdjEvbWU?external_user_id={external_user_id}&account_id=apn_xxxxxxx" \
  -H "Authorization: Bearer {access_token}" \
  -H "x-pd-environment: production"

API proxy guide →

SDK

Run Databricks actions from your backend

Connect a user's Databricks account once, then run Cancel All Runs on their behalf from your own code — TypeScript, Python, or plain HTTP.

Pipedream Connect

import { PipedreamClient } from "@pipedream/sdk"

const pd = new PipedreamClient({
  projectId: process.env.PIPEDREAM_PROJECT_ID!,
  clientId: process.env.PIPEDREAM_CLIENT_ID!,
  clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
  projectEnvironment: "production",
})

const result = await pd.actions.run({
  id: "databricks-cancel-all-runs",
  externalUserId: "{external_user_id}", // any stable ID for this user in your system
  configuredProps: {
    databricks: { authProvisionId: "apn_xxxxxxx" },
    jobId: "Job",
    allQueuedRuns: true,
  },
})

from pipedream import Pipedream

pd = Pipedream(
    client_id="{oauth_client_id}",
    client_secret="{oauth_client_secret}",
    project_id="{project_id}",
    project_environment="production",
)

result = pd.actions.run(
    id="databricks-cancel-all-runs",
    external_user_id="{external_user_id}",  # any stable ID for this user in your system
    configured_props={
        "databricks": {"authProvisionId": "apn_xxxxxxx"},
        "jobId": "Job",
        "allQueuedRuns": True,
    },
)

curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \
  -H "Content-Type: application/json" \
  -H "X-PD-Environment: production" \
  -H "Authorization: Bearer {access_token}" \
  -d '{
    "external_user_id": "{external_user_id}",
    "id": "databricks-cancel-all-runs",
    "configured_props": {
      "databricks": { "authProvisionId": "apn_xxxxxxx" },
      "jobId": "Job",
      "allQueuedRuns": true
    }
  }'