CONNECT APP
Build with Databricks (Service Principal)
Data Analytics
- OAuth
MCP
Give your agent Databricks (Service Principal) tools
Every Databricks (Service Principal) action is exposed as an MCP tool on Pipedream's remote server. Point a client at it with your end user's ID and Connect resolves that user's Databricks (Service Principal) account for each tool call — you store no tokens.
// accessToken: mint a short-lived token with the Connect SDK — see the MCP guide
const transport = new StreamableHTTPClientTransport(
new URL("https://remote.mcp.pipedream.net/v3"),
{
requestInit: {
headers: {
Authorization: `Bearer ${accessToken}`,
"x-pd-project-id": "{project_id}",
"x-pd-environment": "production",
"x-pd-external-user-id": "{external_user_id}", // any stable ID for this user in your system
"x-pd-app-slug": "databricks_oauth",
},
},
},
)
const mcp = new Client({ name: "my-agent", version: "1.0.0" })
await mcp.connect(transport)
const { tools } = await mcp.listTools()
// e.g. run Cancel All Runs:
const result = await mcp.callTool({
name: "databricks_oauth-cancel-all-runs",
arguments: {
jobId: "Job",
allQueuedRuns: true,
},
})# access_token: mint a short-lived token with the Connect SDK — see the MCP guide
headers = {
"Authorization": f"Bearer {access_token}",
"x-pd-project-id": "{project_id}",
"x-pd-environment": "production",
"x-pd-external-user-id": "{external_user_id}", # any stable ID for this user in your system
"x-pd-app-slug": "databricks_oauth",
}
async with streamablehttp_client("https://remote.mcp.pipedream.net/v3", headers=headers) as (read, write, _):
async with ClientSession(read, write) as session:
await session.initialize()
tools = await session.list_tools()
# e.g. run Cancel All Runs:
result = await session.call_tool("databricks_oauth-cancel-all-runs", {
"jobId": "Job",
"allQueuedRuns": True,
})API PROXY
Call the Databricks (Service Principal) API directly
For an endpoint with no pre-built tool, the Connect proxy forwards your request to the Databricks (Service Principal) API with the connected user's credentials attached. You store no tokens and write no refresh logic.
const resp = await pd.proxy.get({
externalUserId: "{external_user_id}", // any stable ID for this user in your system
accountId: "apn_xxxxxxx",
url: "https://api.example.com/v1/me",
})
// Any allowed Databricks (Service Principal) endpoint works here. Pipedream attaches the
// connected account's credentials to the outgoing request.# The path segment is the target URL, URL-safe base64 encoded:
# https://api.example.com/v1/me
curl "https://api.pipedream.com/v1/connect/{project_id}/proxy/aHR0cHM6Ly9hcGkuZXhhbXBsZS5jb20vdjEvbWU?external_user_id={external_user_id}&account_id=apn_xxxxxxx" \
-H "Authorization: Bearer {access_token}" \
-H "x-pd-environment: production"SDK
Run Databricks (Service Principal) actions from your backend
Connect a user's Databricks (Service Principal) account once, then run Cancel All Runs on their behalf from your own code — TypeScript, Python, or plain HTTP.
import { PipedreamClient } from "@pipedream/sdk"
const pd = new PipedreamClient({
projectId: process.env.PIPEDREAM_PROJECT_ID!,
clientId: process.env.PIPEDREAM_CLIENT_ID!,
clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
projectEnvironment: "production",
})
const result = await pd.actions.run({
id: "databricks_oauth-cancel-all-runs",
externalUserId: "{external_user_id}", // any stable ID for this user in your system
configuredProps: {
databricks_oauth: { authProvisionId: "apn_xxxxxxx" },
jobId: "Job",
allQueuedRuns: true,
},
})from pipedream import Pipedream
pd = Pipedream(
client_id="{oauth_client_id}",
client_secret="{oauth_client_secret}",
project_id="{project_id}",
project_environment="production",
)
result = pd.actions.run(
id="databricks_oauth-cancel-all-runs",
external_user_id="{external_user_id}", # any stable ID for this user in your system
configured_props={
"databricks_oauth": {"authProvisionId": "apn_xxxxxxx"},
"jobId": "Job",
"allQueuedRuns": True,
},
)curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \
-H "Content-Type: application/json" \
-H "X-PD-Environment: production" \
-H "Authorization: Bearer {access_token}" \
-d '{
"external_user_id": "{external_user_id}",
"id": "databricks_oauth-cancel-all-runs",
"configured_props": {
"databricks_oauth": { "authProvisionId": "apn_xxxxxxx" },
"jobId": "Job",
"allQueuedRuns": true
}
}'TOOLS
Databricks (Service Principal) actions
On-demand operations your product or agent can configure and run on behalf of a connected user.
-
Cancel All Runs
actionCancel all active runs for a job. The runs are canceled asynchronously, so it doesn't prevent new runs from being started. See the documentationWritev0.0.1 -
Cancel Run
actionCancel a job run. The run is canceled asynchronously, so it may still be running when this request completes. See the documentationWritev0.0.1 -
Create Endpoint
actionCreate a new vector search endpoint. See the documentationWritev0.0.1 -
Create Job
actionCreate a job. See the documentationWritev0.0.1 -
Create SQL Warehouse
actionCreates a new SQL Warehouse in Databricks. See the documentationWritev0.0.1 -
Create Vector Search Index
actionCreates a new vector search index in Databricks. See the documentationWritev0.0.1 -
Delete Data from Vector Search Index
actionDeletes rows from a Direct Access vector index by primary-key values. See the documentationWritev0.0.1 -
Delete Endpoint
actionDelete a vector search endpoint. See the documentationWritev0.0.1 -
Delete Job
actionDelete a job. Deleted jobs cannot be recovered. See the documentationWritev0.0.1 -
Delete Run
actionDelete a non-active run. Returns an error if the run is active. See the documentationWritev0.0.1 -
Delete SQL Warehouse
actionDeletes a SQL Warehouse by ID. See the documentationWritev0.0.1 -
Delete Vector Search Index
actionDeletes a vector search index in Databricks. See the documentationWritev0.0.1 -
Edit SQL Warehouse
actionEdits the configuration of an existing SQL Warehouse. See the documentationWritev0.0.1 -
Export Run
actionExport and retrieve the job run task. See the documentationRead-onlyv0.0.1 -
Get Endpoint
actionGet details of a specific vector search endpoint. See the documentationRead-onlyv0.0.1 -
Get Job
actionRetrieves the details for a single job. See the documentationRead-onlyv0.0.1 -
Get Job Permissions
actionGet permissions of a job. See the documentationRead-onlyv0.0.1 -
Get Run
actionRetrieve the metadata of a run. See the documentationRead-onlyv0.0.1 -
Get Run Output
actionRetrieve the output and metadata of a single task run. See the documentationRead-onlyv0.0.1 -
Get SQL Warehouse
actionRetrieves details for a specific SQL Warehouse. See docsRead-onlyv0.0.1 -
Get SQL Warehouse Config
actionRetrieves the global configuration for SQL Warehouses. See the documentationRead-onlyv0.0.1 -
Get SQL Warehouse Permissions
actionRetrieves the permissions for a specific SQL Warehouse. See the documentationRead-onlyv0.0.1 -
Get Vector Search Index
actionRetrieves details about a specific vector search index. See the documentationRead-onlyv0.0.1 -
List Endpoints
actionList all vector search endpoints. See the documentationRead-onlyv0.0.2 -
List Jobs
actionList all jobs using automatic pagination. See the documentationRead-onlyv0.0.1 -
List Runs
actionLists all runs available to the user. See the documentationRead-onlyv0.0.1 -
List SQL Warehouses
actionLists all SQL Warehouses available in the Databricks workspace. See the documentationRead-onlyv0.0.1 -
List Vector Search Indexes
actionLists all vector search indexes for a given endpoint. See the documentationRead-onlyv0.0.1 -
Query Vector Search Index
actionQuery a specific vector search index in Databricks. See the documentationRead-onlyv0.0.1 -
Query Vector Search Index Next Page
actionFetch the next page of results from a previously executed Databricks Vector Search query, using thenext_page_tokenreturned by a prior call. Run Query Vector Search Index first to get an initial page and itsnext_page_token, then pass that token here (and again on each subsequent call) to walk through all pages. Results are returned in groups of up to 1000, with a maximum of 10,000 total; an emptynext_page_tokenin the response means there are no more pages. See the documentation.Read-onlyv0.0.1 -
Repair Run
actionRe-run one or more tasks. See the documentationWritev0.0.1 -
Reset Job
actionOverwrite all settings for the given job. See the documentationWritev0.0.1 -
Run Job Now
actionRun a job now and return the id of the triggered run. See the documentationWritev0.0.1 -
Scan Vector Search Index
actionScans a vector search index and returns entries after the given primary key. See the documentationRead-onlyv0.0.1 -
Set Job Permissions
actionSet permissions on a job. See the documentationWritev0.0.1 -
Set SQL Warehouse Config
actionUpdates the global configuration for SQL Warehouses. See the documentationWritev0.0.1 -
Set SQL Warehouse Permissions
actionUpdates the permissions for a specific SQL Warehouse. See the documentationWritev0.0.1 -
Start SQL Warehouse
actionStarts a SQL Warehouse by ID. See the documentationWritev0.0.1 -
Stop SQL Warehouse
actionStops a SQL Warehouse by ID. See the documentationWritev0.0.1 -
Sync Vector Search Index
actionSynchronize a Delta Sync vector search index in Databricks. See the documentationWritev0.0.1 -
Update Job
actionUpdate an existing job. Only the fields that are provided will be updated. See the documentationWritev0.0.1 -
Upsert Vector Search Index Data
actionUpserts (inserts/updates) data into an existing vector search index. See the documentationWritev0.0.1
EVENTS
Databricks (Service Principal) triggers
Event sources your backend can deploy for users and receive through a webhook.
No Databricks (Service Principal) triggers are available yet.
REFERENCE
App details
Reference metadata for the Databricks (Service Principal) connector in the Pipedream registry.
- App slug
- databricks_oauth
- Authentication
- OAuth
- Categories
- Data Analytics
- Actions
- 42
- Triggers
- 0
- API proxy
- Available
OAuth scopes
These are the scopes Pipedream's managed Databricks (Service Principal) OAuth client requests when one of your users connects an account. Supply your own OAuth client to request a different set.
- all-apis
- sql