Get Run Output with Databricks API on New Webhook Event (Instant) from GitHub API

Pipedream makes it easy to connect APIs for Databricks, GitHub and 2,900+ other apps remarkably fast.

Trigger workflow on

New Webhook Event (Instant) from the GitHub API

Next, do this

Get Run Output with the Databricks API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a GitHub trigger and Databricks action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Webhook Event (Instant) trigger
1. Configure docsInfo
2. Connect your GitHub account
3. Select a Repository
4. Select one or more Webhook Events
Configure the Get Run Output action
1. Connect your Databricks account
2. Optional- Select a Job
3. Select a Run
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Webhook Event (Instant) on GitHub

Description:Emit new event for each selected event type

Version:1.0.11

Key:github-webhook-events

View on GitHub

GitHub Overview#

The GitHub API is a powerful gateway to interaction with GitHub's vast web of data and services, offering a suite of endpoints to manipulate and retrieve information on repositories, pull requests, issues, and more. Harnessing this API on Pipedream, you can orchestrate automated workflows that respond to events in real-time, manage repository data, streamline collaborative processes, and connect GitHub with other services for a more integrated development lifecycle.

Trigger Code#

import common from "../common/common-webhook.mjs";
import constants from "../common/constants.mjs";
import { getRelevantHeaders } from "../common/utils.mjs";

export default {
  ...common,
  key: "github-webhook-events",
  name: "New Webhook Event (Instant)",
  description: "Emit new event for each selected event type",
  type: "source",
  version: "1.0.11",
  dedupe: "unique",
  props: {
    docsInfo: {
      type: "alert",
      alertType: "info",
      content: "[See the GitHub documentation](https://docs.github.com/en/webhooks/webhook-events-and-payloads) for more information on available events.",
    },
    ...common.props,
    events: {
      label: "Webhook Events",
      description: "The event types to be emitted",
      type: "string[]",
      options: constants.REPOSITORY_WEBHOOK_EVENTS,
      reloadProps: true,
    },
  },
  async additionalProps() {
    await this.requireAdminPermission();
    const props = {};
    if (this.events?.length && this.events.includes("pull_request_review")) {
      props.reviewState = {
        type: "string",
        label: "Review State",
        description: "Filter `pull_request_review` events by review state",
        options: constants.PULL_REQUEST_REVIEW_STATES,
        optional: true,
      };
    }
    return props;
  },
  methods: {
    ...common.methods,
    getWebhookEvents() {
      return this.events;
    },
  },
  async run(event) {
    const {
      headers,
      body,
    } = event;

    // skip initial response from Github
    if (body?.zen) {
      console.log(body.zen);
      return;
    }

    if (headers["x-github-event"] === "pull_request_review" && this.reviewState && body.review.state !== this.reviewState) {
      return;
    }

    this.$emit({
      ...getRelevantHeaders(headers),
      ...body,
    }, {
      id: headers["x-github-delivery"],
      summary: `New event ${headers["x-github-hook-installation-target-id"]} of type ${headers["x-github-hook-installation-target-type"]}`,
      ts: new Date(),
    });
  },
  async activate() {
    const isAdmin = await this.checkAdminPermission();
    if (!isAdmin) {
      throw new Error("Webhooks are only supported on repos where you have admin access.");
    }
    await this.createWebhook();
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
GitHub	`github`	`app`	This component uses the GitHub app.
Repository	`repoFullname`	`string`	Select a value from the drop down menu.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.
Webhook Events	`events`	`string[]`	Select a value from the drop down menu:`{ "label": "Activity related to a branch protection rule", "value": "branch_protection_rule" }{ "label": "Check run activity has occurred", "value": "check_run" }{ "label": "Check suite activity has occurred", "value": "check_suite" }{ "label": "Activity related to code scanning alerts in a repository", "value": "code_scanning_alert" }{ "label": "A commit comment is created", "value": "commit_comment" }{ "label": "A Git branch or tag is created", "value": "create" }{ "label": "A Git branch or tag is deleted", "value": "delete" }{ "label": "A deploy key is added or removed from a repository", "value": "deploy_key" }{ "label": "A deployment is created", "value": "deployment_status" }{ "label": "Activity related to a discussion", "value": "discussion" }{ "label": "Activity related to a comment in a discussion", "value": "discussion_comment" }{ "label": "A user forks a repository", "value": "fork" }{ "label": "A wiki page is created or updated", "value": "gollum" }{ "label": "Activity related to an issue or pull request comment", "value": "issue_comment" }{ "label": "Activity related to an issue", "value": "issues" }{ "label": "Activity related to a label", "value": "label" }{ "label": "Activity related to repository collaborators", "value": "member" }{ "label": "The webhook this event is configured on was deleted", "value": "meta" }{ "label": "Activity related to milestones", "value": "milestone" }{ "label": "Activity related to GitHub Packages", "value": "package" }{ "label": "Activity on attempted build of a GitHub Pages site, whether successful or not", "value": "page_build" }{ "label": "Activity related to project boards", "value": "project" }{ "label": "Activity related to project cards", "value": "project_card" }{ "label": "Activity related to columns in a project board", "value": "project_column" }{ "label": "When a private repository is made public", "value": "public" }{ "label": "Activity related to pull requests", "value": "pull_request" }{ "label": "Activity related to pull request reviews", "value": "pull_request_review" }{ "label": "Activity related to pull request review comments in the pull request's unified diff", "value": "pull_request_review_comment" }{ "label": "Activity related to a comment thread on a pull request being marked as resolved or unresolved", "value": "pull_request_review_thread" }{ "label": "One or more commits are pushed to a repository branch or tag", "value": "push" }{ "label": "Activity related to a release", "value": "release" }{ "label": "Activity related to a repository", "value": "repository" }{ "label": "Activity related to a repository being imported to GitHub", "value": "repository_import" }{ "label": "Activity related to security vulnerability alerts in a repository", "value": "repository_vulnerability_alert" }{ "label": "Activity related to a repository being starred", "value": "star" }{ "label": "When the status of a Git commit changes", "value": "status" }{ "label": "When a repository is added to a team", "value": "team_add" }{ "label": "When someone stars a repository", "value": "watch" }{ "label": "A GitHub Actions workflow job has been queued, is in progress, or has been completed on a repository", "value": "workflow_job" }{ "label": "When a GitHub Actions workflow run is requested or completed", "value": "workflow_run" }`

Trigger Authentication#

GitHub uses OAuth authentication. When you connect your GitHub account, Pipedream will open a popup window where you can sign into GitHub and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any GitHub API.

Pipedream requests the following authorization scopes when you connect your account:

reporead:orgadmin:org_hookgistprojectnotificationsread:userwrite:discussionadmin:repo_hook

About GitHub#

Where the world builds software. Millions of developers and companies build, ship, and maintain their software on GitHub—the largest and most advanced development platform in the world.

Action#

Get Run Output on Databricks

Description:Retrieve the output and metadata of a single task run. [See the documentation](https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#runs-get-output)

Version:0.0.6

Key:databricks-get-run-output

View on GitHub

Databricks Overview#

The Databricks API allows you to interact programmatically with Databricks services, enabling you to manage clusters, jobs, notebooks, and other resources within Databricks environments. Through Pipedream, you can leverage these APIs to create powerful automations and integrate with other apps for enhanced data processing, transformation, and analytics workflows. This unlocks possibilities like automating cluster management, dynamically running jobs based on external triggers, and orchestrating complex data pipelines with ease.

Action Code#

import databricks from "../../databricks.app.mjs";

export default {
  key: "databricks-get-run-output",
  name: "Get Run Output",
  description: "Retrieve the output and metadata of a single task run. [See the documentation](https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#runs-get-output)",
  version: "0.0.6",
  annotations: {
    destructiveHint: false,
    openWorldHint: true,
    readOnlyHint: true,
  },
  type: "action",
  props: {
    databricks,
    jobId: {
      propDefinition: [
        databricks,
        "jobId",
      ],
      optional: true,
    },
    runId: {
      propDefinition: [
        databricks,
        "runId",
        (c) => ({
          jobId: c.jobId,
        }),
      ],
    },
  },
  async run({ $ }) {
    const response = await this.databricks.getRunOutput({
      params: {
        run_id: this.runId,
      },
      $,
    });

    if (response) {
      $.export("$summary", `Successfully retrieved output for run with ID ${this.runId}.`);
    }

    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Databricks	`databricks`	`app`	This component uses the Databricks app.
Job	`jobId`	`string`	Select a value from the drop down menu.
Run	`runId`	`string`	Select a value from the drop down menu.

Get Run Output with Databricks API on New Webhook Event (Instant) from GitHub API

Pipedream makes it easy to connect APIs for Databricks, GitHub and 2,900+ other apps remarkably fast.

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

1-24of2,900+apps by most popular

1
-
24
of
2,900+
apps by most popular