Run Job Now with Databricks API on New Event from Jira API

Pipedream makes it easy to connect APIs for Databricks, Jira and 2,800+ other apps remarkably fast.

Trigger workflow on

New Event from the Jira API

Next, do this

Run Job Now with the Databricks API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Jira trigger and Databricks action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Event trigger
1. Connect your Jira account
2. Select a Cloud ID
3. Configure JQL Filter
4. Optional- Configure Override Existing Webhooks
5. Select one or more Event types
Configure the Run Job Now action
1. Connect your Databricks account
2. Select a Job
3. Optional- Configure JAR Params
4. Optional- Configure Notebook Params
5. Optional- Configure Python Params
6. Optional- Configure Spark Submit Params
7. Optional- Configure Idempotency Token
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Event on Jira

Description:Emit new event when an event with subscribed event source triggered, [See the docs](https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-webhooks/#api-rest-api-3-webhook-post)

Version:0.0.13

Key:jira-events

View on GitHub

Jira Overview#

The Jira API opens up a world of possibilities for automating project management tasks, syncing with other tools, and enhancing data visibility. With Pipedream's integration, you can streamline issue tracking by automatically creating, updating, and searching for issues in Jira, as well as managing projects, sprints, users, and more. This integration not only saves time but also ensures real-time data flow across various platforms, keeping teams in sync and projects on track.

Trigger Code#

import common from "../common/common.mjs";
import eventsTypes from "../common/eventTypes.mjs";

export default {
  key: "jira-events",
  name: "New Event",
  description: "Emit new event when an event with subscribed event source triggered, [See the docs](https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-webhooks/#api-rest-api-3-webhook-post)",
  version: "0.0.13",
  type: "source",
  dedupe: "unique",
  ...common,
  props: {
    ...common.props,
    events: {
      type: "string[]",
      label: "Event types",
      description: "he Jira events that trigger the webhook.",
      options: eventsTypes,
    },
  },
  methods: {
    ...common.methods,
    getEvents() {
      return this.events;
    },
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Jira	`jira`	`app`	This component uses the Jira app.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
Cloud ID	`cloudId`	`string`	Select a value from the drop down menu.
JQL Filter	`jqlFilter`	`string`	The JQL filter that specifies which issues the webhook is sent for, only a subset of JQL can be used, e.g. `project = P1` See supported JQL filters
Override Existing Webhooks	`overrideExistingWebhooks`	`boolean`	Override existing webhooks with this new Pipedream source's webhook. Recommend to set this to `true` if you have an existing Jira webhook that you no longer use and want to override with the new Pipedream source.
Event types	`events`	`string[]`	Select a value from the drop down menu:`{ "label": "Issue Created", "value": "jira:issue_created" }{ "label": "Issue Updated", "value": "jira:issue_updated" }{ "label": "Issue Deleted", "value": "jira:issue_deleted" }`

Trigger Authentication#

Jira uses OAuth authentication. When you connect your Jira account, Pipedream will open a popup window where you can sign into Jira and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Jira API.

Pipedream requests the following authorization scopes when you connect your account:

read:jira-workmanage:jira-projectread:jira-userwrite:jira-workmanage:jira-data-providerread:meoffline_accessread:issue-details:jiraread:board-scope:jira-softwareread:webhook:jirawrite:webhook:jiraread:field:jiraread:project:jiradelete:webhook:jirawrite:board-scope:jira-softwareread:board-scope.admin:jira-softwareread:sprint:jira-softwarewrite:sprint:jira-softwareread:field.option:jirawrite:field.option:jiraread:custom-field-contextual-configuration:jira write:custom-field-contextual-configuration:jiramanage:jira-configurationread:jql:jira

About Jira#

Jira is the #1 agile project management tool used by teams to plan, track, release, and support great software with confidence

Action#

Run Job Now on Databricks

Description:Run a job now and return the id of the triggered run. [See the documentation](https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#runs-list)

Version:0.0.5

Key:databricks-run-job-now

View on GitHub

Databricks Overview#

The Databricks API allows you to interact programmatically with Databricks services, enabling you to manage clusters, jobs, notebooks, and other resources within Databricks environments. Through Pipedream, you can leverage these APIs to create powerful automations and integrate with other apps for enhanced data processing, transformation, and analytics workflows. This unlocks possibilities like automating cluster management, dynamically running jobs based on external triggers, and orchestrating complex data pipelines with ease.

Action Code#

import databricks from "../../databricks.app.mjs";

export default {
  key: "databricks-run-job-now",
  name: "Run Job Now",
  description: "Run a job now and return the id of the triggered run. [See the documentation](https://docs.databricks.com/en/workflows/jobs/jobs-2.0-api.html#runs-list)",
  version: "0.0.5",
  type: "action",
  props: {
    databricks,
    jobId: {
      propDefinition: [
        databricks,
        "jobId",
      ],
    },
    jarParams: {
      type: "string[]",
      label: "JAR Params",
      description: "A list of parameters for jobs with JAR tasks, e.g. \"jar_params\": [\"john doe\", \"35\"]. The parameters will be used to invoke the main function of the main class specified in the Spark JAR task. If not specified upon run-now, it will default to an empty list. jar_params cannot be specified in conjunction with notebook_params. The JSON representation of this field (i.e. {\"jar_params\":[\"john doe\",\"35\"]}) cannot exceed 10,000 bytes.",
      optional: true,
    },
    notebookParams: {
      type: "object",
      label: "Notebook Params",
      description: "A map from keys to values for jobs with notebook task, e.g. \"notebook_params\": {\"name\": \"john doe\", \"age\":  \"35\"}. The map is passed to the notebook and is accessible through the dbutils.widgets.get function. If not specified upon run-now, the triggered run uses the job’s base parameters. You cannot specify notebook_params in conjunction with jar_params. The JSON representation of this field (i.e. {\"notebook_params\":{\"name\":\"john doe\",\"age\":\"35\"}}) cannot exceed 10,000 bytes.",
      optional: true,
    },
    pythonParams: {
      type: "string[]",
      label: "Python Params",
      description: "A list of parameters for jobs with Python tasks, e.g. \"python_params\": [\"john doe\", \"35\"]. The parameters will be passed to Python file as command-line parameters. If specified upon run-now, it would overwrite the parameters specified in job setting. The JSON representation of this field (i.e. {\"python_params\":[\"john doe\",\"35\"]}) cannot exceed 10,000 bytes.",
      optional: true,
    },
    sparkSubmitParams: {
      type: "string[]",
      label: "Spark Submit Params",
      description: "A list of parameters for jobs with spark submit task, e.g. \"spark_submit_params\": [\"--class\", \"org.apache.spark.examples.SparkPi\"]. The parameters will be passed to spark-submit script as command-line parameters. If specified upon run-now, it would overwrite the parameters specified in job setting. The JSON representation of this field cannot exceed 10,000 bytes.",
      optional: true,
    },
    idempotencyToken: {
      type: "string",
      label: "Idempotency Token",
      description: "An optional token to guarantee the idempotency of job run requests. If a run with the provided token already exists, the request does not create a new run but returns the ID of the existing run instead. If a run with the provided token is deleted, an error is returned. If you specify the idempotency token, upon failure you can retry until the request succeeds. Databricks guarantees that exactly one run is launched with that idempotency token. This token must have at most 64 characters. For more information, see [How to ensure idempotency for jobs](https://kb.databricks.com/jobs/jobs-idempotency.html).",
      optional: true,
    },
  },
  async run({ $ }) {
    const response = await this.databricks.runJobNow({
      data: {
        job_id: this.jobId,
        jar_params: this.jarParams,
        notebook_params: this.notebookParams,
        python_params: this.pythonParams,
        spark_submit_params: this.sparkSubmitParams,
        idempotency_token: this.idempotencyToken,
      },
      $,
    });

    if (response) {
      $.export("$summary", `Successfully initiated job with ID ${this.jobId}.`);
    }

    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Databricks	`databricks`	`app`	This component uses the Databricks app.
Job	`jobId`	`string`	Select a value from the drop down menu.
JAR Params	`jarParams`	`string[]`	A list of parameters for jobs with JAR tasks, e.g. "jar_params": ["john doe", "35"]. The parameters will be used to invoke the main function of the main class specified in the Spark JAR task. If not specified upon run-now, it will default to an empty list. jar_params cannot be specified in conjunction with notebook_params. The JSON representation of this field (i.e. {"jar_params":["john doe","35"]}) cannot exceed 10,000 bytes.
Notebook Params	`notebookParams`	`object`	A map from keys to values for jobs with notebook task, e.g. "notebook_params": {"name": "john doe", "age": "35"}. The map is passed to the notebook and is accessible through the dbutils.widgets.get function. If not specified upon run-now, the triggered run uses the job’s base parameters. You cannot specify notebook_params in conjunction with jar_params. The JSON representation of this field (i.e. {"notebook_params":{"name":"john doe","age":"35"}}) cannot exceed 10,000 bytes.
Python Params	`pythonParams`	`string[]`	A list of parameters for jobs with Python tasks, e.g. "python_params": ["john doe", "35"]. The parameters will be passed to Python file as command-line parameters. If specified upon run-now, it would overwrite the parameters specified in job setting. The JSON representation of this field (i.e. {"python_params":["john doe","35"]}) cannot exceed 10,000 bytes.
Spark Submit Params	`sparkSubmitParams`	`string[]`	A list of parameters for jobs with spark submit task, e.g. "spark_submit_params": ["--class", "org.apache.spark.examples.SparkPi"]. The parameters will be passed to spark-submit script as command-line parameters. If specified upon run-now, it would overwrite the parameters specified in job setting. The JSON representation of this field cannot exceed 10,000 bytes.
Idempotency Token	`idempotencyToken`	`string`	An optional token to guarantee the idempotency of job run requests. If a run with the provided token already exists, the request does not create a new run but returns the ID of the existing run instead. If a run with the provided token is deleted, an error is returned. If you specify the idempotency token, upon failure you can retry until the request succeeds. Databricks guarantees that exactly one run is launched with that idempotency token. This token must have at most 64 characters. For more information, see How to ensure idempotency for jobs

Action Authentication#

Databricks uses API keys for authentication. When you connect your Databricks account, Pipedream securely stores the keys so you can easily authenticate to Databricks APIs in both code and no-code steps.

About Databricks#

Databricks is the lakehouse company, helping data teams solve the world’s toughest problems.

More Ways to Connect Databricks + Jira#

Other Popular Integrations#

List Runs with Databricks API on New Event from Jira API

Jira + Databricks

Try it

List Runs with Databricks API on New Issue Created Event from Jira API

Jira + Databricks

Try it

List Runs with Databricks API on New Issue Deleted Event from Jira API

Jira + Databricks

Try it

List Runs with Databricks API on New Issue Updated Event from Jira API

Jira + Databricks

Try it

Get Run Output with Databricks API on New Event from Jira API

Jira + Databricks

Try it

Popular Triggers#

New Event from the Jira API

Emit new event when an event with subscribed event source triggered, See the docs

Try it

New Issue Created Event (Instant) from the Jira API

Emit new event when an issue is created. Note that Jira supports only one webhook, if more sources are needed please use New Event source and select multiple events.

Try it

New Issue Deleted Event (Instant) from the Jira API

Emit new event when an issue is deleted. Note that Jira supports only one webhook, if more sources are needed please use New Event source and select multiple events.

Try it

New Issue Updated Event (Instant) from the Jira API

Emit new event when an issue is updated. Note that Jira supports only one webhook, if more sources are needed please use New Event source and select multiple events.

Try it

Popular Actions#

Add Attachment To Issue with the Jira API

Adds an attachment to an issue, See the docs

Try it

Add Comment To Issue with the Jira API

Adds a new comment to an issue, See the docs

Try it

Add Multiple Attachments To Issue with the Jira API

Adds multiple attachments to an issue, See the docs

Try it

Add Watcher To Issue with the Jira API

Adds a user as a watcher of an issue by passing the account ID of the user, For example, 5b10ac8d82e05b22cc7d4ef5, If no user is specified the calling user is added. See the docs

Try it

Assign Issue with the Jira API

Assigns an issue to a user. See the docs

Try it