Edit SQL Warehouse with Databricks API

Pipedream makes it easy to connect APIs for Databricks and 2,800+ other apps remarkably fast.

Trigger workflow on

HTTP requests, schedules and app events

Next, do this

Edit SQL Warehouse with the Databricks API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

Create a workflow to Edit SQL Warehouse with the Databricks API. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Configure the Edit SQL Warehouse action
1. Connect your Databricks account
2. Select a Warehouse ID
3. Optional- Configure Warehouse Name
4. Optional- Select a Cluster Size
5. Optional- Configure Auto Stop (minutes)
6. Optional- Configure Min Number of Clusters
7. Optional- Configure Max Number of Clusters
8. Optional- Configure Enable Photon
9. Optional- Configure Enable Serverless Compute
10. Optional- Select a Warehouse Type
11. Optional- Select a Spot Instance Policy
12. Optional- Configure Tags
13. Optional- Configure Channel
14. Optional- Configure Instance Profile ARN (Deprecated)
Select a trigger to run your workflow on HTTP requests, schedules or app events
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Integrations#

Edit SQL Warehouse with Databricks API on New Requests (Payload Only) from HTTP / Webhook API

HTTP / Webhook + Databricks

Try it

Edit SQL Warehouse with Databricks API on New Submission from Typeform API

Typeform + Databricks

Try it

Edit SQL Warehouse with Databricks API on New Submission (Instant) from Jotform API

Jotform + Databricks

Try it

Edit SQL Warehouse with Databricks API on New Scheduled Tasks from Pipedream API

Pipedream + Databricks

Try it

Edit SQL Warehouse with Databricks API on New Download Counts from npm API

npm + Databricks

Try it

Details#

This is a pre-built, source-available component from Pipedream's GitHub repo. The component is developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Edit SQL Warehouse on Databricks

Description:Edits the configuration of an existing SQL Warehouse. [See the documentation](https://docs.databricks.com/api/workspace/warehouses/edit)

Version:0.0.1

Key:databricks-edit-sql-warehouse

View on GitHub

Code#

import databricks from "../../databricks.app.mjs";
import constants from "../../common/constants.mjs";
import utils from "../../common/utils.mjs";
import { ConfigurationError } from "@pipedream/platform";

export default {
  key: "databricks-edit-sql-warehouse",
  name: "Edit SQL Warehouse",
  description: "Edits the configuration of an existing SQL Warehouse. [See the documentation](https://docs.databricks.com/api/workspace/warehouses/edit)",
  version: "0.0.1",
  type: "action",
  props: {
    databricks,
    warehouseId: {
      description: "The ID of the SQL Warehouse to edit",
      propDefinition: [
        databricks,
        "warehouseId",
      ],
    },
    name: {
      type: "string",
      label: "Warehouse Name",
      description: "Logical name for the warehouse. Must be unique within an org and under 100 characters.",
      optional: true,
    },
    clusterSize: {
      type: "string",
      label: "Cluster Size",
      description: "Size of clusters allocated for this warehouse.",
      options: constants.CLUSTER_SIZES,
      optional: true,
    },
    autoStopMins: {
      type: "integer",
      label: "Auto Stop (minutes)",
      description: "Minutes of inactivity before auto-stop. 0 disables autostop. Must be 0 or ≥ 10.",
      optional: true,
    },
    minNumClusters: {
      type: "integer",
      label: "Min Number of Clusters",
      description: "Minimum number of available clusters (> 0 and ≤ min(max_num_clusters, 30)).",
      optional: true,
    },
    maxNumClusters: {
      type: "integer",
      label: "Max Number of Clusters",
      description: "Maximum number of clusters for autoscaler (≥ min_num_clusters and ≤ 30).",
      optional: true,
    },
    enablePhoton: {
      type: "boolean",
      label: "Enable Photon",
      description: "Use Photon optimized clusters.",
      optional: true,
    },
    enableServerlessCompute: {
      type: "boolean",
      label: "Enable Serverless Compute",
      description: "Use serverless compute for this warehouse.",
      optional: true,
    },
    warehouseType: {
      type: "string",
      label: "Warehouse Type",
      description: "Set to PRO (recommended) or CLASSIC. Set PRO + enable serverless to use serverless.",
      options: [
        "TYPE_UNSPECIFIED",
        "CLASSIC",
        "PRO",
      ],
      optional: true,
    },
    spotInstancePolicy: {
      type: "string",
      label: "Spot Instance Policy",
      description: "Whether the warehouse should use spot instances.",
      options: [
        "POLICY_UNSPECIFIED",
        "COST_OPTIMIZED",
        "RELIABILITY_OPTIMIZED",
      ],
      optional: true,
    },
    tags: {
      type: "object",
      label: "Tags",
      description: "Key-value tags for all resources associated with this warehouse (fewer than 45 tags).",
      optional: true,
    },
    channel: {
      type: "object",
      label: "Channel",
      description: "Channel details. Example: `{ \"name\": \"CHANNEL_NAME_CUSTOM\", \"dbsql_version\": \"2023.35\" }`",
      optional: true,
    },
    instanceProfileArn: {
      type: "string",
      label: "Instance Profile ARN (Deprecated)",
      description: "Deprecated. Instance profile used to pass IAM role to the cluster.",
      optional: true,
    },
  },
  async run({ $ }) {
    const payload = {};

    if (this.name !== undefined) {
      if (typeof this.name !== "string" || this.name.length >= 100) {
        throw new ConfigurationError("name must be a string with fewer than 100 characters.");
      }
      payload.name = this.name;
    }
    if (this.clusterSize !== undefined) payload.cluster_size = this.clusterSize;

    if (this.autoStopMins !== undefined) {
      if (this.autoStopMins !== 0 && this.autoStopMins < 10) {
        throw new ConfigurationError("autoStopMins must be 0 or >= 10.");
      }
      payload.auto_stop_mins = this.autoStopMins;
    }

    if (this.minNumClusters !== undefined) {
      if (this.minNumClusters < 1 || this.minNumClusters > 30) {
        throw new ConfigurationError("minNumClusters must be between 1 and 30.");
      }
      payload.min_num_clusters = this.minNumClusters;
    }

    if (this.maxNumClusters !== undefined) {
      if (this.maxNumClusters < 1 || this.maxNumClusters > 30) {
        throw new ConfigurationError("maxNumClusters must be between 1 and 30.");
      }
      if (this.minNumClusters !== undefined && this.maxNumClusters < this.minNumClusters) {
        throw new ConfigurationError("maxNumClusters must be >= minNumClusters.");
      }
      payload.max_num_clusters = this.maxNumClusters;
    }

    if (this.enablePhoton !== undefined) payload.enable_photon = this.enablePhoton;
    if (this.enableServerlessCompute !== undefined) {
      if (this.warehouseType === "CLASSIC" && this.enableServerlessCompute) {
        throw new ConfigurationError("Serverless compute requires warehouseType = PRO.");
      }
      payload.enable_serverless_compute = this.enableServerlessCompute;
    }

    const parsedTags = utils.parseObject(this.tags);
    const tagArray = Object.entries(parsedTags).map(([
      key,
      value,
    ]) => ({
      key,
      value,
    }));
    if (tagArray.length) {
      payload.tags = {
        custom_tags: tagArray,
      };
    }
    if (this.warehouseType !== undefined) payload.warehouse_type = this.warehouseType;
    if (this.spotInstancePolicy !== undefined) {
      payload.spot_instance_policy = this.spotInstancePolicy;
    }
    if (this.channel !== undefined) payload.channel = utils.parseObject(this.channel);
    if (this.instanceProfileArn !== undefined) {
      payload.instance_profile_arn = this.instanceProfileArn;
    }

    if (!Object.keys(payload).length) {
      throw new ConfigurationError("No fields to update. Provide at least one property.");
    }

    const response = await this.databricks.editSQLWarehouse({
      warehouseId: this.warehouseId,
      data: payload,
      $,
    });

    $.export("$summary", `Successfully edited SQL Warehouse ID ${this.warehouseId}`);
    return response;
  },
};

Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Databricks	`databricks`	`app`	This component uses the Databricks app.
Warehouse ID	`warehouseId`	`string`	Select a value from the drop down menu.
Warehouse Name	`name`	`string`	Logical name for the warehouse. Must be unique within an org and under 100 characters.
Cluster Size	`clusterSize`	`string`	Select a value from the drop down menu:`2X-SmallX-SmallSmallMediumLargeX-Large2X-Large3X-Large4X-Large`
Auto Stop (minutes)	`autoStopMins`	`integer`	Minutes of inactivity before auto-stop. 0 disables autostop. Must be 0 or ≥ 10.
Min Number of Clusters	`minNumClusters`	`integer`	Minimum number of available clusters (> 0 and ≤ min(max_num_clusters, 30)).
Max Number of Clusters	`maxNumClusters`	`integer`	Maximum number of clusters for autoscaler (≥ min_num_clusters and ≤ 30).
Enable Photon	`enablePhoton`	`boolean`	Use Photon optimized clusters.
Enable Serverless Compute	`enableServerlessCompute`	`boolean`	Use serverless compute for this warehouse.
Warehouse Type	`warehouseType`	`string`	Select a value from the drop down menu:`TYPE_UNSPECIFIEDCLASSICPRO`
Spot Instance Policy	`spotInstancePolicy`	`string`	Select a value from the drop down menu:`POLICY_UNSPECIFIEDCOST_OPTIMIZEDRELIABILITY_OPTIMIZED`
Tags	`tags`	`object`	Key-value tags for all resources associated with this warehouse (fewer than 45 tags).
Channel	`channel`	`object`	Channel details. Example: `{ "name": "CHANNEL_NAME_CUSTOM", "dbsql_version": "2023.35" }`
Instance Profile ARN (Deprecated)	`instanceProfileArn`	`string`	Deprecated. Instance profile used to pass IAM role to the cluster.