Databricks
Databricks is the lakehouse company, helping data teams solve the world’s toughest problems.
Integrate the Databricks API with the Filter API
Setup the Databricks API trigger to run a workflow which integrates with the Filter API. Pipedream's integration platform allows you to integrate Databricks and Filter remarkably fast. Free for developers.
Connect Databricks#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
databricks: {
type: "app",
app: "databricks",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://${this.databricks.$auth.domain}.cloud.databricks.com/api/2.0/clusters/list`,
headers: {
Authorization: `Bearer ${this.databricks.$auth.access_token}`,
},
})
},
})
Overview of Filter#
The Filter API is a great way to build powerful workflows that can manipulate
and transform data. Here are some examples of what you can build using the
Filter API:
- A workflow that transforms data from one format to another
- A workflow that filters out data that does not meet certain criteria
- A workflow that sorts data in a specific order
- A workflow that calculates statistics or aggregates data
- A workflow that combines data from multiple sources
Connect Filter#
1
2
3
4
5
6
7
8
9
10
export default defineComponent({
async run({ steps, $ }) {
let condition = false
if (condition == false) {
$.flow.exit("Ending workflow early because the condition is false")
} else {
$.export("$summary", "Continuing workflow, since condition for ending was not met.")
}
},
})