with Databricks and Neon Postgres?
Emit new event when a new column is added to a table. See the documentation
Emit new event when a row is added or modified. See the documentation
Emit new event when a new row is added to a table. See the documentation
Emit new event when new rows are returned from a custom query that you provide. See the documentation
Emit new event when a new table is added to the database. See the documentation
Retrieve the output and metadata of a single task run. See the documentation
Run a job now and return the id of the triggered run. See the documentation
The Databricks API allows you to interact programmatically with Databricks services, enabling you to manage clusters, jobs, notebooks, and other resources within Databricks environments. Through Pipedream, you can leverage these APIs to create powerful automations and integrate with other apps for enhanced data processing, transformation, and analytics workflows. This unlocks possibilities like automating cluster management, dynamically running jobs based on external triggers, and orchestrating complex data pipelines with ease.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
databricks: {
type: "app",
app: "databricks",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://${this.databricks.$auth.domain}.cloud.databricks.com/api/2.0/preview/scim/v2/Me`,
headers: {
Authorization: `Bearer ${this.databricks.$auth.access_token}`,
},
})
},
})
import { axios } from "@pipedream/platform";
import { neon } from "@neondatabase/serverless";
export default defineComponent({
props: {
neon_postgres: {
type: "app",
app: "neon_postgres",
}
},
async run({ steps, $ }) {
const { host, port, user, password, database } = this.neon_postgres.$auth
const connectionString = `postgresql://${user}:${password}@${host}:${port}/${database}`
const sql = neon(connectionString)
const result = await sql`SELECT version()`
const { version } = result[0]
return version
},
})