with Databricks and Playwright?
Retrieve the output and metadata of a single task run. See the documentation
Run a job now and return the id of the triggered run. See the documentation
The Databricks API allows you to interact programmatically with Databricks services, enabling you to manage clusters, jobs, notebooks, and other resources within Databricks environments. Through Pipedream, you can leverage these APIs to create powerful automations and integrate with other apps for enhanced data processing, transformation, and analytics workflows. This unlocks possibilities like automating cluster management, dynamically running jobs based on external triggers, and orchestrating complex data pipelines with ease.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
databricks: {
type: "app",
app: "databricks",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://${this.databricks.$auth.domain}.cloud.databricks.com/api/2.0/clusters/list`,
headers: {
Authorization: `Bearer ${this.databricks.$auth.access_token}`,
},
})
},
})
Playwright is a Node.js library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Playwright runs in headless mode on Chromium on Pipedream.
Using Playwright you can perform tasks including:
import { playwright } from '@pipedream/browsers';
export default defineComponent({
async run({steps, $}) {
const browser = await playwright.launch();
// Interact with the web page programmatically
// See Playwright's Page documentation for available methods:
// https://playwright.dev/docs/api/class-page
const page = await browser.newPage();
await page.goto('https://pipedream.com/');
const title = await page.title();
const content = await page.content();
// Close context and browser otherwise the step will hang
await page.context().close()
await browser.close();
return { title, content }
},
})