with Google Cloud and Puppeteer?
Get the HTML of a webpage using Puppeteer. See the documentation for details.
Get the title of a webpage using Puppeteer. See the documentation
Inserts rows into a BigQuery table. See the docs and for an example here
Captures a screenshot of a page using Puppeteer. See the documentation
The Google Cloud API opens a world of possibilities for enhancing cloud operations and automating tasks. It empowers you to manage, scale, and fine-tune various services within the Google Cloud Platform (GCP) programmatically. With Pipedream, you can harness this power to create intricate workflows, trigger cloud functions based on events from other apps, manage resources, and analyze data, all in a serverless environment. The ability to interconnect GCP services with numerous other apps enriches automation, making it easier to synchronize data, streamline development workflows, and deploy applications efficiently.
module.exports = defineComponent({
props: {
google_cloud: {
type: "app",
app: "google_cloud",
}
},
async run({steps, $}) {
// Required workaround to get the @google-cloud/storage package
// working correctly on Pipedream
require("@dylburger/umask")()
const { Storage } = require('@google-cloud/storage')
const key = JSON.parse(this.google_cloud.$auth.key_json)
const storage = new Storage({
projectId: key.project_id,
credentials: {
client_email: key.client_email,
private_key: key.private_key,
}
})
await storage.authClient.getCredentials()
return {
status: "success",
authenticated: true,
projectId: key.project_id,
serviceAccount: key.client_email
}
},
})
Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Puppeteer runs in headless mode on Chromium on Pipedream.
Using Puppeteer you can perform tasks including:
import { puppeteer } from '@pipedream/browsers';
export default defineComponent({
async run({steps, $}) {
const browser = await puppeteer.browser();
// Interact with the web page programmatically
// See Puppeeter's Page documentation for available methods:
// https://pptr.dev/api/puppeteer.page
const page = await browser.newPage();
await page.goto('https://pipedream.com/');
const title = await page.title();
const content = await page.content();
// The browser needs to be closed, otherwise the step will hang
await browser.close();
return { title, content }
},
})