with DigitalOcean Spaces and Puppeteer?
Get the HTML of a webpage using Puppeteer. See the documentation for details.
Get the title of a webpage using Puppeteer. See the documentation
DigitalOcean Spaces API permits you to manage object storage, allowing for the storage and serving of massive amounts of data. This API is great for backing up, archiving, and providing public access to data or assets. On Pipedream, you can use this API to automate file operations like uploads, downloads, and deletions, as well as manage permissions and metadata. You can integrate it with other services for end-to-end workflow automation.
import { S3 } from "@aws-sdk/client-s3";
import { ListBucketsCommand } from "@aws-sdk/client-s3";
export default defineComponent({
props: {
digitalocean_spaces: {
type: "app",
app: "digitalocean_spaces"
}
},
async run({ steps, $ }) {
console.log(this.digitalocean_spaces.$auth)
const s3Client = new S3({
forcePathStyle: false, // Configures to use subdomain/virtual calling format.
endpoint: `https://${this.digitalocean_spaces.$auth.region}.digitaloceanspaces.com`,
region: "us-east-1",
credentials: {
accessKeyId: this.digitalocean_spaces.$auth.key,
secretAccessKey: this.digitalocean_spaces.$auth.secret
}
});
const data = await s3Client.send(new ListBucketsCommand({}));
return data.Buckets;
},
})
Puppeteer is a Node.js library which provides a high-level API to control Chrome/Chromium over the DevTools Protocol. Puppeteer runs in headless mode on Chromium on Pipedream.
Using Puppeteer you can perform tasks including:
import { puppeteer } from '@pipedream/browsers';
export default defineComponent({
async run({steps, $}) {
const browser = await puppeteer.browser();
// Interact with the web page programmatically
// See Puppeeter's Page documentation for available methods:
// https://pptr.dev/api/puppeteer.page
const page = await browser.newPage();
await page.goto('https://pipedream.com/');
const title = await page.title();
const content = await page.content();
// The browser needs to be closed, otherwise the step will hang
await browser.close();
return { title, content }
},
})