Making web data extraction easy and accessible for everyone.
Emit new event when a page scraping job has completed. See the docs here
Creates a scraping job (scrapes a sitemap). See the docs here
Create a HTML or a plain text email template. See the docs
Creates a sitemap for the selected website. See the docs here
Retrieves a list of scraping jobs for a sitemap. See the docs here
The WebScraper.IO API allows you to programmatically perform web scraping tasks, extracting structured data from websites. With the API, you can automate the gathering of web content for analysis, monitoring, and integration with other data sources. In Pipedream, you can leverage this API to build workflows that process, analyze, and act on the data you scrape without writing code for backend infrastructure.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
webscraper_io: {
type: "app",
app: "webscraper_io",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.webscraper.io/api/v1/sitemaps`,
params: {
api_token: `${this.webscraper_io.$auth.api_key}`,
},
})
},
})
Amazon Simple Email Service (SES) is a powerful cloud-based email sending service designed to help digital marketers and application developers send marketing, notification, and transactional emails. With the SES API, you can reliably send emails at scale, manage sender reputations, view email sending statistics, and maintain a high deliverability rate. Leveraging Pipedream's capabilities, you can integrate SES seamlessly into serverless workflows, automate email responses based on triggers from other apps, and analyze the effectiveness of your email campaigns by connecting to data analytics platforms.
module.exports = defineComponent({
props: {
amazon_ses: {
type: "app",
app: "amazon_ses",
}
},
async run({steps, $}) {
const AWS = require("aws-sdk")
const { accessKeyId, secretAccessKey } = this.amazon_ses.$auth
const ses = new AWS.SES({
accessKeyId,
secretAccessKey,
region: 'us-east-1',
})
const sesParams = {
Destination: {
ToAddresses: ["<your email here>"],
},
Message: {
Body: {
Html: {
Charset: "UTF-8",
Data: "<h1>This is a test</h1>",
},
Text: {
Charset: "UTF-8",
Data: "This is a test",
}
},
Subject: {
Charset: "UTF-8",
Data: "Test email",
}
},
Source: "<your from address here",
};
this.resp = await ses.sendEmail(sesParams).promise()
},
})