The easiest way to scrape websites via #API. ScrapingAnt uses the latest Chrome browser and rotates proxies to automate your data mining tasks.
Emit new event when you add or modify a new row in a table. See the docs here
Emit new event when new rows are returned from a custom query. See the docs here
Emit new event when a new table is added to a database. See the docs here
The ScrapingAnt API allows you to scrape web pages without getting blocked. It can handle JavaScript rendering, cookies, sessions, and can even interact with web pages as if a real person were browsing. Using Pipedream, you can integrate ScrapingAnt with countless other apps to automate data extraction and feed this data into various business processes, analytics tools, or databases.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
scrapingant: {
type: "app",
app: "scrapingant",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.scrapingant.com/v2/general`,
headers: {
"x-api-key": `${this.scrapingant.$auth.api_token}`,
},
params: {
url: `pipedream.com`,
},
})
},
})
The MySQL application on Pipedream enables direct interaction with your MySQL databases, allowing you to perform CRUD operations—create, read, update, delete—on your data with ease. You can leverage these capabilities to automate data synchronization, report generation, and event-based triggers that kick off workflows in other apps. With Pipedream's serverless platform, you can connect MySQL to hundreds of other services without managing infrastructure, crafting complex code, or handling authentication.
import mysql from '@pipedream/mysql';
export default defineComponent({
props: {
mysql,
},
async run({steps, $}) {
// Component source code:
// https://github.com/PipedreamHQ/pipedream/tree/master/components/mysql
const queryObj = {
sql: "SELECT NOW()",
values: [], // Ignored since query does not contain placeholders
};
return await this.mysql.executeQuery(queryObj);
},
});