What do you want to automate
with Oxylabs and ScrapingBee?
Prompt, edit and deploy AI agents that connect to Oxylabs, ScrapingBee and 2,500+ other apps in seconds.
Trusted by 1,000,000+ developers from startups to Fortune 500 companies
Popular Oxylabs and ScrapingBee Triggers#
Emit new event when a new scheduled run is completed. See the documentation
Popular Oxylabs and ScrapingBee Actions#
Establish a proxy session using the Residential Proxy endpoint. See the documentation
Overview of Oxylabs#
The Oxylabs API provides robust tools for web data extraction, enabling you to gather vast amounts of data efficiently from various sources across the web. With Pipedream's serverless platform, you can harness this power to create automated workflows that trigger on specific events, process the data, and connect to other services. Whether you're monitoring brand sentiment, tracking prices, or conducting market research, integrating Oxylabs with Pipedream can significantly streamline your data collection and analysis tasks.
Connect Oxylabs#
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
oxylabs: {
type: "app",
app: "oxylabs",
}
},
async run({steps, $}) {
const data = {
"geo_location": `United States`,
"url": `https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html`,
"source": `universal`,
}
return await axios($, {
method: "post",
url: `https://${this.oxylabs.$auth.api_name}.oxylabs.io/v1/queries`,
headers: {
"Content-Type": `application/json`,
},
auth: {
username: `${this.oxylabs.$auth.username}`,
password: `${this.oxylabs.$auth.password}`,
},
data,
})
},
})
Overview of ScrapingBee#
The ScrapingBee API lets you extract data from web pages programmatically. It handles headless browsers and rotating proxies, so you can focus on data extraction without worrying about common web scraping issues like getting blocked. On Pipedream, you can tap into ScrapingBee's capability to create serverless workflows that scrape web data and connect them with numerous services for processing, storing, or triggering actions.
Connect ScrapingBee#
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
scrapingbee: {
type: "app",
app: "scrapingbee",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://app.scrapingbee.com/api/v1/`,
params: {
api_key: `${this.scrapingbee.$auth.api_key}`,
json_response: `True`,
url: `https://pipedream.com`,
},
})
},
})
Trusted by 1,000,000+ developers from startups to Fortune 500 companies