with Oxylabs and ParseHub?
Emit new event when a new scheduled run is completed. See the documentation
Establish a proxy session using the Residential Proxy endpoint. See the documentation
Cancels a run and changes its status to cancelled. See the documentation
Cancels a run if running, and deletes the run and its data. See the documentation
Returns the data extracted by a specified run. See the documentation
The Oxylabs API provides robust tools for web data extraction, enabling you to gather vast amounts of data efficiently from various sources across the web. With Pipedream's serverless platform, you can harness this power to create automated workflows that trigger on specific events, process the data, and connect to other services. Whether you're monitoring brand sentiment, tracking prices, or conducting market research, integrating Oxylabs with Pipedream can significantly streamline your data collection and analysis tasks.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
oxylabs: {
type: "app",
app: "oxylabs",
}
},
async run({steps, $}) {
const data = {
"geo_location": `United States`,
"url": `https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html`,
"source": `universal`,
}
return await axios($, {
method: "post",
url: `https://${this.oxylabs.$auth.api_name}.oxylabs.io/v1/queries`,
headers: {
"Content-Type": `application/json`,
},
auth: {
username: `${this.oxylabs.$auth.username}`,
password: `${this.oxylabs.$auth.password}`,
},
data,
})
},
})
The ParseHub API allows you to leverage the power of web scraping directly within Pipedream. By integrating ParseHub, you can automate the collection of data from web pages, manipulate and transform it with Pipedream’s built-in code steps or pre-built actions, and connect it to hundreds of other apps. You can extract structured data from any website, run scraping jobs, retrieve results and integrate with other services for data processing, visualization, or storage.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
parsehub: {
type: "app",
app: "parsehub",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://www.parsehub.com/api/v2/projects`,
params: {
api_key: `${this.parsehub.$auth.api_key}`,
},
})
},
})