What do you want to automate

with Oxylabs and FireCrawl?

Prompt, edit and deploy AI agents that connect to Oxylabs, FireCrawl and 2,500+ other apps in seconds.

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Adyen logo
Appcues logo
Bandwidth logo
Checkr logo
ChartMogul logo
Dataminr logo
Gopuff logo
Gorgias logo
LinkedIn logo
Logitech logo
Replicated logo
Rudderstack logo
SAS logo
Scale AI logo
Webflow logo
Warner Bros. logo
Adyen logo
Appcues logo
Bandwidth logo
Checkr logo
ChartMogul logo
Dataminr logo
Gopuff logo
Gorgias logo
LinkedIn logo
Logitech logo
Replicated logo
Rudderstack logo
SAS logo
Scale AI logo
Webflow logo
Warner Bros. logo
Crawl URL with FireCrawl API on New Scheduled Run Completed from Oxylabs API
Oxylabs + FireCrawl
 
Try it
Extract Data with FireCrawl API on New Scheduled Run Completed from Oxylabs API
Oxylabs + FireCrawl
 
Try it
Get Crawl Data with FireCrawl API on New Scheduled Run Completed from Oxylabs API
Oxylabs + FireCrawl
 
Try it
Get Extract Data with FireCrawl API on New Scheduled Run Completed from Oxylabs API
Oxylabs + FireCrawl
 
Try it
Map URL with FireCrawl API on New Scheduled Run Completed from Oxylabs API
Oxylabs + FireCrawl
 
Try it
New Scheduled Run Completed from the Oxylabs API

Emit new event when a new scheduled run is completed. See the documentation

 
Try it
Create Proxy Session with the Oxylabs API

Establish a proxy session using the Residential Proxy endpoint. See the documentation

 
Try it
Crawl URL with the FireCrawl API

Crawls a given URL and returns the contents of sub-pages. See the documentation

 
Try it
Create Schedule with the Oxylabs API

Create a schedule for a scraping job. See the documentation

 
Try it
Extract Data with the FireCrawl API

Extract structured data from one or multiple URLs. See the documentation

 
Try it
Get Crawl Data with the FireCrawl API

Obtains the status and data from a previous crawl operation. See the documentation

 
Try it
Integrate the Oxylabs API with the FireCrawl API
Setup the Oxylabs API trigger to run a workflow which integrates with the FireCrawl API. Pipedream's integration platform allows you to integrate Oxylabs and FireCrawl remarkably fast. Free for developers.

Overview of Oxylabs

The Oxylabs API provides robust tools for web data extraction, enabling you to gather vast amounts of data efficiently from various sources across the web. With Pipedream's serverless platform, you can harness this power to create automated workflows that trigger on specific events, process the data, and connect to other services. Whether you're monitoring brand sentiment, tracking prices, or conducting market research, integrating Oxylabs with Pipedream can significantly streamline your data collection and analysis tasks.

Connect Oxylabs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import { axios } from "@pipedream/platform"
export default defineComponent({
  props: {
    oxylabs: {
      type: "app",
      app: "oxylabs",
    }
  },
  async run({steps, $}) {
    const data = {
      "geo_location": `United States`,
      "url": `https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html`,
      "source": `universal`,
    }
    return await axios($, {
      method: "post",
      url: `https://${this.oxylabs.$auth.api_name}.oxylabs.io/v1/queries`,
      headers: {
        "Content-Type": `application/json`,
      },
      auth: {
        username: `${this.oxylabs.$auth.username}`,
        password: `${this.oxylabs.$auth.password}`,
      },
      data,
    })
  },
})

Connect FireCrawl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
import { axios } from "@pipedream/platform"
export default defineComponent({
  props: {
    firecrawl: {
      type: "app",
      app: "firecrawl",
    }
  },
  async run({steps, $}) {
    const data = {
      "url": "https://pipedream.com",
    }
    return await axios($, {
      method: "post",
      url: `https://api.firecrawl.dev/v0/crawl`,
      headers: {
        Authorization: `Bearer ${this.firecrawl.$auth.api_key}`,
      },
      data,
    })
  },
})

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Adyen logo
Appcues logo
Bandwidth logo
Checkr logo
ChartMogul logo
Dataminr logo
Gopuff logo
Gorgias logo
LinkedIn logo
Logitech logo
Replicated logo
Rudderstack logo
SAS logo
Scale AI logo
Webflow logo
Warner Bros. logo
Adyen logo
Appcues logo
Bandwidth logo
Checkr logo
ChartMogul logo
Dataminr logo
Gopuff logo
Gorgias logo
LinkedIn logo
Logitech logo
Replicated logo
Rudderstack logo
SAS logo
Scale AI logo
Webflow logo
Warner Bros. logo