WebScraping.AI

Simple and powerful web scraping API with automated proxies rotation and Chrome JS rendering

Integrate the WebScraping.AI API with the Data Stores API

Setup the WebScraping.AI API trigger to run a workflow which integrates with the Data Stores API. Pipedream's integration platform allows you to integrate WebScraping.AI and Data Stores remarkably fast. Free for developers.

Add or update a single record with the Data Stores API

Add or update a single record in your Pipedream Data Store.

 
Try it
Add or update multiple records with the Data Stores API

Add or update multiple records to your Pipedream Data Store.

 
Try it
Append to record with the Data Stores API

Append to a record in your data store Pipedream Data Store. If the record does not exist, a new record will be created in an array format.

 
Try it
Check for existence of key with the Data Stores API

Check if a key exists in your Pipedream Data Store or create one if it doesn't exist.

 
Try it
Delete a single record with the Data Stores API

Delete a single record in your Pipedream Data Store.

 
Try it

Overview of WebScraping.AI

WebScraping.AI is an API designed to make it easy to access and extract online
content from websites. Using the WebScraping.AI API, you can quickly and easily
build a range of automated data retrieval applications, such as:

  • Web data scraping for lead generation
  • Automated web scraping to build market intelligence systems
  • Automated web scraping to monitor web content for competitive intelligence
  • Extracting data for specific products or services
  • Image/logo recognition for scraping
  • Automated web scraping for eCommerce monitoring
  • Automated web scraping for pricing information
  • Extracting data from job postings
  • Monitoring social media for marketing trends
  • Extracting data from financial statements

Connect WebScraping.AI

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import { axios } from "@pipedream/platform"
export default defineComponent({
  props: {
    webscraping_ai: {
      type: "app",
      app: "webscraping_ai",
    }
  },
  async run({steps, $}) {
    return await axios($, {
      url: `https://api.webscraping.ai/account`,
      params: {
        api_key: `${this.webscraping_ai.$auth.api_key}`,
      },
    })
  },
})

Overview of Data Stores

With the Data Stores API, you can build applications that:

  • Store data for later retrieval
  • Retrieve data from a store
  • Update data in a store
  • Delete data from a store

Connect Data Stores

1
2
3
4
5
6
7
8
9
10
11
export default defineComponent({
  props: {
    myDataStore: {
      type: "data_store",
    },
  },
  async run({ steps, $ }) {
    await this.myDataStore.set("key_here","Any serializable JSON as the value")
    return await this.myDataStore.get("key_here")
  },
})