Request Social Media Scraping with ScrapingBot API

Pipedream makes it easy to connect APIs for ScrapingBot and 2,000+ other apps remarkably fast.

Trigger workflow on

HTTP requests, schedules and app events

Next, do this

Request Social Media Scraping with the ScrapingBot API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 800,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

Create a workflow to Request Social Media Scraping with the ScrapingBot API. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Configure the Request Social Media Scraping action
1. Connect your ScrapingBot account
2. Select a Scraper
3. Optional- Configure URL
4. Optional- Configure Account
5. Optional- Configure Search
6. Optional- Configure Hashtag
Select a trigger to run your workflow on HTTP requests, schedules or app events
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Integrations#

Request Social Media Scraping with ScrapingBot API on New Requests from HTTP / Webhook API

HTTP / Webhook + ScrapingBot

Try it

Request Social Media Scraping with ScrapingBot API on New Submission from Typeform API

Typeform + ScrapingBot

Try it

Request Social Media Scraping with ScrapingBot API on New Submission (Instant) from Jotform API

Jotform + ScrapingBot

Try it

Request Social Media Scraping with ScrapingBot API on New Custom Webhook Events from Stripe API

Stripe + ScrapingBot

Try it

Request Social Media Scraping with ScrapingBot API on New Scheduled Tasks from Pipedream API

Pipedream + ScrapingBot

Try it

Details#

This is a pre-built, source-available component from Pipedream's GitHub repo. The component is developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Request Social Media Scraping on ScrapingBot

Description:Use ScrapingBot API to initiate scraping data from a social media site. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)

Version:0.0.1

Key:scrapingbot-request-social-media-scraping

View on GitHub

Code#

import scrapingbot from "../../scrapingbot.app.mjs";
import { ConfigurationError } from "@pipedream/platform";
import constants from "../../common/constants.mjs";

export default {
  key: "scrapingbot-request-social-media-scraping",
  name: "Request Social Media Scraping",
  description: "Use ScrapingBot API to initiate scraping data from a social media site. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)",
  version: "0.0.1",
  type: "action",
  props: {
    scrapingbot,
    scraper: {
      propDefinition: [
        scrapingbot,
        "scraper",
      ],
    },
    url: {
      type: "string",
      label: "URL",
      description: "URL of the page you want to scrape",
      optional: true,
    },
    account: {
      type: "string",
      label: "Account",
      description: "The account name you want to extract data from",
      optional: true,
    },
    search: {
      propDefinition: [
        scrapingbot,
        "search",
      ],
      optional: true,
    },
    hashtag: {
      type: "string",
      label: "Hashtag",
      description: "The hashtag you wish to extract data from",
      optional: true,
    },
  },
  async run({ $ }) {
    const {
      scraper,
      url,
      account,
      search,
      hashtag,
    } = this;

    if (!url && constants.REQUIRES_URL.includes(scraper)) {
      throw new ConfigurationError(`URL is required for scraper ${scraper}`);
    }

    if (!account && constants.REQUIRES_ACCOUNT.includes(scraper)) {
      throw new ConfigurationError(`Account is required for scraper ${scraper}`);
    }

    if (!search && constants.REQUIRES_SEARCH.includes(scraper)) {
      throw new ConfigurationError(`Search is required for scraper ${scraper}`);
    }

    if (!hashtag && constants.REQUIRES_HASHTAG.includes(scraper)) {
      throw new ConfigurationError(`Hashtag is required for scraper ${scraper}`);
    }

    const response = await this.scrapingbot.scrapeSocialMedia({
      data: {
        scraper,
        url,
        account,
        search,
        hashtag,
      },
      $,
    });

    if (response.responseId) {
      $.export("$summary", `Successfully initiated scraping with responseId ${response.responseId}. Scraping Job may take a few minutes to complete.`);
    }
    return response;
  },
};

Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
ScrapingBot	`scrapingbot`	`app`	This component uses the ScrapingBot app.
Scraper	`scraper`	`string`	Select a value from the drop down menu:`linkedinProfilelinkedinCompanyProfilelinkedinPostinstagramProfileinstagramPostinstagramHashtagfacebookProfilefacebookPostfacebookOrganizationtwitterProfiletwitterSearchtiktokProfiletiktokHashtag`
URL	`url`	`string`	URL of the page you want to scrape
Account	`account`	`string`	The account name you want to extract data from
Search	`search`	`string`	The term to search for
Hashtag	`hashtag`	`string`	The hashtag you wish to extract data from