Request Social Media Scraping with ScrapingBot API on New Review (Instant) from Judge.me API

Pipedream makes it easy to connect APIs for ScrapingBot, Judge.me and 2,900+ other apps remarkably fast.

Trigger workflow on

New Review (Instant) from the Judge.me API

Next, do this

Request Social Media Scraping with the ScrapingBot API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Judge.me trigger and ScrapingBot action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Review (Instant) trigger
1. Connect your Judge.me account
Configure the Request Social Media Scraping action
1. Connect your ScrapingBot account
2. Select a Scraper
3. Optional- Configure URL
4. Optional- Configure Account
5. Optional- Configure Search
6. Optional- Configure Hashtag
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Review (Instant) on Judge.me

Description:Emit new event when a new review is posted. [See the documentation](https://judge.me/api/docs#tag/Webhooks/operation/webhooks#create)

Version:0.0.1

Key:judge_me-new-review

View on GitHub

Judge.me Overview#

The Judge.me API lets you tap into a rich repository of product review data, enabling you to automate the gathering, moderation, and publication of reviews on your store. You can also trigger actions based on review events, like thanking a customer after they leave a positive review or addressing negative feedback promptly. With Pipedream's serverless platform, you can craft workflows to interact with other apps, streamline processes, and react to review-related activities in real-time, without writing any backend code.

Trigger Code#

import app from "../../judge_me.app.mjs";
import sampleEmit from "./test-event.mjs";

export default {
  name: "New Review (Instant)",
  description: "Emit new event when a new review is posted. [See the documentation](https://judge.me/api/docs#tag/Webhooks/operation/webhooks#create)",
  type: "source",
  key: "judge_me-new-review",
  version: "0.0.1",
  props: {
    app,
    db: "$.service.db",
    http: "$.interface.http",
  },
  methods: {
    _getWebhookData() {
      return JSON.parse(this.db.get("webhook"));
    },
    _setWebhookData(data) {
      this.db.set("webhook", JSON.stringify(data));
    },
  },
  hooks: {
    async activate() {
      const webhook = {
        key: "review/created",
        url: this.http.endpoint,
      };
      await this.app.createWebhook({
        data: {
          webhook,
        },
      });
      this._setWebhookData(webhook);
    },
    async deactivate() {
      const webhook = this._getWebhookData();
      await this.app.removeWebhook({
        data: webhook,
      });
    },
  },
  async run(event) {
    const { body } = event;
    this.$emit(body, {
      id: body.review.id,
      summary: `New review: ${body.review.title}`,
      ts: new Date(body.review.created_at),
    });
  },
  sampleEmit,
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Judge.me	`app`	`app`	This component uses the Judge.me app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.

Trigger Authentication#

Judge.me uses OAuth authentication. When you connect your Judge.me account, Pipedream will open a popup window where you can sign into Judge.me and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Judge.me API.

Pipedream requests the following authorization scopes when you connect your account:

read_reviewsread_productswrite_replieswrite_private_replies

About Judge.me#

Judge.me is the gold standard of product reviews and store reviews on Shopify.

Action#

Request Social Media Scraping on ScrapingBot

Description:Use ScrapingBot API to initiate scraping data from a social media site. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)

Version:0.0.2

Key:scrapingbot-request-social-media-scraping

View on GitHub

ScrapingBot Overview#

ScrapingBot API on Pipedream allows you to scrape websites without getting blocked, fetching crucial information while bypassing common defenses. Whether you're extracting product details, real estate listings, or automating competitor research, this API combined with Pipedream's serverless platform offers you the tools to automate these tasks efficiently. Pipedream's ability to trigger workflows via HTTP requests, schedule them, or react to events, means you can create robust scraping operations that integrate seamlessly with hundreds of other apps.

Action Code#

import scrapingbot from "../../scrapingbot.app.mjs";
import { ConfigurationError } from "@pipedream/platform";
import constants from "../../common/constants.mjs";

export default {
  key: "scrapingbot-request-social-media-scraping",
  name: "Request Social Media Scraping",
  description: "Use ScrapingBot API to initiate scraping data from a social media site. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)",
  version: "0.0.2",
  annotations: {
    destructiveHint: false,
    openWorldHint: true,
    readOnlyHint: false,
  },
  type: "action",
  props: {
    scrapingbot,
    scraper: {
      propDefinition: [
        scrapingbot,
        "scraper",
      ],
    },
    url: {
      type: "string",
      label: "URL",
      description: "URL of the page you want to scrape",
      optional: true,
    },
    account: {
      type: "string",
      label: "Account",
      description: "The account name you want to extract data from",
      optional: true,
    },
    search: {
      propDefinition: [
        scrapingbot,
        "search",
      ],
      optional: true,
    },
    hashtag: {
      type: "string",
      label: "Hashtag",
      description: "The hashtag you wish to extract data from",
      optional: true,
    },
  },
  async run({ $ }) {
    const {
      scraper,
      url,
      account,
      search,
      hashtag,
    } = this;

    if (!url && constants.REQUIRES_URL.includes(scraper)) {
      throw new ConfigurationError(`URL is required for scraper ${scraper}`);
    }

    if (!account && constants.REQUIRES_ACCOUNT.includes(scraper)) {
      throw new ConfigurationError(`Account is required for scraper ${scraper}`);
    }

    if (!search && constants.REQUIRES_SEARCH.includes(scraper)) {
      throw new ConfigurationError(`Search is required for scraper ${scraper}`);
    }

    if (!hashtag && constants.REQUIRES_HASHTAG.includes(scraper)) {
      throw new ConfigurationError(`Hashtag is required for scraper ${scraper}`);
    }

    const response = await this.scrapingbot.scrapeSocialMedia({
      data: {
        scraper,
        url,
        account,
        search,
        hashtag,
      },
      $,
    });

    if (response.responseId) {
      $.export("$summary", `Successfully initiated scraping with responseId ${response.responseId}. Scraping Job may take a few minutes to complete.`);
    }
    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
ScrapingBot	`scrapingbot`	`app`	This component uses the ScrapingBot app.
Scraper	`scraper`	`string`	Select a value from the drop down menu:`linkedinProfilelinkedinCompanyProfilelinkedinPostinstagramProfileinstagramPostinstagramHashtagfacebookProfilefacebookPostfacebookOrganizationtwitterProfiletwitterSearchtiktokProfiletiktokHashtag`
URL	`url`	`string`	URL of the page you want to scrape
Account	`account`	`string`	The account name you want to extract data from
Search	`search`	`string`	The term to search for
Hashtag	`hashtag`	`string`	The hashtag you wish to extract data from

Action Authentication#

ScrapingBot uses API keys for authentication. When you connect your ScrapingBot account, Pipedream securely stores the keys so you can easily authenticate to ScrapingBot APIs in both code and no-code steps.

ScrapingBot requires their users enter their ScrapingBot username in order to connect to their API. For more info, refer to ScrapingBot’s documentation.

About ScrapingBot#

Best web scraping APIs to extract HTML content without getting blocked.

More Ways to Connect ScrapingBot + Judge.me#

Other Popular Integrations#

Get Social Media Scraping Data with ScrapingBot API on New Review (Instant) from Judge.me API

Judge.me + ScrapingBot

Try it

Scrape Search Engine with ScrapingBot API on New Review (Instant) from Judge.me API

Judge.me + ScrapingBot

Try it

Scrape Webpage with ScrapingBot API on New Review (Instant) from Judge.me API

Judge.me + ScrapingBot

Try it

Popular Triggers#

New Review (Instant) from the Judge.me API

Emit new event when a new review is posted. See the documentation

Try it

Popular Actions#

Private Reply with the Judge.me API

Create a private reply for a review, on behalf of the shop. Private replies are not shown on the widgets, but can be emailed to the reviewers. See the documentation

Try it

Reply with the Judge.me API

Create a public reply for a review on behalf of the shop. Public replies are shown publicly on the widgets. See the documentation

Try it

Get Social Media Scraping Data with the ScrapingBot API

Retrieve data from a social media scraping job by responseId. See the documentation

Try it

Request Social Media Scraping with the ScrapingBot API

Use ScrapingBot API to initiate scraping data from a social media site. See the documentation

Try it

Scrape Search Engine with the ScrapingBot API

Use ScrapingBot API to extract specific data from Google or Bing search results. See the documentation