Scrape Website with Webscrape AI API on Budget Snapshot Event (Instant) from Procore API

Pipedream makes it easy to connect APIs for Webscrape AI, Procore and 2,700+ other apps remarkably fast.

Trigger workflow on

Budget Snapshot Event (Instant) from the Procore API

Next, do this

Scrape Website with the Webscrape AI API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Procore trigger and Webscrape AI action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the Budget Snapshot Event (Instant) trigger
1. Connect your Procore account
2. Select a Company
3. Optional- Select a Project
Configure the Scrape Website action
1. Connect your Webscrape AI account
2. Configure alert
3. Configure URL
4. Configure Command
5. Configure Schema
6. Optional- Configure Pages
7. Optional- Configure Headers
8. Optional- Configure Instructions
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

Budget Snapshot Event (Instant) on Procore

Description:Emits an event each time a Budget Snapshot is created, updated, or deleted in a project.

Version:0.0.2

Key:procore-budget-snapshot

View on GitHub

Procore Overview#

The Procore API empowers developers to interact with its construction management software programmatically, enabling automation of tasks, data synchronization, and enhanced reporting. With APIs covering project management, quality and safety, construction financials, and field productivity, you can craft workflows that streamline operations, reduce manual entry, and provide real-time insights. On Pipedream, these capabilities can be harnessed to create workflows that react to events in Procore or orchestrate actions between Procore and other apps, optimizing construction project workflows.

Trigger Code#

const common = require("../common.js");

module.exports = {
  ...common,
  name: "Budget Snapshot Event (Instant)",
  key: "procore-budget-snapshot",
  description:
    "Emits an event each time a Budget Snapshot is created, updated, or deleted in a project.",
  version: "0.0.2",
  type: "source",
  methods: {
    ...common.methods,
    getResourceName() {
      return "Budget View Snapshots";
    },
    async getDataToEmit(body) {
      const { resource_id: resourceId } = body;
      const limit = 100;
      let offset = 0;
      let total = limit;
      let snapshotRows = [];
      while (total == limit) {
        const resource = await this.procore.getBudgetViewSnapshot(
          this.company,
          this.project,
          resourceId,
          limit,
          offset,
        );
        snapshotRows = snapshotRows.concat(resource);
        total = resource.length;
        offset += limit;
      }
      return {
        ...body,
        snapshotRows,
      };
    },
    getMeta(body) {
      const {
        id,
        event_type: eventType,
        resource_id: resourceId,
        timestamp,
      } = body;
      const ts = new Date(timestamp).getTime();
      return {
        id,
        summary: `${eventType} Budget Snapshot ID:${resourceId}`,
        ts,
      };
    },
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Procore	`procore`	`app`	This component uses the Procore app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.
Company	`company`	`integer`	Select a value from the drop down menu.
Project	`project`	`integer`	Select a value from the drop down menu.

Trigger Authentication#

Procore uses OAuth authentication. When you connect your Procore account, Pipedream will open a popup window where you can sign into Procore and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Procore API.

Pipedream requests the following authorization scopes when you connect your account:

About Procore#

The all-in-one construction management software built to help you finish quality projects — safely, on time, and within budget.

Action#

Scrape Website on Webscrape AI

Description:Scrape the provided URL and store the results in the system. [See the documentation](https://webscrapeai.com/docs)

Version:0.0.1

Key:webscrape_ai-scrape-website

View on GitHub

Action Code#

import webscrapeAi from "../../webscrape_ai.app.mjs";

export default {
  key: "webscrape_ai-scrape-website",
  name: "Scrape Website",
  description: "Scrape the provided URL and store the results in the system. [See the documentation](https://webscrapeai.com/docs)",
  version: "0.0.1",
  type: "action",
  props: {
    webscrapeAi,
    alert: {
      type: "alert",
      alertType: "info",
      content: "This actions sends a synchronous request to the WebScrapeAI API and may require increasing the workflow's default timeout.",
    },
    url: {
      type: "string",
      label: "URL",
      description: "The URL of the website to scrape",
    },
    command: {
      type: "string",
      label: "Command",
      description: "The data you want to extract. E.g. `I want to extract all the news details`",
    },
    schema: {
      type: "string",
      label: "Schema",
      description: "Schema representing the fields you want to scrape. E.g. `{\"author\":\"string\",\"comments_count\":\"integer\",\"points\":\"integer\",\"posted_time\":\"string\",\"title\":\"string\",\"url\":\"url\"}`",
    },
    pages: {
      type: "integer",
      label: "Pages",
      description: "Number of pages to scrape. Default value is 1.",
      optional: true,
    },
    headers: {
      type: "string",
      label: "Headers",
      description: "List of headers in key-value pairs. i.e `Accept: application/json`",
      optional: true,
    },
    instructions: {
      type: "string",
      label: "Instructions",
      description: "List of JavaScript instructions that you want to execute, like clicking a specific button, waiting for a specific code block to appear, etc. Example: `{\"click\": \"#button_id\"}`. [See the documentation](https://webscrapeai.com/docs) for more information.",
      optional: true,
    },
  },
  async run({ $ }) {
    const response = await this.webscrapeAi.scrapeWebsite({
      $,
      params: {
        url: this.url,
        command: this.command,
        schema: typeof this.schema === "object"
          ? JSON.stringify(this.schema)
          : this.schema,
        pages: this.pages,
        headers: this.headers,
        instructions: this.instructions,
      },
    });
    $.export("$summary", `Scraped ${this.url} and got ${response.length} result${response.length === 1
      ? ""
      : "s"}`);
    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Webscrape AI	`webscrapeAi`	`app`	This component uses the Webscrape AI app.
URL	`url`	`string`	The URL of the website to scrape
Command	`command`	`string`	The data you want to extract. E.g. `I want to extract all the news details`
Schema	`schema`	`string`	Schema representing the fields you want to scrape. E.g. `{"author":"string","comments_count":"integer","points":"integer","posted_time":"string","title":"string","url":"url"}`
Pages	`pages`	`integer`	Number of pages to scrape. Default value is 1.
Headers	`headers`	`string`	List of headers in key-value pairs. i.e `Accept: application/json`
Instructions	`instructions`	`string`	List of JavaScript instructions that you want to execute, like clicking a specific button, waiting for a specific code block to appear, etc. Example: `{"click": "#button_id"}`. See the documentation for more information.

Scrape Website with Webscrape AI API on Budget Snapshot Event (Instant) from Procore API

Pipedream makes it easy to connect APIs for Webscrape AI, Procore and 2,700+ other apps remarkably fast.

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

1-24of2,700+apps by most popular

1
-
24
of
2,700+
apps by most popular