# Start Scraping — Scrape-It.Cloud

> Initiate the scraping process for a specific endpoint. See the documentation here.

- Key: `scrape_it_cloud-start-scraping`
- Type: Action (Write)
- Version: 0.0.2
- App: Scrape-It.Cloud (`scrape_it_cloud`) — https://pipedream.com/apps/scrape-it-cloud.md
- This page (HTML): https://pipedream.com/apps/scrape-it-cloud/actions/start-scraping
- Hints: open-world
- Source: https://github.com/PipedreamHQ/pipedream/blob/master/components/scrape_it_cloud/actions/start-scraping/start-scraping.mjs

## Description

Initiate the scraping process for a specific endpoint. [See the documentation here](https://scrape-it.cloud/docs/api-features/basic-request).

## Props

| Prop | Type | Required | Description |
|---|---|---|---|
| `url` | `string` | Yes | Specify the URL of the web page you would like to scrape. |
| `headers` | `object` | No | Specify custom HTTP headers to be passed to the request. For example, in order to override the User-Agent header, use User-Agent as a key and Teapot as a value. |
| `blockResources` | `boolean` | No | By default Scrape-it.cloud do not block images and CSS in the scraped page. To speed up requests and block images and CSS set this parameter value to true. |
| `blockUrls` | `string[]` | No | If you want to block some resources except images and CSS, for example, analytics scripts you can add part of the urls to be blocked. Follow Blocking URLs page to get more information. |
| `wait` | `integer` | No | Some websites may use javascript frameworks that may require a few extra seconds to load their content. This parameters specifies the time in milliseconds to wait for the website. Recommended values are in the interval 5000 - 10000. |
| `waitFor` | `string` | No | Specify a CSS selector and the API will wait 30 seconds until the selector appears. |
| `screenshot` | `boolean` | No | If you want to get a screenshot of the page you want to scrape, set screenshot parameter value to true. |
| `jsScenario` | `string[]` | No | If you want to do some acts on the pages you want to scrape, for example, go to the next one, set JavaScript scenario to API call. Follow JavaScript Execution page to get more information. |

## Run it

**MCP**

```ts
import { Client } from "@modelcontextprotocol/sdk/client/index.js"
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"
import { PipedreamClient } from "@pipedream/sdk"

const pd = new PipedreamClient({
  projectId: process.env.PIPEDREAM_PROJECT_ID!,
  clientId: process.env.PIPEDREAM_CLIENT_ID!,
  clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
  projectEnvironment: "production",
})

const accessToken = await pd.rawAccessToken

const transport = new StreamableHTTPClientTransport(
  new URL("https://remote.mcp.pipedream.net/v3"),
  {
    requestInit: {
      headers: {
        Authorization: `Bearer ${accessToken}`,
        "x-pd-project-id": process.env.PIPEDREAM_PROJECT_ID!,
        "x-pd-environment": "production",
        "x-pd-external-user-id": "{external_user_id}", // any stable ID for this user in your system
        "x-pd-app-slug": "scrape_it_cloud",
      },
    },
  },
)

const mcp = new Client({ name: "my-agent", version: "1.0.0" })
await mcp.connect(transport)

const { tools } = await mcp.listTools()

// listTools() hands your model this tool's input schema, so it can
// fill the arguments itself:
const result = await mcp.callTool({
  name: "scrape_it_cloud-start-scraping",
  arguments: {
    url: "URL",
    headers: "Headers",
  },
})
```

**TypeScript**

```ts
import { PipedreamClient } from "@pipedream/sdk"

const pd = new PipedreamClient({
  projectId: process.env.PIPEDREAM_PROJECT_ID!,
  clientId: process.env.PIPEDREAM_CLIENT_ID!,
  clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
  projectEnvironment: "production",
})

const result = await pd.actions.run({
  id: "scrape_it_cloud-start-scraping",
  externalUserId: "{external_user_id}", // any stable ID for this user in your system
  configuredProps: {
    scrape_it_cloud: { authProvisionId: "apn_xxxxxxx" },
    url: "URL",
    headers: "Headers",
  },
})

console.log(result)
```

**cURL**

```bash
curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \
  -H "Content-Type: application/json" \
  -H "X-PD-Environment: production" \
  -H "Authorization: Bearer {access_token}" \
  -d '{
    "external_user_id": "{external_user_id}",
    "id": "scrape_it_cloud-start-scraping",
    "configured_props": {
      "scrape_it_cloud": { "authProvisionId": "apn_xxxxxxx" },
      "url": "URL",
      "headers": "Headers"
    }
  }'
```

---

- App: https://pipedream.com/apps/scrape-it-cloud.md · All apps: https://pipedream.com/apps