# Scrape Page — Scrapfly

> Extract data from a specified web page. See the documentation

- Key: `scrapfly-scrape-page`
- Type: Action (Read-only)
- Version: 0.0.2
- App: Scrapfly (`scrapfly`) — https://pipedream.com/apps/scrapfly.md
- This page (HTML): https://pipedream.com/apps/scrapfly/actions/scrape-page
- Hints: read-only · open-world
- Source: https://github.com/PipedreamHQ/pipedream/blob/master/components/scrapfly/actions/scrape-page/scrape-page.mjs

## Description

Extract data from a specified web page. [See the documentation](https://scrapfly.io/docs/scrape-api/getting-started)

## Props

| Prop | Type | Required | Description |
|---|---|---|---|
| `url` | `string` | Yes | This URL is used to transform any relative URLs in the document into absolute URLs automatically. It can be either the base URL or the exact URL of the document. Must be url encoded. |
| `headers` | `object` | No | Pass custom headers to the request. |
| `lang` | `string` | No | Select page language. By default it uses the language of the selected proxy location. Behind the scenes, it configures the Accept-Language HTTP header. If the website support the language, the content will be in that lang. Note: you cannot set headers Accept-Language header manually. See the documentation |
| `os` | `string` | No | Operating System, if not selected it's random. Note: you cannot set os parameter and User-Agent header at the same time. See the documentation |
| `timeout` | `integer` | No | Timeout in milliseconds. It represents the maximum time allowed for Scrapfly to perform the scrape. Since timeout is not trivial to understand see our extended documentation on timeouts |
| `format` | `string` | No | Format of the response. |
| `retry` | `boolean` | No | Improve reliability with retries on failure. |
| `proxifiedResponse` | `boolean` | No | Return the content of the page directly. |
| `debug` | `boolean` | No | Store the API result and take a screenshot if rendering js is enabled. |
| `correlationId` | `string` | No | Helper ID for correlating a group of scrapes. |
| `tags` | `string[]` | No | Add tags to your scrapes to group them. |
| `dns` | `boolean` | No | Query and retrieve target DNS information. |
| `ssl` | `boolean` | No | SSL option. |
| `proxyPool` | `string` | No | Select the proxy pool to use. |

## Run it

**MCP**

```ts
import { Client } from "@modelcontextprotocol/sdk/client/index.js"
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"
import { PipedreamClient } from "@pipedream/sdk"

const pd = new PipedreamClient({
  projectId: process.env.PIPEDREAM_PROJECT_ID!,
  clientId: process.env.PIPEDREAM_CLIENT_ID!,
  clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
  projectEnvironment: "production",
})

const accessToken = await pd.rawAccessToken

const transport = new StreamableHTTPClientTransport(
  new URL("https://remote.mcp.pipedream.net/v3"),
  {
    requestInit: {
      headers: {
        Authorization: `Bearer ${accessToken}`,
        "x-pd-project-id": process.env.PIPEDREAM_PROJECT_ID!,
        "x-pd-environment": "production",
        "x-pd-external-user-id": "{external_user_id}", // any stable ID for this user in your system
        "x-pd-app-slug": "scrapfly",
      },
    },
  },
)

const mcp = new Client({ name: "my-agent", version: "1.0.0" })
await mcp.connect(transport)

const { tools } = await mcp.listTools()

// listTools() hands your model this tool's input schema, so it can
// fill the arguments itself:
const result = await mcp.callTool({
  name: "scrapfly-scrape-page",
  arguments: {
    url: "URL",
    headers: "Headers",
  },
})
```

**TypeScript**

```ts
import { PipedreamClient } from "@pipedream/sdk"

const pd = new PipedreamClient({
  projectId: process.env.PIPEDREAM_PROJECT_ID!,
  clientId: process.env.PIPEDREAM_CLIENT_ID!,
  clientSecret: process.env.PIPEDREAM_CLIENT_SECRET!,
  projectEnvironment: "production",
})

const result = await pd.actions.run({
  id: "scrapfly-scrape-page",
  externalUserId: "{external_user_id}", // any stable ID for this user in your system
  configuredProps: {
    scrapfly: { authProvisionId: "apn_xxxxxxx" },
    url: "URL",
    headers: "Headers",
  },
})

console.log(result)
```

**cURL**

```bash
curl -X POST https://api.pipedream.com/v1/connect/{project_id}/actions/run \
  -H "Content-Type: application/json" \
  -H "X-PD-Environment: production" \
  -H "Authorization: Bearer {access_token}" \
  -d '{
    "external_user_id": "{external_user_id}",
    "id": "scrapfly-scrape-page",
    "configured_props": {
      "scrapfly": { "authProvisionId": "apn_xxxxxxx" },
      "url": "URL",
      "headers": "Headers"
    }
  }'
```

---

- App: https://pipedream.com/apps/scrapfly.md · All apps: https://pipedream.com/apps