What do you want to automate
with Webcrawler API and Python?
Prompt, edit and deploy AI agents that connect to Webcrawler API, Python and 3,000+ other apps in seconds.
Trusted by 1,000,000+ developers from startups to Fortune 500 companies
Start for free — no credit card required
Watch a demo
Popular Webcrawler API and Python Actions#
Run Python Code with the Python API
Write Python and use any of the 350k+ PyPi packages available. Refer to the Pipedream Python docs to learn more.
Integrate the Webcrawler API API with the Python API
Setup the Webcrawler API API trigger to run a workflow which integrates with the Python API. Pipedream's integration platform allows you to integrate Webcrawler API and Python remarkably fast. Free for developers.
Connect Webcrawler API#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { WebcrawlerClient } from "webcrawlerapi-js";
export default defineComponent({
props: {
webcrawler_api: {
type: "app",
app: "webcrawler_api",
}
},
async run({ steps, $ }) {
const client = new WebcrawlerClient(this.webcrawler_api.$auth.api_key);
return await client.crawl({
"items_limit": 10,
"url": "https://books.toscrape.com/",
"scrape_type": "markdown",
});
},
})
Overview of Python#
Develop, run and deploy your Python code in Pipedream workflows. Integrate seamlessly between no-code steps, with connected accounts, or integrate Data Stores and manipulate files within a workflow
This includes installing PyPI packages, within your code without having to manage a requirements.txt file or running pip.
Below is an example of using Python to access data from the trigger of the workflow, and sharing it with subsequent workflow steps:
Connect Python#
1
2
3
4
5
def handler(pd: "pipedream"):
# Reference data from previous steps
print(pd.steps["trigger"]["context"]["id"])
# Return data for use in future steps
return {"foo": {"test":True}}Start for free — no credit card required
Watch a demo
Trusted by 1,000,000+ developers from startups to Fortune 500 companies