Hi, we’re Zyte, the central point of entry for all your web data needs.
Write Python and use any of the 350k+ PyPi packages available. Refer to the Pipedream Python docs to learn more.
The Zyte API provides programmatic access to web data extraction services, allowing you to pull structured data from websites efficiently. Within Pipedream, you can leverage the Zyte API to create powerful serverless workflows that automate data collection, monitor web content changes, or enrich your datasets with web-sourced information. By connecting Zyte to other apps on Pipedream, you can easily integrate web scraping into your data processing pipelines, event-driven applications, and more, with minimal setup and no server maintenance.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
zyte_api: {
type: "app",
app: "zyte_api",
}
},
async run({steps, $}) {
const data = {
"url": "https://books.toscrape.com/",
"httpResponseBody": true
}
return await axios($, {
method: "post",
url: `https://api.zyte.com/v1/extract`,
headers: {
"Content-Type": `application/json`,
},
auth: {
username: `${this.zyte_api.$auth.api_key}`,
password: ``,
},
data,
})
},
})
Develop, run and deploy your Python code in Pipedream workflows. Integrate seamlessly between no-code steps, with connected accounts, or integrate Data Stores and manipulate files within a workflow.
This includes installing PyPI packages, within your code without having to manage a requirements.txt
file or running pip
.
Below is an example of using Python to access data from the trigger of the workflow, and sharing it with subsequent workflow steps:
def handler(pd: "pipedream"):
# Reference data from previous steps
print(pd.steps["trigger"]["context"]["id"])
# Return data for use in future steps
return {"foo": {"test":True}}