What do you want to automate
with Cloudflare R2 and Octoparse?
Prompt, edit and deploy AI agents that connect to Cloudflare R2, Octoparse and 2,500+ other apps in seconds.
Trusted by 1,000,000+ developers from startups to Fortune 500 companies
Overview of Cloudflare R2#
The Cloudflare R2 API lets you interact with Cloudflare's object storage service, providing a cost-effective way to store large amounts of data with no egress fees. On Pipedream, you can harness this API to build automated workflows that can store, retrieve, and manage data within your R2 buckets. By combining Cloudflare R2 with Pipedream’s capabilities, you can create serverless workflows that trigger on various events, process data in-flight, and integrate with over 800+ apps available on the platform.
Connect Cloudflare R2#
import { S3 } from "@aws-sdk/client-s3";
import { ListBucketsCommand } from "@aws-sdk/client-s3";
export default defineComponent({
props: {
cloudflare_r2: {
type: "app",
app: "cloudflare_r2",
}
},
async run({ steps, $ }) {
const cloudflare_r2 = {
account_id: this.cloudflare_r2.$auth.account_id
};
const s3Client = new S3({
forcePathStyle: false,
endpoint: `https://${cloudflare_r2.account_id}.r2.cloudflarestorage.com`,
region: "auto",
credentials: {
accessKeyId: this.cloudflare_r2.$auth.access_key_id,
secretAccessKey: this.cloudflare_r2.$auth.access_key_secret
}
});
const data = await s3Client.send(new ListBucketsCommand({}));
return data.Buckets;
}
}
)
Overview of Octoparse#
The Octoparse API allows you to automate the extraction of web data without coding, making it a powerful tool for data-driven workflows. With this API, you can control your scraping tasks, retrieve extracted data, and manage your account programmatically. When combined with Pipedream's serverless execution environment, you can build custom workflows to process, store, or act upon the data fetched by Octoparse. This integration can be a cornerstone for solutions in market research, competitor analysis, price monitoring, or lead generation.
Connect Octoparse#
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
octoparse: {
type: "app",
app: "octoparse",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://openapi.octoparse.com/taskGroup`,
headers: {
Authorization: `Bearer ${this.octoparse.$auth.oauth_access_token}`,
},
})
},
})
Trusted by 1,000,000+ developers from startups to Fortune 500 companies