Octoparse

Easy Web Scraping for Anyone. No code is the best code. Octoparse allows everyone to build reliable web scrapers they need - no coding needed.

Go to site
Explore
/
Apps
/
Octoparse

Octoparse API Integrations

Build and run workflows using the Octoparse API. Use 1000s of source-available triggers and actions across 2,000+ apps. Or write custom code to integrate any app or API in seconds.

Overview

The Octoparse API allows you to automate the extraction of web data without coding, making it a powerful tool for data-driven workflows. With this API, you can control your scraping tasks, retrieve extracted data, and manage your account programmatically. When combined with Pipedream's serverless execution environment, you can build custom workflows to process, store, or act upon the data fetched by Octoparse. This integration can be a cornerstone for solutions in market research, competitor analysis, price monitoring, or lead generation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import { axios } from "@pipedream/platform"
export default defineComponent({
  props: {
    octoparse: {
      type: "app",
      app: "octoparse",
    }
  },
  async run({steps, $}) {
    return await axios($, {
      url: `https://openapi.octoparse.com/taskGroup`,
      headers: {
        Authorization: `Bearer ${this.octoparse.$auth.oauth_access_token}`,
      },
    })
  },
})

Choose an API to Connect with Octoparse API

1
-
24
of
2,000+
apps by most popular

HTTP / Webhook
HTTP / Webhook
Get a unique URL where you can send HTTP or webhook requests
Node
Node
Anything you can do with Node.js, you can do in a Pipedream workflow. This includes using most of npm's 400,000+ packages.
Python
Python
Anything you can do in Python can be done in a Pipedream Workflow. This includes using any of the 350,000+ PyPi packages available in your Python powered workflows.
OpenAI (ChatGPT)
OpenAI (ChatGPT)
OpenAI is an AI research and deployment company with the mission to ensure that artificial general intelligence benefits all of humanity. They are the makers of popular models like ChatGPT, DALL-E, and Whisper.
Salesforce (REST API)
Salesforce (REST API)
Web services API for interacting with Salesforce
HubSpot
HubSpot
HubSpot's CRM platform contains the marketing, sales, service, operations, and website-building software you need to grow your business.
Zoho CRM
Zoho CRM
Zoho CRM is an online Sales CRM software that manages your sales, marketing, and support in one CRM platform.
Stripe
Stripe
Stripe powers online and in-person payment processing and financial solutions for businesses of all sizes.
Shopify Developer App
Shopify Developer App
Shopify is a user-friendly e-commerce platform that helps small businesses build an online store and sell online through one streamlined dashboard.
WooCommerce
WooCommerce
WooCommerce is the open-source ecommerce platform for WordPress.
Snowflake
Snowflake
A data warehouse built for the cloud
MongoDB
MongoDB
MongoDB is an open source NoSQL database management program.
Supabase
Supabase
Supabase is an open source Firebase alternative.
MySQL
MySQL
MySQL is an open-source relational database management system.
PostgreSQL
PostgreSQL
PostgreSQL is a free and open-source relational database management system emphasizing extensibility and SQL compliance.
AWS
AWS
Amazon Web Services (AWS) offers reliable, scalable, and inexpensive cloud computing services.
Twilio SendGrid
Twilio SendGrid
Send marketing and transactional email through the Twilio SendGrid platform with the Email API, proprietary mail transfer agent, and infrastructure for scalable delivery.
Amazon SES
Amazon SES
Amazon SES is a cloud-based email service provider that can integrate into any application for high volume email automation
Klaviyo
Klaviyo
Email Marketing and SMS Marketing Platform
Zendesk
Zendesk
Zendesk is award-winning customer service software trusted by 200K+ customers. Make customers happy via text, mobile, phone, email, live chat, social media.
ServiceNow
ServiceNow
The smarter way to workflow
Notion
Notion
Notion is a new tool that blends your everyday work apps into one. It's the all-in-one workspace for you and your team.
Slack
Slack
Slack is a channel-based messaging platform. With Slack, people can work together more effectively, connect all their software tools and services, and find the information they need to do their best work — all within a secure, enterprise-grade environment.
Microsoft Teams
Microsoft Teams
Microsoft Teams has communities, events, chats, channels, meetings, storage, tasks, and calendars in one place.

Example Use Cases

  • Automated Data Collection to Google Sheets: Trigger a scraping task in Octoparse and, upon completion, send the extracted data to Google Sheets. This workflow is ideal for users needing regular updates for market research or competitive analysis without manual intervention.

  • Lead Generation with CRM Integration: Use Octoparse to scrape contact information from target websites and automatically add these leads to a CRM platform like Salesforce. This workflow streamlines the process of expanding your sales funnel with fresh prospects.

  • Price Monitoring and Alerts: Set Octoparse to track product prices on e-commerce websites and, using Pipedream, create a workflow that sends alerts via email or Slack if prices drop below a specified threshold. This can be an asset for businesses monitoring pricing strategies or for consumers looking for deals.

Authentication

Octoparse uses OAuth authentication. When you connect your Octoparse account, Pipedream will open a popup window where you can sign into Octoparse and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Octoparse API.

Pipedream requests the following authorization scopes when you connect your account:

OAuth Request Configurations:
  1. accessToken
    POSThttps://openapi.octoparse.com/tokenContent-Type: application/json
    username={{custom_fields.username}}&password={{custom_fields.password}}&grant_type=password
  2. refreshToken
    POSThttps://openapi.octoparse.com/tokencontent-type: application/json
    grant_type=refresh_token&refresh_token={{custom_fields.refresh_token}}