Making web data extraction easy and accessible for everyone.
Emit new event when a new message is posted to one or more channels
Emit new event when a page scraping job has completed. See the docs here
Emit new event when a message was posted in a direct message channel
Emit new events on new Slack interactivity events sourced from Block Kit interactive elements, Slash commands, or Shortcuts.
Creates a scraping job (scrapes a sitemap). See the docs here
Creates a sitemap for the selected website. See the docs here
Send a message to a user, group, private channel or public channel. See the documentation
Retrieves a list of scraping jobs for a sitemap. See the docs here
Configure custom blocks and send to a channel, group, or user. See the documentation.
The WebScraper.IO API allows you to programmatically perform web scraping tasks, extracting structured data from websites. With the API, you can automate the gathering of web content for analysis, monitoring, and integration with other data sources. In Pipedream, you can leverage this API to build workflows that process, analyze, and act on the data you scrape without writing code for backend infrastructure.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
webscraper_io: {
type: "app",
app: "webscraper_io",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.webscraper.io/api/v1/sitemaps`,
params: {
api_token: `${this.webscraper_io.$auth.api_key}`,
},
})
},
})
The Pipedream Slack app enables you to build event-driven workflows that interact with the Slack API. Once you authorize the Pipedream app's access to your workspace, you can use Pipedream workflows to perform common Slack actions or write your own code against the Slack API.
The Pipedream Slack app is not a typical app. You don't interact with it directly as a bot, and it doesn't add custom functionality to your workspace out of the box. It makes it easier to automate anything you'd typically use the Slack API for, using Pipedream workflows.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
slack: {
type: "app",
app: "slack",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://slack.com/api/users.profile.get`,
headers: {
Authorization: `Bearer ${this.slack.$auth.oauth_access_token}`,
},
})
},
})