Making web data extraction easy and accessible for everyone.
Emit new event for each message posted to one or more channels in a Discord server
Emit new event for each command posted to one or more channels in a Discord server
Emit new event when a page scraping job has completed. See the docs here
Creates a scraping job (scrapes a sitemap). See the docs here
Creates a sitemap for the selected website. See the docs here
Retrieves a list of scraping jobs for a sitemap. See the docs here
Send a simple or structured message (using embeds) to a Discord channel
The WebScraper.IO API allows you to programmatically perform web scraping tasks, extracting structured data from websites. With the API, you can automate the gathering of web content for analysis, monitoring, and integration with other data sources. In Pipedream, you can leverage this API to build workflows that process, analyze, and act on the data you scrape without writing code for backend infrastructure.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
webscraper_io: {
type: "app",
app: "webscraper_io",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.webscraper.io/api/v1/sitemaps`,
params: {
api_token: `${this.webscraper_io.$auth.api_key}`,
},
})
},
})
The Discord API interacts seamlessly with Pipedream, empowering you to craft customized automations and workflows for your Discord server. With this powerful integration, you can automate tasks like message posting, user management, and notifications, based on a myriad of triggers and actions from different apps. These automations can enhance the Discord experience for community moderators and members, by synchronizing with external tools, organizing community engagement, and streamlining notifications.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
discord: {
type: "app",
app: "discord",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://discord.com/api/users/@me`,
headers: {
Authorization: `Bearer ${this.discord.$auth.oauth_access_token}`,
"accept": `application/json`,
},
})
},
})