with Dev.to and ScrapeNinja?
Emit new event for each new story that has a matching tag (e.g., javascript)
Emit new event for each new article published on your Dev.to account
Emit new event for each new reading list item on your Dev.to account
Uses the ScrapeNinja real Chrome browser engine to scrape pages that require JS rendering. See the documentation
Use high-performance web scraping endpoint with Chrome browser TLS fingerprint, but without JavaScript execution and real browser overhead. See the documentation
The Dev.to API enables programmatic interaction with Dev.to, a community for software developers to share articles and engage with content. Through the API, you can automate content creation, management, and analysis. Accessing user articles, managing comments, and triggering actions based on Dev.to events are just a few capabilities at your disposal. By leveraging these with Pipedream's serverless platform, you can craft powerful automations that respond in real-time to activities on Dev.to, streamlining your content strategy and community engagement.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
dev_to: {
type: "app",
app: "dev_to",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://dev.to/api/articles/me`,
headers: {
"api-key": `${this.dev_to.$auth.api_key}`,
},
})
},
})
ScrapeNinja API on Pipedream allows you to craft powerful serverless workflows for web scraping without the hassle of managing proxies or browsers. It's a tool that can extract data from websites, handling JavaScript rendering and anti-bot measures with ease. By integrating ScrapeNinja with Pipedream, you can automate data collection, collate and process the scraped data, and connect it to numerous other services for further analysis, alerting, or storage.
import { axios } from '@pipedream/platform';
export default defineComponent({
props: {
scrapeninja: {
type: "app",
app: "scrapeninja",
}
},
async run({steps, $}) {
return await axios($, {
method: 'POST',
url: 'https://scrapeninja.p.rapidapi.com/scrape',
headers: {
'content-type': 'application/json',
'X-RapidAPI-Key': this.scrapeninja.$auth.rapid_api_key,
'X-RapidAPI-Host': 'scrapeninja.p.rapidapi.com'
},
data: {
url:"https://news.ycombinator.com/"
}
})
},
})