with ScrapingBee and News API?
Search through millions of articles from over 150,000 large and small news sources and blogs. See the documentation
Retrieve live top and breaking headlines for a category, single source, multiple sources, or keywords. See the documentation
The ScrapingBee API lets you extract data from web pages programmatically. It handles headless browsers and rotating proxies, so you can focus on data extraction without worrying about common web scraping issues like getting blocked. On Pipedream, you can tap into ScrapingBee's capability to create serverless workflows that scrape web data and connect them with numerous services for processing, storing, or triggering actions.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
scrapingbee: {
type: "app",
app: "scrapingbee",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://app.scrapingbee.com/api/v1/`,
params: {
api_key: `${this.scrapingbee.$auth.api_key}`,
json_response: `True`,
url: `https://pipedream.com`,
},
})
},
})
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
news_api: {
type: "app",
app: "news_api",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://newsapi.org/v2/everything`,
headers: {
"X-Api-Key": `${this.news_api.$auth.api_key}`,
},
params: {
"q": `Computer Science`,
},
})
},
})