with Hacker News and FireCrawl?
Crawls a given URL and returns the contents of sub-pages. See the documentation
Extract structured data from one or multiple URLs. See the documentation
Obtains the status and data from a previous crawl operation. See the documentation
Obtains the status and data from a previous extract operation. See the documentation
Scrapes a URL and returns content from that page. See the documentation
The Hacker News API on Pipedream allows you to tap into the vibrant pool of stories, comments, and user data from one of tech's favorite forums. By leveraging this API, you can automate the process of extracting data for analysis, tracking mentions of specific topics, or even monitoring the performance of your own submissions. The real power lies in integrating this wealth of information with other services to create custom, automated workflows that save time and keep you informed.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
hacker_news: {
type: "app",
app: "hacker_news",
}
},
async run({steps, $}) {
return await axios($, {
url: `http://hn.algolia.com/api/v1/search_by_date?query=pipedream`,
})
},
})
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
firecrawl: {
type: "app",
app: "firecrawl",
}
},
async run({steps, $}) {
const data = {
"url": "https://pipedream.com",
}
return await axios($, {
method: "post",
url: `https://api.firecrawl.dev/v0/crawl`,
headers: {
Authorization: `Bearer ${this.firecrawl.$auth.api_key}`,
},
data,
})
},
})