with WebScraper.IO and Plaid?
Emit new event when a page scraping job has completed. See the docs here
Emit new event when there are new accounts available at the Financial Institution. See the documentation
Emit new event when there are changes to Plaid Items or the status of asynchronous processes. See the documentation
Emit new event when there are new updates available for a connected account. See the documentation
Creates a scraping job (scrapes a sitemap). See the docs here
Exchange a Link public_token
for an API access_token
. See the documentation
Creates a sitemap for the selected website. See the docs here
Creates a valid public_token
for an arbitrary institution ID, initial products, and test credentials. See the documentation
Retrieves a list of scraping jobs for a sitemap. See the docs here
The WebScraper.IO API allows you to programmatically perform web scraping tasks, extracting structured data from websites. With the API, you can automate the gathering of web content for analysis, monitoring, and integration with other data sources. In Pipedream, you can leverage this API to build workflows that process, analyze, and act on the data you scrape without writing code for backend infrastructure.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
webscraper_io: {
type: "app",
app: "webscraper_io",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.webscraper.io/api/v1/sitemaps`,
params: {
api_token: `${this.webscraper_io.$auth.api_key}`,
},
})
},
})
The Plaid API offers a multitude of financial data operations, enabling developers to manage and interact with user bank accounts, transactions, identity info, and more, all within a secure and compliant ecosystem. Integrating Plaid with Pipedream can unlock powerful automations, like syncing transaction data to accounting software, verifying user identities for KYC compliance, and automating financial alerts or reporting.
module.exports = defineComponent({
props: {
plaid: {
type: "app",
app: "plaid",
}
},
async run({steps, $}) {
const { Configuration, PlaidApi } = require('plaid');
const client = new PlaidApi(
new Configuration({
basePath: this.plaid.$auth.environment,
baseOptions: {
headers: {
'PLAID-CLIENT-ID': this.plaid.$auth.client_id,
'PLAID-SECRET': this.plaid.$auth.client_secret,
},
},
})
);
// Test request
const request = {
count: 10,
offset: 0,
country_codes: ['US'],
};
const response = await client.institutionsGet(request);
return response.data.institutions;
},
})