with SFTP (password-based auth) and Airbyte?
Emit new events when files get created, changed or deleted from a remote directory. See the docs
SFTP on Pipedream enables secure file management on remote servers directly from your workflows. Utilize this app to automate file uploads, downloads, and synchronization tasks between your systems and SFTP servers. Implement robust data pipelines or deploy content to web servers without manual intervention, all while maintaining a high level of security with password-based authentication.
module.exports = defineComponent({
props: {
sftp_password_based_auth: {
type: "app",
app: "sftp_password_based_auth",
}
},
async run({steps, $}) {
const Client = require('ssh2-sftp-client');
const {
host,
username,
password,
} = this.sftp_password_based_auth.$auth
const config = {
host,
username,
password,
}
const sftp = new Client()
await sftp.connect(config)
this.cwd = await sftp.cwd();
return await sftp.end();
},
})
The Airbyte API allows for creating and managing data integration pipelines between various sources and destinations, automating data synchronization tasks, and monitoring the status of those pipelines. On Pipedream, you can leverage the Airbyte API to build intricate workflows that react to data events, manipulate and store data, and connect to other services to create rich, automated data pipelines.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
https_airbyte_com: {
type: "app",
app: "https_airbyte_com",
}
},
async run({steps, $}) {
return await axios($, {
url: `${this.https_airbyte_com.$auth.url}/v1/connections`,
headers: {
Authorization: `Bearer ${this.https_airbyte_com.$auth.api_key}`,
},
})
},
})