with SFTP (key-based auth) and Google Cloud?
Emit new events when files get created, changed or deleted from a remote directory. See the docs
Emit new Pub/Sub topic in your GCP account. Messages published to this topic are emitted from the Pipedream source.
Emit new events with the results of an arbitrary query
Uploads a file or string in UTF-8 format to the SFTP server. See the documentation
Inserts rows into a BigQuery table. See the docs and for an example here
Creates a scheduled query in Google Cloud. See the documentation
Gets Google Cloud Storage bucket metadata. See the docs
The SFTP (key-based auth) app on Pipedream allows you to securely transfer and manage files over a network. It uses SSH for data transfer and provides the same level of security as SSH, without requiring password authentication, making automation workflows more secure and less prone to human error. You can automate file uploads, downloads, synchronization tasks, and efficiently manage your remote files in a serverless environment. Integrating this with other Pipedream-supported apps enables you to craft powerful and secure data flow systems.
module.exports = defineComponent({
props: {
sftp: {
type: "app",
app: "sftp",
}
},
async run({steps, $}) {
const Client = require('ssh2-sftp-client');
const {
host,
username,
privateKey,
} = this.sftp.$auth
const config = {
host,
username,
privateKey,
}
const sftp = new Client()
await sftp.connect(config)
this.cwd = await sftp.cwd();
return await sftp.end();
},
})
The Google Cloud API opens a world of possibilities for enhancing cloud operations and automating tasks. It empowers you to manage, scale, and fine-tune various services within the Google Cloud Platform (GCP) programmatically. With Pipedream, you can harness this power to create intricate workflows, trigger cloud functions based on events from other apps, manage resources, and analyze data, all in a serverless environment. The ability to interconnect GCP services with numerous other apps enriches automation, making it easier to synchronize data, streamline development workflows, and deploy applications efficiently.
module.exports = defineComponent({
props: {
google_cloud: {
type: "app",
app: "google_cloud",
}
},
async run({steps, $}) {
// Required workaround to get the @google-cloud/storage package
// working correctly on Pipedream
require("@dylburger/umask")()
const { Storage } = require('@google-cloud/storage')
const key = JSON.parse(this.google_cloud.$auth.key_json)
const storage = new Storage({
projectId: key.project_id,
credentials: {
client_email: key.client_email,
private_key: key.private_key,
}
})
await storage.authClient.getCredentials()
return {
status: "success",
authenticated: true,
projectId: key.project_id,
serviceAccount: key.client_email
}
},
})