Google Cloud

The Google Cloud Platform, including BigQuery

Integrate the Google Cloud API with the Snowflake API

Setup the Google Cloud API trigger to run a workflow which integrates with the Snowflake API. Pipedream's integration platform allows you to integrate Google Cloud and Snowflake remarkably fast. Free for developers.

Bigquery Insert Rows with Google Cloud API on New Row from Snowflake API
Snowflake + Google Cloud
 
Try it
Create Bucket with Google Cloud API on New Row from Snowflake API
Snowflake + Google Cloud
 
Try it
Create Scheduled Query with Google Cloud API on New Row from Snowflake API
Snowflake + Google Cloud
 
Try it
Get Bucket Metadata [See the docs](https://googleapis.dev/nodejs/storage/latest/Bucket.html#getMetadata) with Google Cloud API on New Row from Snowflake API
Snowflake + Google Cloud
 
Try it
Get Object with Google Cloud API on New Row from Snowflake API
Snowflake + Google Cloud
 
Try it
New Pub/Sub Messages from the Google Cloud API

Emit new Pub/Sub topic in your GCP account. Messages published to this topic are emitted from the Pipedream source.

 
Try it
New Row from the Snowflake API

Emit new event when a row is added to a table

 
Try it
BigQuery - New Row from the Google Cloud API

Emit new events when a new row is added to a table

 
Try it
New Query Results from the Snowflake API

Run a SQL query on a schedule, triggering a workflow for each row of results

 
Try it
BigQuery - Query Results from the Google Cloud API

Emit new events with the results of an arbitrary query

 
Try it
Bigquery Insert Rows with the Google Cloud API

Inserts rows into a BigQuery table. See the docs and for an example here.

 
Try it
Insert Multiple Rows with the Snowflake API

Insert multiple rows into a table

 
Try it
Create Bucket with the Google Cloud API

Creates a bucket on Google Cloud Storage See the docs

 
Try it
Insert Single Row with the Snowflake API

Insert a row into a table

 
Try it
Create Scheduled Query with the Google Cloud API

Creates a scheduled query in Google Cloud. See the documentation

 
Try it

Overview of Google Cloud

The Google Cloud API opens a world of possibilities for enhancing cloud operations and automating tasks. It empowers you to manage, scale, and fine-tune various services within the Google Cloud Platform (GCP) programmatically. With Pipedream, you can harness this power to create intricate workflows, trigger cloud functions based on events from other apps, manage resources, and analyze data, all in a serverless environment. The ability to interconnect GCP services with numerous other apps enriches automation, making it easier to synchronize data, streamline development workflows, and deploy applications efficiently.

Connect Google Cloud

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
module.exports = defineComponent({
  props: {
    google_cloud: {
      type: "app",
      app: "google_cloud",
    }
  },
  async run({steps, $}) {
    // Required workaround to get the @google-cloud/storage package
    // working correctly on Pipedream
    require("@dylburger/umask")()
    
    const { Storage } = require('@google-cloud/storage')
    
    const key = JSON.parse(this.google_cloud.$auth.key_json)
     
    // Creates a client from a Google service account key.
    // See https://cloud.google.com/nodejs/docs/reference/storage/1.6.x/global#ClientConfig
    const storage = new Storage({
      projectId: key.project_id,
      credentials: {
        client_email: key.client_email,
        private_key: key.private_key,
      }
    })
    
    // Uncomment this section and rename for your bucket before running this code
    // const bucketName = 'pipedream-test-bucket';
    
    await storage.createBucket(bucketName)
    console.log(`Bucket ${bucketName} created.`)
  },
})

Overview of Snowflake

Snowflake offers a cloud database and related tools to help developers create robust, secure, and scalable data warehouses. See Snowflake's Key Concepts & Architecture.

Getting Started

1. Create a user, role and warehouse in Snowflake

Snowflake recommends you create a new user, role, and warehouse when you integrate a third-party tool like Pipedream. This way, you can control permissions via the user / role, and separate Pipedream compute and costs with the warehouse. You can do this directly in the Snowflake UI.

We recommend you create a read-only account if you only need to query Snowflake. If you need to insert data into Snowflake, add permissions on the appropriate objects after you create your user.

2. Enter those details in Pipedream

Visit https://pipedream.com/accounts. Click the button to Connect an App. Enter the required Snowflake account data.

You'll only need to connect your account once in Pipedream. You can connect this account to multiple workflows to run queries against Snowflake, insert data, and more.

3. Build your first workflow

Visit https://pipedream.com/new to build your first workflow. Pipedream workflows let you connect Snowflake with 1,000+ other apps. You can trigger workflows on Snowflake queries, sending results to Slack, Google Sheets, or any app that exposes an API. Or you can accept data from another app, transform it with Python, Node.js, Go or Bash code, and insert it into Snowflake.

Learn more at Pipedream University.

Connect Snowflake

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import { promisify } from 'util'
import snowflake from 'snowflake-sdk'

export default defineComponent({
  props: {
    snowflake: {
      type: "app",
      app: "snowflake",
    }
  },
  async run({steps, $}) {
    const connection = snowflake.createConnection({
      ...this.snowflake.$auth,
      application: "PIPEDREAM_PIPEDREAM",
    })
    const connectAsync = promisify(connection.connect)
    await connectAsync()
    
    async function connExecuteAsync(options) {
      return new Promise((resolve, reject) => {
        connection.execute({
          ...options,
          complete: function(err, stmt, rows) {
            if (err) {
              reject(err)
            } else {
              resolve({stmt, rows})
            }
          }
        })
      })
    }
    
    // See https://docs.snowflake.com/en/user-guide/nodejs-driver-use.html#executing-statements
    const { rows } = await connExecuteAsync({
      sqlText: `SELECT CURRENT_TIMESTAMP()`,
    })
    return rows
  },
})