Integrate the Snowflake API with the Python API

Snowflake offers a cloud database and related tools to help developers create robust, secure, and scalable data warehouses. See Snowflake's Key Concepts & Architecture.

Getting Started

1. Create a user, role and warehouse in Snowflake

Snowflake recommends you create a new user, role, and warehouse when you integrate a third-party tool like Pipedream. This way, you can control permissions via the user / role, and separate Pipedream compute and costs with the warehouse. You can do this directly in the Snowflake UI.

We recommend you create a read-only account if you only need to query Snowflake. If you need to insert data into Snowflake, add permissions on the appropriate objects after you create your user.

2. Enter those details in Pipedream

Visit https://pipedream.com/accounts. Click the button to Connect an App. Enter the required Snowflake account data.

You'll only need to connect your account once in Pipedream. You can connect this account to multiple workflows to run queries against Snowflake, insert data, and more.

3. Build your first workflow

Visit https://pipedream.com/new to build your first workflow. Pipedream workflows let you connect Snowflake with 1,000+ other apps. You can trigger workflows on Snowflake queries, sending results to Slack, Google Sheets, or any app that exposes an API. Or you can accept data from another app, transform it with Python, Node.js, Go or Bash code, and insert it into Snowflake.

Learn more at Pipedream University.

Connect Snowflake#

import { promisify } from 'util'
import snowflake from 'snowflake-sdk'

export default defineComponent({
  props: {
    snowflake: {
      type: "app",
      app: "snowflake",
    }
  },
  async run({steps, $}) {
    const connection = snowflake.createConnection({
      ...this.snowflake.$auth,
      application: "PIPEDREAM_PIPEDREAM",
    })
    const connectAsync = promisify(connection.connect)
    await connectAsync()
    
    async function connExecuteAsync(options) {
      return new Promise((resolve, reject) => {
        connection.execute({
          ...options,
          complete: function(err, stmt, rows) {
            if (err) {
              reject(err)
            } else {
              resolve({stmt, rows})
            }
          }
        })
      })
    }
    
    // See https://docs.snowflake.com/en/user-guide/nodejs-driver-use.html#executing-statements
    const { rows } = await connExecuteAsync({
      sqlText: `SELECT CURRENT_TIMESTAMP()`,
    })
    return rows
  },
})

Overview of Python#

Python API on Pipedream offers developers to build or automate a variety of
tasks from their web and cloud apps. With the Python API, users are able to
create comprehensive and flexible scripts, compose and manage environment
variables, and configure resources to perform a range of functions.

By using Pipedream, you can easily:

Create automated workflows that run on a specific schedule
Compose workflows across various apps and services
React to events in cloud services or form data
Automatically create content and notifications
Construct classifications and predictions
Analyze and react to sentiment, sentiment analysis and sentiment score
Connect backends to the frontend with serverless functions
Work with files and databases
Perform web requests and fetch data
Integrate third-party APIs into your apps
Orchestrate data processing tasks and pipelines
Create powerful application APIs with authentication and authorization
Design CI/CD pipelines and Continuous Delivery services
Connect databases like MongoDB and MySQL
Monitor connections and events
Generate alerts and notifications for corresponding events

Connect Python#

def handler(pd: "pipedream"):
  # Reference data from previous steps
  print(pd.steps["trigger"]["context"]["id"])
  # Return data for use in future steps
  return {"foo": {"test":True}}