Why am I getting a TypeError in my workflow when pulling data from Snowflake and how to fix it?

Bugs
1

This topic was automatically generated from Slack. You can find the original thread here.

I started getting this error on a workflow today when pulling data from Snowflake. Anyone have any ideas? Seems to have started at ~7a CST today.

TypeError
this.determineConnectionDomain is not a function

Details
    at null.Connection.connect (/pipedream/dist/code/4206a01bc15d120d97e1f8da4884716b65f7779c826526554ca1061e224e68eb/node_modules/.pnpm/snowflake-sdk@1.12.0_asn1.js@5.4.1/node_modules/snowflake-sdk/lib/connection/connection.js:197:35)
    at null.null (node:internal/util:430:7)
    at null.Promise (null:null:null)
    at null.null (node:internal/util:416:12)
    at Object.run (file:///pipedream/dist/code/4206a01bc15d120d97e1f8da4884716b65f7779c826526554ca1061e224e68eb/component.mjs:29:11)
    at null.executeComponent (/var/task/launch_worker.js:292:53)
    at MessagePort.messageHandler (/var/task/launch_worker.js:792:28)
2

Hi , would you mind sharing the screenshot of Snowflake action configurqtion you’re using?

3 
import { promisify } from 'util'
import snowflake from 'snowflake-sdk'

function extract_id(url) {
  // Split the URL by '/'
  const parts = url.split('/');

  //return parts

  // The ID is the last part of the URL
  const id = parts[4];

  return id
}

export default defineComponent({
  props: {
    snowflake: {
      type: "app",
      app: "snowflake",
    }
  },
  async run({steps, $}) {
    const connection = snowflake.createConnection({
      ...this.snowflake.$auth,
      application: "PIPEDREAM_PIPEDREAM",
    })
    const connectAsync = promisify(connection.connect)
    await connectAsync()
    
    async function connExecuteAsync(options) {
      return new Promise((resolve, reject) => {
        connection.execute({
          ...options,
          complete: function(err, stmt, rows) {
            if (err) {
              reject(err)
            } else {
              resolve({stmt, rows})
            }
          }
        })
      })
    }

    // Example usage
    const url = steps.Grab_Order_Data.$return_value.data.order.id;
    const numeric_id = extract_id(url);

    //return numeric_id
    
    // See https://docs.snowflake.com/en/user-guide/nodejs-driver-use.html#executing-statements
    const { rows } = await connExecuteAsync({
      sqlText: `
      select *
      from FIVETRAN_DATABASE.RECHARGE_PROD.ORDER_LINE_ITEM
      where order_id in 
      (select distinct id
      from "FIVETRAN_DATABASE"."RECHARGE_PROD"."ORDER"
      where external_order_id_ecommerce = '${numeric_id}')
      `,
    })
    return rows
  },
})

image.png
image.png872×528 39.2 KB

4

No changes have been made to this workflow in some time, just started seeing these errors this AM

5

LMK if you need anymore deets, happy to provide and thanks in advance for your support.

6

Hi , I suspect the reason is the recent version 1.12.0 from snowflake-sdk library. Maybe you want to pin the version to 1.11.0 import snowflake from 'snowflake-sdk@1.11.0'

7

That did the trick!! Thanks so much !!!

8

I’m having the same issue. How can I specify the issue with

const snowflake = require('snowflake-sdk');

This was fixed in 1.15 BTW, can you roll that out?

Thanks!

9

I could still reproduce the issue with the 1.15.0 version. Are there specific changes you require in this latest version?

We also provide a helper package @pipedream/snowflake based on our registry components:

import snowflake from '@pipedream/snowflake';

export default defineComponent({
  props: {
    snowflake,
  },
  async run({ $ }) {
    // Component source code:
    // https://github.com/PipedreamHQ/pipedream/tree/master/components/snowflake
    
    return this.snowflake.executeQuery({
      sqlText: `SELECT CURRENT_TIMESTAMP()`,
      binds: [],
    });
  },
});