Why is Google Cloud BigQuery Integration in Pipedream Failing with Authentication Errors Despite Configured Service Account Credentials?

Bugs
1

This topic was automatically generated from Slack. You can find the original thread here.

Bug Report: BigQuery Authentication Failure in Pipedream Connect

Issue Description: The Google Cloud BigQuery integration in Pipedream is failing with authentication errors. When attempting to run queries against BigQuery tables through the run action, the system consistently returns an error related to missing authentication credentials, despite having properly configured service account credentials.

Technical Details:

  • Integration: Google Cloud BigQuery function calls via Pipedream connection
  • Error: TypeError: Cannot read properties of undefined (reading ‘key_json’)
  • Expected Behavior: Successfully authenticate and execute BigQuery queries using the provided service account
  • Actual Behavior: Authentication fails with key_json undefined error

Error Message:

"TypeError: Cannot read properties of undefined (reading 'key_json')\n    at Object.authKeyJson (file:///tmp/__pdg__/dist/code/36c85f22e6620c6ac4e12e9f22542988ce936b37977cb3645b382688b0986a32/code/google_cloud.app.mjs:112:36)\n    at Object.getBigQueryClient (file:///tmp/__pdg__/dist/code/36c85f22e6620c6ac4e12e9f22542988ce936b37977cb3645b382688b0986a32/code/google_cloud.app.mjs:201:32)\n    at Object.run (file:///tmp/__pdg__/dist/code/36c85f22e6620c6ac4e12e9f22542988ce936b37977cb3645b382688b0986a32/code/actions/run-query/run-query.mjs:29:37)\n    at file:///var/task/component_maker.mjs:166:42\n    at captureObservations (/var/task/node_modules/@lambda-v2/component-runtime/src/captureObservations.js:28:11)\n    at run (file:///var/task/component_maker.mjs:156:26)\n    at async MessagePort.anonymous (file:///var/task/component_maker.mjs:194:9)"

Service Account Used (Redacted):

json
{
  "type": "service_account",
  "project_id": "[REDACTED]",
  "private_key_id": "[REDACTED]",
  "private_key": "[REDACTED]",
  "client_email": "big-query@[REDACTED].[iam.gserviceaccount.com](http://iam.gserviceaccount.com)",
  "client_id": "[REDACTED]",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/big-query%40[REDACTED].iam.gserviceaccount.com",
  "universe_domain": "[googleapis.com](http://googleapis.com)"
}

Steps to Reproduce:

  1. Configure Pipedream connection with the above service account credentials
  2. Use the google_cloud-run-query function in the assistant
  3. Provide any valid SQL query parameter
  4. Execute the function call
  5. Observe authentication error with key_json undefined message

Example Query:

sql
SELECT ** FROM `reporte_25M` LIMIT 10

Impact:
This issue prevents any BigQuery data analysis functionality from working through the Pipedream connection in the assistant environment, making the Google Cloud integration completely unusable. Users cannot access or analyze their BigQuery data through the assistant despite having valid credentials.

Expected Resolution:

  1. Fix the authentication mechanism in the Pipedream connection to properly load and use the provided service account credentials
  2. Ensure the key_json property is correctly accessed in the authentication flow
  3. Update the error handling to provide more informative messages when authentication fails

*Urgency:
High - This affects all users trying to access BigQuery data through the Pipedream connection, rendering the Google Cloud integration non-functional even with valid credentials.

2

The action is running this args:

{
  externalUserId: '[redacted]',
  actionId: 'google_cloud-run-query',
  configuredProps: {
    google_cloud: { authProvisionId: '[redacted]' },
    query: '[redacted]'
  }
}
3

If anyone has this same problem the correct args are:

{
  externalUserId: '[redacted]',
  actionId: 'google_cloud-run-query',
  configuredProps: {
    googleCloud: { authProvisionId: '[redacted]' },
    query: '[redacted]'
  }
}