This topic was automatically generated from Slack. You can find the original thread here.
can you show me how to use the node app to create a new dataset in Google BigQuery?
I’m not seeing a pre-built integration for creating a new dataset in Google BigQuery using the Node app in Pipedream. However, you can make an API request to BigQuery using the @pipedream/platform axios package. First, you’ll need to set up a Google Cloud project and enable the BigQuery API. Then, you’ll need to obtain the necessary credentials (API key or OAuth 2.0) to authenticate your requests. You can find more information on setting up the project and authentication in the BigQuery documentation. Once you have the credentials, you can use the following code as a starting point to create a new dataset in BigQuery: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { projectId: { type: "string", label: "Google Cloud Project ID", }, datasetId: { type: "string", label: "Dataset ID", }, apiKey: { type: "string", label: "API Key", }, }, async run({ steps, $ }) { const url = `https://bigquery.googleapis.com/bigquery/v2/projects/${this.projectId}/datasets`; const data = { datasetReference: { projectId: this.projectId, datasetId: this.datasetId, }, }; try { const response = await axios($, { method: "POST", url, headers: { "Content-Type": "application/json", "Authorization": `Bearer ${this.apiKey}`, }, data, }); return response; } catch (error) { console.error("Error creating dataset:", error); } }, }); Replace the projectId, datasetId, and apiKey with your own values. This code will make a POST request to the BigQuery API to create a new dataset. You can customize the dataset configuration by modifying the data object as needed, following the BigQuery API reference. Please note that using API keys for authentication might have limitations. It’s recommended to use OAuth 2.0 for better security and access control. You can refer to the [Google OAuth 2.0 guide](Google Identity | Google for Developers
Can you show me how to run a query against a table in BigQuery?
To run a query against a table in BigQuery, you can use the Google BigQuery API. First, you’ll need to set up a Google Cloud project, enable the BigQuery API, and obtain the necessary credentials (OAuth 2.0) to authenticate your requests. You can find more information on setting up the project and authentication in the BigQuery documentation. Once you have the credentials, you can use the following code as a starting point to run a query in BigQuery: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { google: { type: "app", app: "google", }, projectId: { type: "string", label: "Google Cloud Project ID", }, query: { type: "string", label: "BigQuery SQL Query", }, }, async run({ $ }) { const url = `https://bigquery.googleapis.com/bigquery/v2/projects/${this.projectId}/queries`; const data = { query: this.query, }; try { const response = await axios($, { method: "POST", url, headers: { "Content-Type": "application/json", "Authorization": `Bearer ${this.google.$auth.oauth_access_token}`, }, data, }); return response; } catch (error) { console.error("Error running query:", error); } }, }); Replace the projectId and query with your own values. This code will make a POST request to the BigQuery API to run the specified query. You can customize the query and other configurations by modifying the data object as needed, following the BigQuery API reference.