← AWS + Google Sheets integrations

Add Multiple Rows with Google Sheets API on New Records Returned by CloudWatch Logs Insights Query from AWS API

Pipedream makes it easy to connect APIs for Google Sheets, AWS and 1000+ other apps remarkably fast.

Trigger workflow on
New Records Returned by CloudWatch Logs Insights Query from the AWS API
Next, do this
Add Multiple Rows with the Google Sheets API
No credit card required
Into to Pipedream
Watch us build a workflow
Watch us build a workflow
7 min
Watch now ➜

Trusted by 500,000+ developers from startups to Fortune 500 companies

Adyen logo
Brex logo
Carta logo
Checkr logo
Chameleon logo
DevRev logo
LinkedIn logo
Netflix logo
New Relic logo
OnDeck logo
Replicated logo
Scale AI logo
Teamwork logo
Warner Bros. logo
Xendit logo

Developers Pipedream

Getting Started

This integration creates a workflow with a AWS trigger and Google Sheets action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

  1. Select this integration
  2. Configure the New Records Returned by CloudWatch Logs Insights Query trigger
    1. Connect your AWS account
    2. Select a AWS Region
    3. Select one or more CloudWatch Log Groups
    4. Configure Logs Insights Query
    5. Optional- Configure Emit query results as a single event
    6. Configure Polling schedule
  3. Configure the Add Multiple Rows action
    1. Connect your Google Sheets account
    2. Optional- Select a Drive
    3. Select a Spreadsheet
    4. Select a Sheet Name
    5. Configure Row Values
  4. Deploy the workflow
  5. Send a test event to validate your setup
  6. Turn on the trigger

Details

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger

Description:Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch
Version:0.2.1
Key:aws-new-records-returned-by-cloudwatch-logs-insights-query

AWS Overview

You can use the AWS API to build a variety of applications and services. Here
are some examples:

  • A mobile application that allows users to access data stored in the cloud
  • A web application that allows users to upload and download files
  • A web service that allows developers to access Amazon's vast product catalog
  • A desktop application that allows users to access their Amazon account
    information

Trigger Code

import common from "../../common/common-cloudwatch-logs.mjs";
import { DEFAULT_POLLING_SOURCE_TIMER_INTERVAL } from "@pipedream/platform";

export default {
  ...common,
  key: "aws-new-records-returned-by-cloudwatch-logs-insights-query",
  name: "New Records Returned by CloudWatch Logs Insights Query",
  // eslint-disable-next-line pipedream/source-description
  description: "Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch",
  version: "0.2.1",
  type: "source",
  props: {
    aws: common.props.aws,
    region: common.props.region,
    db: "$.service.db",
    // eslint-disable-next-line pipedream/props-label
    logGroupNames: {
      ...common.props.logGroupName,
      type: "string[]",
      description: "The log groups you'd like to query",
    },
    queryString: {
      label: "Logs Insights Query",
      description: "The query you'd like to run. See [this AWS doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html) for help with query syntax",
      type: "string",
    },
    emitResultsInBatch: {
      type: "boolean",
      label: "Emit query results as a single event",
      description: "If `true`, all events are emitted as an array, within a single Pipedream event. If `false`, each row of results is emitted as its own event. Defaults to `true`",
      optional: true,
      default: true,
    },
    timer: {
      label: "Polling schedule",
      description: "How often you want to query CloudWatch Logs Insights for results",
      type: "$.interface.timer",
      default: {
        intervalSeconds: DEFAULT_POLLING_SOURCE_TIMER_INTERVAL,
      },
    },
  },
  async run() {
    const now = +new Date();
    const startTime = this.db.get("startTime") || now - 1000 * 60 * 60;

    // First, start the query
    const params = {
      queryString: this.queryString,
      startTime,
      endTime: now,
      logGroupNames: this.logGroupNames,
    };

    const { queryId } = await this.startQuery(params);

    // Then poll for its status, emitting each record as its own event when completed
    async function sleep(ms) {
      return new Promise((r) => setTimeout(r, ms));
    }

    let result, res;
    do {
      await sleep(1000);
      res = await this.getQueryResults({
        queryId,
      });
      result = res.status;
    } while (result === "Running" || result === "Scheduled");

    if (result !== "Complete") {
      throw new Error(`Query ${queryId} failed with status ${result}`);
    }

    console.log(JSON.stringify(res, null, 2));
    const { results } = res;

    if (!results || !results.length) {
      console.log("No results, exiting");
      this.db.set("startTime", now);
      return;
    }

    if (this.emitResultsInBatch === true) {
      this.$emit(results, {
        summary: JSON.stringify(results),
      });
    } else {
      for (const item of results) {
        this.$emit(item, {
          summary: JSON.stringify(item),
        });
      }
    }

    // The next time this source runs, query for data starting now
    this.db.set("startTime", now);
  },
};

Trigger Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.
LabelPropTypeDescription
AWSawsappThis component uses the AWS app.
AWS RegionregionstringSelect a value from the drop down menu.
N/Adb$.service.dbThis component uses $.service.db to maintain state between executions.
CloudWatch Log GroupslogGroupNamesstring[]Select a value from the drop down menu.
Logs Insights QueryqueryStringstring

The query you'd like to run. See this AWS doc for help with query syntax

Emit query results as a single eventemitResultsInBatchboolean

If true, all events are emitted as an array, within a single Pipedream event. If false, each row of results is emitted as its own event. Defaults to true

Polling scheduletimer$.interface.timer

How often you want to query CloudWatch Logs Insights for results

Trigger Authentication

AWS uses API keys for authentication. When you connect your AWS account, Pipedream securely stores the keys so you can easily authenticate to AWS APIs in both code and no-code steps.

Follow the AWS Instructions for creating an IAM user with an associated access and secret key.

As a best practice, attach the minimum set of IAM permissions necessary to perform the specific task in Pipedream. If your workflow only needs to perform a single API call, you should create a user and associate an IAM group / policy with permission to do only that task. You can create as many linked AWS accounts in Pipedream as you'd like.

Enter your access and secret key below.

About AWS

On-demand cloud computing platforms and APIs

Action

Description:Add multiple rows of data to a Google Sheet
Version:0.1.5
Key:google_sheets-add-multiple-rows

Google Sheets Overview

Some examples of things you can build using the Google Sheets API include:

  • A web app that lets users input data into a Google Sheet
  • A script that automatically updates a Google Sheet with data from another
    source
  • A tool that generates graphs and charts from data in a Google Sheet
  • A service that sends data from a Google Sheet to another API or application

Action Code

import googleSheets from "../../google_sheets.app.mjs";

export default {
  key: "google_sheets-add-multiple-rows",
  name: "Add Multiple Rows",
  description: "Add multiple rows of data to a Google Sheet",
  version: "0.1.5",
  type: "action",
  props: {
    googleSheets,
    drive: {
      propDefinition: [
        googleSheets,
        "watchedDrive",
      ],
    },
    sheetId: {
      propDefinition: [
        googleSheets,
        "sheetID",
        (c) => ({
          driveId: googleSheets.methods.getDriveId(c.drive),
        }),
      ],
    },
    sheetName: {
      propDefinition: [
        googleSheets,
        "sheetName",
        (c) => ({
          sheetId: c.sheetId,
        }),
      ],
    },
    rows: {
      propDefinition: [
        googleSheets,
        "rows",
      ],
    },
  },
  async run() {
    let rows = this.rows;

    let inputValidated = true;

    if (!Array.isArray(rows)) {
      rows = JSON.parse(this.rows);
    }

    if (!rows || !rows.length || !Array.isArray(rows)) {
      inputValidated = false;
    } else {
      rows.forEach((row) => { if (!Array.isArray(row)) { inputValidated = false; } });
    }

    // Throw an error if input validation failed
    if (!inputValidated) {
      console.error("Data Submitted:");
      console.error(rows);
      throw new Error("Rows data is not an array of arrays. Please enter an array of arrays in the `Rows` parameter above. If you're trying to send a single rows to Google Sheets, search for the action to add a single row to Sheets or try modifying the code for this step.");
    }

    return await this.googleSheets.addRowsToSheet({
      spreadsheetId: this.sheetId,
      range: this.sheetName,
      rows,
    });
  },
};

Action Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

LabelPropTypeDescription
Google SheetsgoogleSheetsappThis component uses the Google Sheets app.
DrivedrivestringSelect a value from the drop down menu.
SpreadsheetsheetIdstringSelect a value from the drop down menu.
Sheet NamesheetNamestringSelect a value from the drop down menu.
Row Valuesrowsstring

Provide an array of arrays. Each nested array should represent a row, with each element of the nested array representing a cell/column value (e.g., passing [["Foo",1,2],["Bar",3,4]] will insert two rows of data with three columns each). The most common pattern is to reference an array of arrays exported by a previous step (e.g., {{steps.foo.$return_value}}). You may also enter or construct a string that will JSON.parse() to an array of arrays.

Action Authentication

Google Sheets uses OAuth authentication. When you connect your Google Sheets account, Pipedream will open a popup window where you can sign into Google Sheets and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Google Sheets API.

Pipedream requests the following authorization scopes when you connect your account:

emailprofilehttps://www.googleapis.com/auth/drive

About Google Sheets

With Google Sheets, you can create, edit, and collaborate wherever you are

More Ways to Connect Google Sheets + AWS

Add Multiple Rows with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Get Values in Range with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Get Values with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New Records Returned by CloudWatch Logs Insights Query from AWS API
AWS + Google Sheets
 
Try it
New Scheduled Tasks from the AWS API

Creates a Step Function State Machine to publish a message to an SNS topic at a specific timestamp. The SNS topic delivers the message to this Pipedream source, and the source emits it as a new event.

 
Try it
New SNS Messages from the AWS API

Creates an SNS topic in your AWS account. Messages published to this topic are emitted from the Pipedream source.

 
Try it
New Inbound SES Emails from the AWS API

The source subscribes to all emails delivered to a specific domain configured in AWS SES. When an email is sent to any address at the domain, this event source emits that email as a formatted event. These events can trigger a Pipedream workflow and can be consumed via SSE or REST API.

 
Try it
New Deleted S3 File from the AWS API

Emit new event when a file is deleted from a S3 bucket

 
Try it
New DynamoDB Stream Event from the AWS API

Emit new event when a DynamoDB stream receives new events. See the docs here

 
Try it
CloudWatch Logs - Put Log Event with the AWS API

Uploads a log event to the specified log stream. See docs

 
Try it
DynamoDB - Create Table with the AWS API

Creates a new table to your account. See docs

 
Try it
DynamoDB - Execute Statement with the AWS API

This operation allows you to perform transactional reads or writes on data stored in DynamoDB, using PartiQL. See docs

 
Try it
DynamoDB - Get Item with the AWS API

The Get Item operation returns a set of attributes for the item with the given primary key. If there is no matching item, Get Item does not return any data and there will be no Item element in the response. See docs

 
Try it
DynamoDB - Put Item with the AWS API

Creates a new item, or replaces an old item with a new item. If an item that has the same primary key as the new item already exists in the specified table, the new item completely replaces the existing item. See docs

 
Try it