← AWS + Google Sheets integrations

Add Single Row with Google Sheets API on New Records Returned by CloudWatch Logs Insights Query from AWS API

Pipedream makes it easy to connect APIs for Google Sheets, AWS and 1000+ other apps remarkably fast.

Trigger workflow on
New Records Returned by CloudWatch Logs Insights Query from the AWS API
Next, do this
Add Single Row with the Google Sheets API
No credit card required
Into to Pipedream
Watch us build a workflow
Watch us build a workflow
7 min
Watch now ➜

Trusted by 400,000+ developers from startups to Fortune 500 companies

Adyen logo
Brex logo
Carta logo
Checkr logo
Chameleon logo
DevRev logo
LinkedIn logo
Netflix logo
New Relic logo
OnDeck logo
Replicated logo
Scale AI logo
Teamwork logo
Warner Bros. logo
Xendit logo

Developers Pipedream

Getting Started

This integration creates a workflow with a AWS trigger and Google Sheets action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

  1. Select this integration
  2. Configure the New Records Returned by CloudWatch Logs Insights Query trigger
    1. Connect your AWS account
    2. Select a AWS Region
    3. Select one or more CloudWatch Log Groups
    4. Configure Logs Insights Query
    5. Optional- Configure Emit query results as a single event
    6. Configure Polling schedule
  3. Configure the Add Single Row action
    1. Connect your Google Sheets account
    2. Optional- Select a Drive
    3. Select a Spreadsheet
    4. Select a Sheet Name
    5. Select a Does the first row of the sheet have headers?
  4. Deploy the workflow
  5. Send a test event to validate your setup
  6. Turn on the trigger

Details

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger

Description:Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch
Version:0.2.1
Key:aws-new-records-returned-by-cloudwatch-logs-insights-query

AWS Overview

You can use the AWS API to build a variety of applications and services. Here
are some examples:

  • A mobile application that allows users to access data stored in the cloud
  • A web application that allows users to upload and download files
  • A web service that allows developers to access Amazon's vast product catalog
  • A desktop application that allows users to access their Amazon account
    information

Trigger Code

import common from "../../common/common-cloudwatch-logs.mjs";
import { DEFAULT_POLLING_SOURCE_TIMER_INTERVAL } from "@pipedream/platform";

export default {
  ...common,
  key: "aws-new-records-returned-by-cloudwatch-logs-insights-query",
  name: "New Records Returned by CloudWatch Logs Insights Query",
  // eslint-disable-next-line pipedream/source-description
  description: "Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch",
  version: "0.2.1",
  type: "source",
  props: {
    aws: common.props.aws,
    region: common.props.region,
    db: "$.service.db",
    // eslint-disable-next-line pipedream/props-label
    logGroupNames: {
      ...common.props.logGroupName,
      type: "string[]",
      description: "The log groups you'd like to query",
    },
    queryString: {
      label: "Logs Insights Query",
      description: "The query you'd like to run. See [this AWS doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html) for help with query syntax",
      type: "string",
    },
    emitResultsInBatch: {
      type: "boolean",
      label: "Emit query results as a single event",
      description: "If `true`, all events are emitted as an array, within a single Pipedream event. If `false`, each row of results is emitted as its own event. Defaults to `true`",
      optional: true,
      default: true,
    },
    timer: {
      label: "Polling schedule",
      description: "How often you want to query CloudWatch Logs Insights for results",
      type: "$.interface.timer",
      default: {
        intervalSeconds: DEFAULT_POLLING_SOURCE_TIMER_INTERVAL,
      },
    },
  },
  async run() {
    const now = +new Date();
    const startTime = this.db.get("startTime") || now - 1000 * 60 * 60;

    // First, start the query
    const params = {
      queryString: this.queryString,
      startTime,
      endTime: now,
      logGroupNames: this.logGroupNames,
    };

    const { queryId } = await this.startQuery(params);

    // Then poll for its status, emitting each record as its own event when completed
    async function sleep(ms) {
      return new Promise((r) => setTimeout(r, ms));
    }

    let result, res;
    do {
      await sleep(1000);
      res = await this.getQueryResults({
        queryId,
      });
      result = res.status;
    } while (result === "Running" || result === "Scheduled");

    if (result !== "Complete") {
      throw new Error(`Query ${queryId} failed with status ${result}`);
    }

    console.log(JSON.stringify(res, null, 2));
    const { results } = res;

    if (!results || !results.length) {
      console.log("No results, exiting");
      this.db.set("startTime", now);
      return;
    }

    if (this.emitResultsInBatch === true) {
      this.$emit(results, {
        summary: JSON.stringify(results),
      });
    } else {
      for (const item of results) {
        this.$emit(item, {
          summary: JSON.stringify(item),
        });
      }
    }

    // The next time this source runs, query for data starting now
    this.db.set("startTime", now);
  },
};

Trigger Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.
LabelPropTypeDescription
AWSawsappThis component uses the AWS app.
AWS RegionregionstringSelect a value from the drop down menu.
N/Adb$.service.dbThis component uses $.service.db to maintain state between component invocations.
CloudWatch Log GroupslogGroupNamesstring[]Select a value from the drop down menu.
Logs Insights QueryqueryStringstring

The query you'd like to run. See this AWS doc for help with query syntax

Emit query results as a single eventemitResultsInBatchboolean

If true, all events are emitted as an array, within a single Pipedream event. If false, each row of results is emitted as its own event. Defaults to true

Polling scheduletimer$.interface.timer

How often you want to query CloudWatch Logs Insights for results

Trigger Authentication

AWS uses API keys for authentication. When you connect your AWS account, Pipedream securely stores the keys so you can easily authenticate to AWS APIs in both code and no-code steps.

Follow the AWS Instructions for creating an IAM user with an associated access and secret key.

As a best practice, attach the minimum set of IAM permissions necessary to perform the specific task in Pipedream. If your workflow only needs to perform a single API call, you should create a user and associate an IAM group / policy with permission to do only that task. You can create as many linked AWS accounts in Pipedream as you'd like.

Enter your access and secret key below.

About AWS

On-demand cloud computing platforms and APIs

Action

Description:Add a single row of data to Google Sheets
Version:2.0.9
Key:google_sheets-add-single-row

Google Sheets Overview

Some examples of things you can build using the Google Sheets API include:

  • A web app that lets users input data into a Google Sheet
  • A script that automatically updates a Google Sheet with data from another
    source
  • A tool that generates graphs and charts from data in a Google Sheet
  • A service that sends data from a Google Sheet to another API or application

Action Code

import googleSheets from "../../google_sheets.app.mjs";
import { ConfigurationError } from "@pipedream/platform";

export default {
  key: "google_sheets-add-single-row",
  name: "Add Single Row",
  description: "Add a single row of data to Google Sheets",
  version: "2.0.9",
  type: "action",
  props: {
    googleSheets,
    drive: {
      propDefinition: [
        googleSheets,
        "watchedDrive",
      ],
    },
    sheetId: {
      propDefinition: [
        googleSheets,
        "sheetID",
        (c) => ({
          driveId: googleSheets.methods.getDriveId(c.drive),
        }),
      ],
      description: "",
      withLabel: true,
    },
    sheetName: {
      propDefinition: [
        googleSheets,
        "sheetName",
        (c) => ({
          sheetId: c.sheetId?.value || c.sheetId,
        }),
      ],
      description: "",
    },
    hasHeaders: {
      type: "string",
      label: "Does the first row of the sheet have headers?",
      description: "If the first row of your document has headers we'll retrieve them to make it easy to enter the value for each column.",
      options: [
        "Yes",
        "No",
      ],
      reloadProps: true,
    },
  },
  async additionalProps() {
    const sheetId = this.sheetId?.value || this.sheetId;
    const props = {};
    if (this.hasHeaders === "Yes") {
      const { values } = await this.googleSheets.getSpreadsheetValues(sheetId, `${this.sheetName}!1:1`);
      if (!values[0]?.length) {
        throw new ConfigurationError("Cound not find a header row. Please either add headers and click \"Refresh fields\" or adjust the action configuration to continue.");
      }
      for (let i = 0; i < values[0]?.length; i++) {
        props[`col_${i.toString().padStart(4, "0")}`] = {
          type: "string",
          label: values[0][i],
          optional: true,
        };
      }
    } else if (this.hasHeaders === "No") {
      props.myColumnData = {
        type: "string[]",
        label: "Values",
        description: "Provide a value for each cell of the row. Google Sheets accepts strings, numbers and boolean values for each cell. To set a cell to an empty value, pass an empty string.",
      };
    }
    return props;
  },
  async run({ $ }) {
    const sheetId = this.sheetId?.value || this.sheetId;
    let cells;
    if (this.hasHeaders === "Yes") {
      // TODO: If we could create a variable using this.allColumns in additionalProps, we dont need
      // to call getSpreadsheetValues here again.
      const { values: rows } = await this.googleSheets.getSpreadsheetValues(sheetId, `${this.sheetName}!1:1`);
      const [
        headers,
      ] = rows;
      cells = headers
        .map((_, i) => `col_${i.toString().padStart(4, "0")}`)
        .map((column) => this[column] ?? "");
    } else {
      cells = this.googleSheets.sanitizedArray(this.myColumnData);
    }

    // validate input
    if (!cells || !cells.length) {
      throw new Error("Please enter an array of elements in `Cells / Column Values`.");
    } else if (!Array.isArray(cells)) {
      throw new Error("Cell / Column data is not an array. Please enter an array of elements in `Cells / Column Values`.");
    } else if (Array.isArray(cells[0])) {
      throw new Error("Cell / Column data is a multi-dimensional array. A one-dimensional is expected. If you're trying to send multiple rows to Google Sheets, search for the action to add multiple rows to Sheets.");
    }

    const {
      arr,
      convertedIndexes,
    } = this.googleSheets.arrayValuesToString(cells);

    const data = await this.googleSheets.addRowsToSheet({
      spreadsheetId: sheetId,
      range: this.sheetName,
      rows: [
        arr,
      ],
    });

    let summary = `Added 1 row to [${this.sheetId?.label || this.sheetId} (${data.updatedRange})](https://docs.google.com/spreadsheets/d/${sheetId}).`;
    if (convertedIndexes.length > 0) {
      summary += " We detected something other than a string/number/boolean in at least one of the fields and automatically converted it to a string.";
    }
    $.export("$summary", summary);

    return data;
  },
};

Action Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

LabelPropTypeDescription
Google SheetsgoogleSheetsappThis component uses the Google Sheets app.
DrivedrivestringSelect a value from the drop down menu.
SpreadsheetsheetIdstringSelect a value from the drop down menu.
Sheet NamesheetNamestringSelect a value from the drop down menu.
Does the first row of the sheet have headers?hasHeadersstringSelect a value from the drop down menu:YesNo

Action Authentication

Google Sheets uses OAuth authentication. When you connect your Google Sheets account, Pipedream will open a popup window where you can sign into Google Sheets and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Google Sheets API.

Pipedream requests the following authorization scopes when you connect your account:

emailprofilehttps://www.googleapis.com/auth/drive

About Google Sheets

With Google Sheets, you can create, edit, and collaborate wherever you are

More Ways to Connect Google Sheets + AWS

Add Multiple Rows with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Get Values in Range with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Get Values with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Add Multiple Rows with Google Sheets API on New Records Returned by CloudWatch Logs Insights Query from AWS API
AWS + Google Sheets
 
Try it
New Scheduled Tasks from the AWS API

Creates a Step Function State Machine to publish a message to an SNS topic at a specific timestamp. The SNS topic delivers the message to this Pipedream source, and the source emits it as a new event.

 
Try it
New SNS Messages from the AWS API

Creates an SNS topic in your AWS account. Messages published to this topic are emitted from the Pipedream source.

 
Try it
New Inbound SES Emails from the AWS API

The source subscribes to all emails delivered to a specific domain configured in AWS SES. When an email is sent to any address at the domain, this event source emits that email as a formatted event. These events can trigger a Pipedream workflow and can be consumed via SSE or REST API.

 
Try it
New Deleted S3 File from the AWS API

Emit new event when a file is deleted from a S3 bucket

 
Try it
New DynamoDB Stream Event from the AWS API

Emit new event when a DynamoDB stream receives new events. See the docs here

 
Try it
CloudWatch Logs - Put Log Event with the AWS API

Uploads a log event to the specified log stream. See docs

 
Try it
DynamoDB - Create Table with the AWS API

Creates a new table to your account. See docs

 
Try it
DynamoDB - Execute Statement with the AWS API

This operation allows you to perform transactional reads or writes on data stored in DynamoDB, using PartiQL. See docs

 
Try it
DynamoDB - Get Item with the AWS API

The Get Item operation returns a set of attributes for the item with the given primary key. If there is no matching item, Get Item does not return any data and there will be no Item element in the response. See docs

 
Try it
DynamoDB - Put Item with the AWS API

Creates a new item, or replaces an old item with a new item. If an item that has the same primary key as the new item already exists in the specified table, the new item completely replaces the existing item. See docs

 
Try it