← AWS + Google Sheets integrations

Add Single Row with Google Sheets API on New Records Returned by CloudWatch Logs Insights Query from AWS API

Pipedream makes it easy to connect APIs for Google Sheets, AWS and 900+ other apps remarkably fast.

Trigger workflow on
New Records Returned by CloudWatch Logs Insights Query from the AWS API
Next, do this
Add Single Row with the Google Sheets API
No credit card required
Into to Pipedream
Watch us build a workflow
Watch us build a workflow
7 min
Watch now ➜
Trusted by 250,000+ developers from startups to Fortune 500 companies:
Trusted by 250,000+ developers from startups to Fortune 500 companies

Developers Pipedream

Getting Started

This integration creates a workflow with a AWS trigger and Google Sheets action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

  1. Select this integration
  2. Configure the New Records Returned by CloudWatch Logs Insights Query trigger
    1. Connect your AWS account
    2. Select a AWS Region
    3. Select one or more CloudWatch Log Groups
    4. Configure Logs Insights Query
    5. Optional- Configure Emit query results as a single event
    6. Configure Polling schedule
  3. Configure the Add Single Row action
    1. Connect your Google Sheets account
    2. Optional- Select a Drive
    3. Select a Spreadsheet
    4. Select a Sheet Name
    5. Select a Does the first row of the sheet have headers?
  4. Deploy the workflow
  5. Send a test event to validate your setup
  6. Turn on the trigger

Details

This integration uses pre-built, open source components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger

Description:Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch
Version:0.2.0
Key:aws-new-records-returned-by-cloudwatch-logs-insights-query

Trigger Code

import common from "../../common/common-cloudwatch-logs.mjs";

export default {
  ...common,
  key: "aws-new-records-returned-by-cloudwatch-logs-insights-query",
  name: "New Records Returned by CloudWatch Logs Insights Query",
  // eslint-disable-next-line pipedream/source-description
  description: "Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch",
  version: "0.2.0",
  type: "source",
  props: {
    aws: common.props.aws,
    region: common.props.region,
    db: "$.service.db",
    // eslint-disable-next-line pipedream/props-label
    logGroupNames: {
      ...common.props.logGroupName,
      type: "string[]",
      description: "The log groups you'd like to query",
    },
    queryString: {
      label: "Logs Insights Query",
      description: "The query you'd like to run. See [this AWS doc](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html) for help with query syntax",
      type: "string",
    },
    emitResultsInBatch: {
      type: "boolean",
      label: "Emit query results as a single event",
      description: "If `true`, all events are emitted as an array, within a single Pipedream event. If `false`, each row of results is emitted as its own event. Defaults to `true`",
      optional: true,
      default: true,
    },
    timer: {
      label: "Polling schedule",
      description: "How often you want to query CloudWatch Logs Insights for results",
      type: "$.interface.timer",
      default: {
        intervalSeconds: 5 * 60,
      },
    },
  },
  async run() {
    const now = +new Date();
    const startTime = this.db.get("startTime") || now - 1000 * 60 * 60;

    // First, start the query
    const params = {
      queryString: this.queryString,
      startTime,
      endTime: now,
      logGroupNames: this.logGroupNames,
    };

    const { queryId } = await this.startQuery(params);

    // Then poll for its status, emitting each record as its own event when completed
    async function sleep(ms) {
      return new Promise((r) => setTimeout(r, ms));
    }

    let result, res;
    do {
      await sleep(1000);
      res = await this.getQueryResults({
        queryId,
      });
      result = res.status;
    } while (result === "Running" || result === "Scheduled");

    if (result !== "Complete") {
      throw new Error(`Query ${queryId} failed with status ${result}`);
    }

    console.log(JSON.stringify(res, null, 2));
    const { results } = res;

    if (!results || !results.length) {
      console.log("No results, exiting");
      this.db.set("startTime", now);
      return;
    }

    if (this.emitResultsInBatch === true) {
      this.$emit(results, {
        summary: JSON.stringify(results),
      });
    } else {
      for (const item of results) {
        this.$emit(item, {
          summary: JSON.stringify(item),
        });
      }
    }

    // The next time this source runs, query for data starting now
    this.db.set("startTime", now);
  },
};

Trigger Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.
LabelPropTypeDescription
AWSawsappThis component uses the AWS app.
AWS RegionregionstringSelect a value from the drop down menu.
N/Adb$.service.dbThis component uses $.service.db to maintain state between component invocations.
CloudWatch Log GroupslogGroupNamesstring[]Select a value from the drop down menu.
Logs Insights QueryqueryStringstring

The query you'd like to run. See this AWS doc for help with query syntax

Emit query results as a single eventemitResultsInBatchboolean

If true, all events are emitted as an array, within a single Pipedream event. If false, each row of results is emitted as its own event. Defaults to true

Polling scheduletimer$.interface.timer

How often you want to query CloudWatch Logs Insights for results

Trigger Authentication

AWS uses API keys for authentication. When you connect your AWS account, Pipedream securely stores the keys so you can easily authenticate to AWS APIs in both code and no-code steps.

Follow the AWS Instructions for creating an IAM user with an associated access and secret key.

As a best practice, attach the minimum set of IAM permissions necessary to perform the specific task in Pipedream. If your workflow only needs to perform a single API call, you should create a user and associate an IAM group / policy with permission to do only that task. You can create as many linked AWS accounts in Pipedream as you'd like.

Enter your access and secret key below.

About AWS

On-demand cloud computing platforms and APIs

Action

Description:Add a single row of data to Google Sheets
Version:2.0.8
Key:google_sheets-add-single-row

Action Code

import googleSheets from "../../google_sheets.app.mjs";
import { ConfigurationError } from "@pipedream/platform";

export default {
  key: "google_sheets-add-single-row",
  name: "Add Single Row",
  description: "Add a single row of data to Google Sheets",
  version: "2.0.8",
  type: "action",
  props: {
    googleSheets,
    drive: {
      propDefinition: [
        googleSheets,
        "watchedDrive",
      ],
    },
    sheetId: {
      propDefinition: [
        googleSheets,
        "sheetID",
        (c) => ({
          driveId: googleSheets.methods.getDriveId(c.drive),
        }),
      ],
      description: "",
      withLabel: true,
    },
    sheetName: {
      propDefinition: [
        googleSheets,
        "sheetName",
        (c) => ({
          sheetId: c.sheetId?.value || c.sheetId,
        }),
      ],
      description: "",
    },
    hasHeaders: {
      type: "string",
      label: "Does the first row of the sheet have headers?",
      description: "If the first row of your document has headers we'll retrieve them to make it easy to enter the value for each column.",
      options: [
        "Yes",
        "No",
      ],
      reloadProps: true,
    },
  },
  async additionalProps() {
    const sheetId = this.sheetId?.value || this.sheetId;
    const props = {};
    if (this.hasHeaders === "Yes") {
      const { values } = await this.googleSheets.getSpreadsheetValues(sheetId, `${this.sheetName}!1:1`);
      if (!values[0]?.length) {
        throw new ConfigurationError("Cound not find a header row. Please either add headers and click \"Refresh fields\" or adjust the action configuration to continue.");
      }
      for (let i = 0; i < values[0]?.length; i++) {
        props[`col_${i.toString().padStart(4, "0")}`] = {
          type: "string",
          label: values[0][i],
          optional: true,
        };
      }
    } else if (this.hasHeaders === "No") {
      props.myColumnData = {
        type: "string[]",
        label: "Values",
        description: "Provide a value for each cell of the row. Google Sheets accepts strings, numbers and boolean values for each cell. To set a cell to an empty value, pass an empty string.",
      };
    }
    return props;
  },
  async run({ $ }) {
    const sheetId = this.sheetId?.value || this.sheetId;
    let cells;
    if (this.hasHeaders === "Yes") {
      // TODO: If we could create a variable using this.allColumns in additionalProps, we dont need
      // to call getSpreadsheetValues here again.
      const { values: rows } = await this.googleSheets.getSpreadsheetValues(sheetId, `${this.sheetName}!1:1`);
      const [
        headers,
      ] = rows;
      cells = headers
        .map((_, i) => `col_${i.toString().padStart(4, "0")}`)
        .map((column) => this[column] ?? "");
    } else {
      cells = this.googleSheets.sanitizedArray(this.myColumnData);
    }

    // validate input
    if (!cells || !cells.length) {
      throw new Error("Please enter an array of elements in `Cells / Column Values`.");
    } else if (!Array.isArray(cells)) {
      throw new Error("Cell / Column data is not an array. Please enter an array of elements in `Cells / Column Values`.");
    } else if (Array.isArray(cells[0])) {
      throw new Error("Cell / Column data is a multi-dimensional array. A one-dimensional is expected. If you're trying to send multiple rows to Google Sheets, search for the action to add multiple rows to Sheets.");
    }

    const {
      arr,
      convertedIndexes,
    } = this.googleSheets.arrayValuesToString(cells);

    const data = await this.googleSheets.addRowsToSheet({
      spreadsheetId: sheetId,
      range: this.sheetName,
      rows: [
        arr,
      ],
    });

    let summary = `Added 1 row to [${this.sheetId?.label || this.sheetId} (${data.updatedRange})](https://docs.google.com/spreadsheets/d/${sheetId}).`;
    if (convertedIndexes.length > 0) {
      summary += " We detected something other than a string/number/boolean in at least one of the fields and automatically converted it to a string.";
    }
    $.export("$summary", summary);

    return data;
  },
};

Action Configuration

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

LabelPropTypeDescription
Google SheetsgoogleSheetsappThis component uses the Google Sheets app.
DrivedrivestringSelect a value from the drop down menu.
SpreadsheetsheetIdstringSelect a value from the drop down menu.
Sheet NamesheetNamestringSelect a value from the drop down menu.
Does the first row of the sheet have headers?hasHeadersstringSelect a value from the drop down menu:YesNo

Action Authentication

Google Sheets uses OAuth authentication. When you connect your Google Sheets account, Pipedream will open a popup window where you can sign into Google Sheets and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Google Sheets API.

Pipedream requests the following authorization scopes when you connect your account:

emailprofilehttps://www.googleapis.com/auth/drive

About Google Sheets

With Google Sheets, you can create, edit, and collaborate wherever you are

More Ways to Connect Google Sheets + AWS

Add Single Row with Google Sheets API on New Deleted S3 File from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New Emails sent to SES Catch-all Domain from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New Restored S3 File from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New S3 Event from AWS API
AWS + Google Sheets
 
Try it
Add Single Row with Google Sheets API on New S3 File from AWS API
AWS + Google Sheets
 
Try it
New Scheduled Tasks from the AWS API

Creates a Step Function State Machine to publish a message to an SNS topic at a specific timestamp. The SNS topic delivers the message to this Pipedream source, and the source emits it as a new event.

 
Try it
New SNS Messages from the AWS API

Creates an SNS topic in your AWS account. Messages published to this topic are emitted from the Pipedream source.

 
Try it
New Inbound SES Emails from the AWS API

The source subscribes to all emails delivered to a specific domain configured in AWS SES. When an email is sent to any address at the domain, this event source emits that email as a formatted event. These events can trigger a Pipedream workflow and can be consumed via SSE or REST API.

 
Try it
New Deleted S3 File from the AWS API

Emit new event when a file is deleted from a S3 bucket

 
Try it
New Records Returned by CloudWatch Logs Insights Query from the AWS API

Executes a CloudWatch Logs Insights query on a schedule, and emits the records as invidual events (default) or in batch

 
Try it
AWS - EventBridge - Send event to Event Bus with the AWS API

Sends an event to an EventBridge event bus

 
Try it
AWS - Lambda - Invoke Function with the AWS API

Invoke a Lambda function using the AWS API

 
Try it
AWS - SQS - Send Message with the AWS API

Sends a message to an SQS queue

 
Try it
CloudWatch Logs - Put Log Event with the AWS API

Uploads a log event to the specified log stream. See docs

 
Try it
Cloudwatch Logs - Put Log Events with the AWS API

Uploads a batch of log events to the specified log stream. See the docs for more information

 
Try it