[Date/Time] Add/Subtract Time with Formatting API on Redshift - Updated Row from AWS API

Pipedream makes it easy to connect APIs for Formatting, AWS and 2,700+ other apps remarkably fast.

Trigger workflow on

Redshift - Updated Row from the AWS API

Next, do this

[Date/Time] Add/Subtract Time with the Formatting API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a AWS trigger and Formatting action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the Redshift - Updated Row trigger
1. Connect your AWS account
2. Configure info
3. Configure timer
4. Select a AWS Region
5. Select a Workgroup Name
6. Select a Database Name
7. Select a Schema
8. Select a Table
9. Select a Timestamp Column
Configure the [Date/Time] Add/Subtract Time action
1. Connect your Formatting account
2. Configure Input Date
3. Optional- Select a Input Format
4. Select a Operation
5. Configure Duration
6. Optional- Select a Output Format
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

Redshift - Updated Row on AWS

Description:Emit new event when a row is updated, based on a selected timestamp column. [See the documentation](https://docs.aws.amazon.com/redshift/latest/APIReference/API_ExecuteStatement.html)

Version:0.0.1

Key:aws-redshift-updated-row

View on GitHub

AWS Overview#

The AWS API unlocks endless possibilities for automation with Pipedream. With this powerful combo, you can manage your AWS services and resources, automate deployment workflows, process data, and react to events across your AWS infrastructure. Pipedream offers a serverless platform for creating workflows triggered by various events that can execute AWS SDK functions, making it an efficient tool to integrate, automate, and orchestrate tasks across AWS services and other apps.

Trigger Code#

import common from "../common/redshift-polling.mjs";

export default {
  ...common,
  key: "aws-redshift-updated-row",
  name: "Redshift - Updated Row",
  description: "Emit new event when a row is updated, based on a selected timestamp column. [See the documentation](https://docs.aws.amazon.com/redshift/latest/APIReference/API_ExecuteStatement.html)",
  type: "source",
  version: "0.0.1",
  dedupe: "unique",
  props: {
    ...common.props,
    info: {
      propDefinition: [
        common.props.app,
        "info",
      ],
    },
    timestampColumn: {
      label: "Timestamp Column",
      description: "A column with a timestamp that is updated when a row is modified.",
      propDefinition: [
        common.props.app,
        "uniqueColumn",
        ({
          region,
          database,
          workgroupName,
          schema,
          table,
        }) => ({
          region,
          database,
          workgroupName,
          schema,
          table,
          filter: (column) => column.typeName === "timestamp",
        }),
      ],
    },
  },
  async run() {
    const {
      region,
      workgroupName,
      database,
      schema,
      table,
      timestampColumn,
      getValue,
    } = this;

    const lastTimestamp = this.db.get("lastTimestamp");

    let sql = `SELECT * FROM ${schema}.${table}`;
    if (lastTimestamp) {
      sql += ` WHERE ${timestampColumn} > :lastTimestamp`;
    }
    sql += ` ORDER BY ${timestampColumn} DESC LIMIT 100`;

    const parameters = lastTimestamp
      ? [
        {
          name: "lastTimestamp",
          value: lastTimestamp,
        },
      ]
      : undefined;

    const response = await this.app.executeStatement({
      region,
      workgroupName,
      database,
      sql,
      parameters,
    });

    if (response?.Records?.length > 0) {
      const columnNames = response.ColumnMetadata.map((c) => c.name);

      const records = response.Records.map((record) =>
        record.reduce((row, field, index) => ({
          ...row,
          [columnNames[index]]: getValue(field),
        }), {}));

      records.reverse().forEach((record) => {
        const id = record[timestampColumn];
        const ts = new Date(record[timestampColumn]).getTime();
        this.$emit(record, {
          id,
          ts,
          summary: `Row Updated ${id}`,
        });
      });

      if (records?.length > 0) {
        const [
          newLastTimestamp,
        ] = records.map((record) => record[timestampColumn]).reverse();
        this.db.set("lastTimestamp", newLastTimestamp);
      }
    }
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
AWS	`app`	`app`	This component uses the AWS app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
	`timer`	`$.interface.timer`
AWS Region	`region`	`string`	Select a value from the drop down menu.
Workgroup Name	`workgroupName`	`string`	Select a value from the drop down menu.
Database Name	`database`	`string`	Select a value from the drop down menu.
Schema	`schema`	`string`	Select a value from the drop down menu.
Table	`table`	`string`	Select a value from the drop down menu.
Timestamp Column	`timestampColumn`	`string`	Select a value from the drop down menu.

Trigger Authentication#

AWS uses API keys for authentication. When you connect your AWS account, Pipedream securely stores the keys so you can easily authenticate to AWS APIs in both code and no-code steps.

Follow the AWS Instructions for creating an IAM user with an associated access and secret key.

As a best practice, attach the minimum set of IAM permissions necessary to perform the specific task in Pipedream. If your workflow only needs to perform a single API call, you should create a user and associate an IAM group / policy with permission to do only that task. You can create as many linked AWS accounts in Pipedream as you'd like.

Enter your access and secret key below.

About AWS#

Amazon Web Services (AWS) offers reliable, scalable, and inexpensive cloud computing services.

Action#

[Date/Time] Add/Subtract Time on Formatting

Description:Add or subtract time from a given input

Version:0.0.5

Key:formatting-add-subtract-time

View on GitHub

Action Code#

import { defineAction } from "@pipedream/types";
import { ConfigurationError } from "@pipedream/platform";
import app from "../../app/formatting.app.mjs";
import commonDateTime from "../../common/date-time/commonDateTime.mjs";
import { DATE_FORMAT_PARSE_MAP, DEFAULT_FORMAT_VALUE, } from "../../common/date-time/dateFormats.mjs";
import { DATE_TIME_UNITS } from "../../common/date-time/dateTimeUnits.mjs";
import sugar from "sugar";
const OPERATION_OPTIONS = {
    ADD: "Add",
    SUBTRACT: "Subtract",
};
export default defineAction({
    ...commonDateTime,
    name: "[Date/Time] Add/Subtract Time",
    description: "Add or subtract time from a given input",
    key: "formatting-add-subtract-time",
    version: "0.0.5",
    type: "action",
    props: {
        ...commonDateTime.props,
        operation: {
            label: "Operation",
            description: "Whether to add or subtract time.",
            type: "string",
            options: Object.values(OPERATION_OPTIONS),
        },
        duration: {
            label: "Duration",
            description: "The duration for the operation. You can use the shorthand duration, for example: `1s`, `1m`, `1h`, `1d`, `1w`, `1y` equal one second, minute, hour, day, week, and year respectively",
            type: "string",
        },
        outputFormat: {
            propDefinition: [
                app,
                "outputFormat",
            ],
        },
    },
    methods: {
        ...commonDateTime.methods,
        getOperationMilliseconds(str) {
            let result = 0;
            const { second, minute, hour, day, week, year, } = DATE_TIME_UNITS;
            Object.entries({
                s: second,
                m: minute,
                h: hour,
                d: day,
                w: week,
                y: year,
            }).forEach(([identifier, multiplier,]) => {
                const substr = str.match(new RegExp(`[0-9]+\\s*${identifier}`))?.[0];
                if (substr) {
                    const value = Number(substr.match(/[0-9]+/));
                    result += value * multiplier;
                }
            });
            return result;
        },
    },
    async run({ $ }) {
        const { operation, duration, outputFormat, } = this;
        const dateObj = this.getDateFromInput();
        const value = dateObj.valueOf();
        let amount = this.getOperationMilliseconds(duration);
        if (operation === OPERATION_OPTIONS.SUBTRACT)
            amount *= -1;
        const result = value + amount;
        const format = outputFormat ?? this.inputFormat ?? DEFAULT_FORMAT_VALUE;
        try {
            const { outputFn } = DATE_FORMAT_PARSE_MAP.get(format);
            const output = outputFn(sugar.Date.create(result));
            $.export("$summary", `Successfully ${operation === OPERATION_OPTIONS.SUBTRACT
                ? "subtracted"
                : "added"} time`);
            return output;
        }
        catch (err) {
            console.log("Error parsing date", err);
            throw new ConfigurationError("**Parse error** - check your input and if the selected format is correct.");
        }
    },
});

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Formatting	`app`	`app`	This component uses the Formatting app.
Input Date	`inputDate`	`string`	A valid date string, in the format selected in Input Format. If the format is not set, Pipedream will attempt to infer it using the parser from Sugar Date library
Input Format	`inputFormat`	`string`	Select a value from the drop down menu:`Sun Jan 22 23:04:05 +0000 2006January 22 2006 23:04:05January 22 2006Jan 22 20062006-01-22T23:04:052006-01-22T23:04:05+00002006-01-22 23:04:05 +00002006-01-22 23:042006-01-2201-22-200601/22/200601/22/0622-01-200622/01/200622/01/06Unix time (seconds) Eg. 1137971045Unix time (milliseconds) Eg. 1137971045000`
Operation	`operation`	`string`	Select a value from the drop down menu:`AddSubtract`
Duration	`duration`	`string`	The duration for the operation. You can use the shorthand duration, for example: `1s`, `1m`, `1h`, `1d`, `1w`, `1y` equal one second, minute, hour, day, week, and year respectively
Output Format	`outputFormat`	`string`	Select a value from the drop down menu:`Sun Jan 22 23:04:05 +0000 2006January 22 2006 23:04:05January 22 2006Jan 22 20062006-01-22T23:04:052006-01-22T23:04:05+00002006-01-22 23:04:05 +00002006-01-22 23:042006-01-2201-22-200601/22/200601/22/0622-01-200622/01/200622/01/06Unix time (seconds) Eg. 1137971045Unix time (milliseconds) Eg. 1137971045000`

Action Authentication#

The Formatting API does not require authentication.

About Formatting#

Pre-built actions to make formatting and manipulating data within your workflows easier.

More Ways to Connect Formatting + AWS#

Other Popular Integrations#

[Date/Time] Add/Subtract Time with Formatting API on New Scheduled Tasks from AWS API

AWS + Formatting

[Date/Time] Add/Subtract Time with Formatting API on Redshift - Updated Row from AWS API

Pipedream makes it easy to connect APIs for Formatting, AWS and 2,700+ other apps remarkably fast.

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

1-24of2,700+apps by most popular

1
-
24
of
2,700+
apps by most popular