Batch Insert Rows with Supabase API on New Links on a Subreddit from Reddit API

Pipedream makes it easy to connect APIs for Supabase, Reddit and 2,400+ other apps remarkably fast.

Trigger workflow on

New Links on a Subreddit from the Reddit API

Next, do this

Batch Insert Rows with the Supabase API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Reddit trigger and Supabase action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Links on a Subreddit trigger
1. Connect your Reddit account
2. Configure Polling schedule
3. Select a Subreddit
Configure the Batch Insert Rows action
1. Connect your Supabase account
2. Configure Table
3. Select a Source of data
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Links on a Subreddit on Reddit

Description:Emit new event each time a new link is added to a subreddit

Version:0.1.0

Key:reddit-new-links-on-a-subreddit

View on GitHub

Reddit Overview#

The Reddit API unlocks the potential to automate interactions with one of the largest online communities. With Pipedream, you can tap into the Reddit ecosystem to monitor trends, engage with audiences, or curate content. Whether it's tracking mentions of your brand, auto-posting to subreddits, or gathering data for analysis, the API offers a breadth of functionalities — from getting posts and comments to creating new threads and managing account settings.

Trigger Code#

import common from "../common.mjs";
import get from "lodash/get.js";
const { reddit } = common.props;

export default {
  ...common,
  type: "source",
  key: "reddit-new-links-on-a-subreddit",
  name: "New Links on a Subreddit",
  description: "Emit new event each time a new link is added to a subreddit",
  version: "0.1.0",
  dedupe: "unique",
  props: {
    ...common.props,
    subreddit: {
      propDefinition: [
        reddit,
        "subreddit",
      ],
    },
  },
  hooks: {
    async deploy() {
      // Emits 10 sample events on the first run during deploy.
      const redditLinks = await this.reddit.getNewSubredditLinks(
        get(this.subreddit, "value", this.subreddit),
        {
          limit: 10,
        },
      );
      const { children: links = [] } = redditLinks.data;
      if (links.length === 0) {
        console.log("No data available, skipping iteration");
        return;
      }
      const { name: before = this._getBefore() } = links[0].data;
      this._setBefore(before);
      const previousEmittedEvents = {};
      const keys = [];
      links.reverse().forEach((event) => {
        this.emitRedditEvent(event);
        previousEmittedEvents[event.data.name] = true;
        keys.push(event.data.name);
      });
      this._setCache(previousEmittedEvents);
      this._setKeys(keys);
    },
  },
  methods: {
    ...common.methods,
    generateEventMetadata(redditEvent) {
      return {
        id: redditEvent.data.name,
        summary: redditEvent.data.title,
        ts: redditEvent.data.created,
      };
    },
    async isBeforeValid(before) {
      // verify this link still exists on the subreddit
      const { data } = await this.reddit.getSubredditByName(before);
      const isOnSubreddit = data?.children[0]?.data?.subreddit === this.subreddit?.value;
      return isOnSubreddit;
    },
  },
  async run() {
    let redditLinks;
    const {
      cache: previousEmittedEvents,
      keys,
    } = await this.validateBefore(this._getCache(),
      this._getBefore(),
      this._getKeys());
    do {
      redditLinks = await this.reddit.getNewSubredditLinks(
        get(this.subreddit, "value", this.subreddit),
        {
          before: this._getBefore(),
        },
      );
      const { children: links = [] } = redditLinks.data;
      if (links.length === 0) {
        console.log("No data available, skipping iteration");
        break;
      }
      const { name: before = this._getBefore() } = links[0].data;
      this._setBefore(before);

      links.reverse().forEach((event) => {
        if (!previousEmittedEvents[event.data.name]) {
          this.emitRedditEvent(event);
          previousEmittedEvents[event.data.name] = true;
          keys.push(event.data.name);
        }
      });
    } while (redditLinks);
    this._setCache(previousEmittedEvents);
    this._setKeys(keys);
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Reddit	`reddit`	`app`	This component uses the Reddit app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
Polling schedule	`timer`	`$.interface.timer`	Pipedream polls Reddit for events on this schedule.
Subreddit	`subreddit`	`string`	Select a value from the drop down menu.

Trigger Authentication#

Reddit uses OAuth authentication. When you connect your Reddit account, Pipedream will open a popup window where you can sign into Reddit and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Reddit API.

Pipedream requests the following authorization scopes when you connect your account:

accountidentityedithistoryreadsubmitmysubredditssubscribe

About Reddit#

Reddit is a network of communities based on people's interests. Find communities you're interested in, and become part of an online community!

Action#

Batch Insert Rows on Supabase

Description:Inserts new rows into a database. [See the documentation](https://supabase.com/docs/reference/javascript/insert)

Version:0.0.1

Key:supabase-batch-insert-rows

View on GitHub

Supabase Overview#

Supabase is a real-time backend-as-a-service that provides developers with a suite of tools to quickly build and scale their applications. It offers database storage, authentication, instant APIs, and real-time subscriptions. With the Supabase API, you can perform CRUD operations on your database, manage users, and listen to database changes in real time. When integrated with Pipedream, you can automate workflows that react to these database events, synchronize data across multiple services, or streamline user management processes.

Action Code#

import supabase from "../../supabase.app.mjs";
import fs from "fs";
import { parse } from "csv-parse/sync";

export default {
  key: "supabase-batch-insert-rows",
  name: "Batch Insert Rows",
  description: "Inserts new rows into a database. [See the documentation](https://supabase.com/docs/reference/javascript/insert)",
  version: "0.0.1",
  type: "action",
  props: {
    supabase,
    table: {
      propDefinition: [
        supabase,
        "table",
      ],
      description: "Name of the table to insert rows into",
    },
    source: {
      type: "string",
      label: "Source of data",
      description: "Whether to enter the row data as an array of objects or to import from a CSV file",
      options: [
        "Array",
        "CSV File",
      ],
      reloadProps: true,
    },
  },
  async additionalProps() {
    const props = {};
    if (this.source === "Array") {
      props.data = {
        type: "string[]",
        label: "Row Data",
        description: "An array of objects, each object representing a row. Enter column names and values as key/value pairs",
      };
    }
    if (this.source === "CSV File") {
      props.filePath = {
        type: "string",
        label: "File Path",
        description: "The path to a csv file in the `/tmp` directory. [See the documentation on working with files](https://pipedream.com/docs/code/nodejs/working-with-files/#writing-a-file-to-tmp)",
      };
    }
    return props;
  },
  methods: {
    parseArray(arr) {
      if (Array.isArray(arr)) {
        return arr.map((item) => {
          return typeof item === "string"
            ? JSON.parse(item)
            : item;
        });
      }
      if (typeof arr === "string") {
        return JSON.parse(arr);
      }
    },
    getRowsFromCSV(filePath) {
      const fileContent = fs.readFileSync(filePath.includes("tmp/")
        ? filePath
        : `/tmp/${filePath}`, "utf-8");
      const rows = parse(fileContent, {
        columns: true,
        skip_empty_lines: true,
      });
      return rows;
    },
  },
  async run({ $ }) {
    const data = this.source === "CSV File"
      ? this.getRowsFromCSV(this.filePath)
      : this.parseArray(this.data);

    const response = await this.supabase.insertRow(this.table, data);

    $.export("$summary", `Successfully inserted rows into table ${this.table}`);
    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Supabase	`supabase`	`app`	This component uses the Supabase app.
Table	`table`	`string`	Name of the table to insert rows into
Source of data	`source`	`string`	Select a value from the drop down menu:`ArrayCSV File`

Action Authentication#

Supabase uses API keys for authentication. When you connect your Supabase account, Pipedream securely stores the keys so you can easily authenticate to Supabase APIs in both code and no-code steps.

Supabase provides a Service Key to get started. You can find the service_role in the API Settings page.

Finding the Supabase subdomain

Batch Insert Rows with Supabase API on New Links on a Subreddit from Reddit API

Pipedream makes it easy to connect APIs for Supabase, Reddit and 2,400+ other apps remarkably fast.

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

1-24of2,400+apps by most popular

1
-
24
of
2,400+
apps by most popular