How to Implement IP-based Rate Limiting in Node.js using Data Stores?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

How do I use data stores to rate limit a workflow by client IP address in Node.js? I’d like to limit requests to 5 calls per minute, and return an HTTP 429 status code from my workflow if the rate limit on a specific IP is hit.

2

This content can’t be displayed.

3

interesting hallucination:

const { datastore } = require("
@pipedream
/platform");
4

it should be using import instead isn`t?

5

Data stores are exposed through props, of type data_store

6

it should have ideally pulled this doc as a source: Using Data Stores

7

that’s another interesting issue — the query didn’t return any sources. I’ll put that on the list to see if I can improve that

8

Aaaah sure, it’s true…

9

I wonder if we just need more examples of prop usages like this to help draw the line between imports and data stores

10

So LangChain has this idea of “tools”, where a LangChain agent can figure out the best tool to use for a question. In this case, I’d rather route the user to the ai.m.pipedream.net service when they’re asking a code generation question, since that’s optimized for generating Pipedream components

11

that can all be abstracted from Mario — the aiqa service would handle the redirection transparently

12

ai.m.pipedream.net returned

import { v4 as uuidv4 } from "uuid";

export default defineComponent({
  props: {
    ip: {
      type: "string",
      label: "Client IP Address",
    },
    dataStore: {
      type: "data_store",
      label: "Data Store",
    },
  },
  async run({ $ }) {                                                                                                                                                                                                      const ip = this.ip;
    const dataStore = this.dataStore;
    const rateLimit = 5;
const windowSize = 60 * 1000; // 1 minute in milliseconds                                                                                                                                                

    const currentTime = Date.now();
    const key = `rate_limit_${ip}`;

    let requestTimestamps = await dataStore.get(key) || [];

    // Remove timestamps older than the window size
    requestTimestamps = requestTimestamps.filter(
      (timestamp) => currentTime - timestamp < windowSize
    );

    if (requestTimestamps.length >= rateLimit) {
      // Rate limit exceeded, return HTTP 429 status code
      return $.respond({
        status: 429,
        body: "Rate limit exceeded. Please wait before making more requests.",
      });
    }

    // Add the current request timestamp and update the data store
    requestTimestamps.push(currentTime);
    await dataStore.set(key, requestTimestamps);

    // Process the request and return the response
    const requestId = uuidv4();
    return {
      requestId,
      message: "Request processed successfully.",
    };
  },
});