Only run once every xx minutes issue

Help
1

Note: I’m using the free/basic level on pipedream. I need this flow so little, that the free level of credits is enough for me.

I’m new here. I have a flow that is triggered from an email sent pipedream. I want this flow to only run at most once every 15 minutes…In other words, if a second email is sent 5 mins later, the final step of my flow (a twillio phone call) isn’t executed.

How I currently have the flow is that it refrences a google spreadsheet. I have cell A2 as a time keeper that stores the last time the flow was executed in full. The flow goes as follows:

-email trigger
-read google sheets date cell
-node code that looks at that time code, current time code, and subtracts the two, and converts that time difference into minutes
-filter: If that differnece is < xx minutuest, stop flow.
-update cell with current time code
-execute twillio command.

This works in my testing and if I manually email this trigger, all works great. However, lately, I ran into a string of “timeout” issues. And it seems to timeout on the update cell with current time.

From what I gather, this nothing wrong with my code, but rather a weakness of google sheets just not being fast enough via API call.

So here is my question, is there a better/more effieient way to do this “only run once every xx minutes” concept. From what I gathered, with my free level of Pipedream, I have no way of storing any sort of data that one flow could save locally so a different flow could read. I’d love to hear any suggestions yall may have.

2

Hi @drew.nadig,

Absolutely, there is a better and more efficient way to implement “run only once every XX minutes” logic in Pipedream, especially for your use case and on the Free plan.

Recommended Solution: Use a Pipedream Data Store

Pipedream offers Data Stores, which are built-in key-value stores you can use to persist data between workflow runs. They are available on the Free plan and are much faster and more reliable than using Google Sheets for this purpose.

Why use Data Stores?

  • Fast and reliable (no external API latency or quota issues)
  • No extra credits required for basic usage
  • Perfect for storing timestamps or simple state

How to Implement Your “Throttle” Logic with a Data Store

Replace your Google Sheets logic with a Data Store step. Here’s how your workflow would look:

  1. Email trigger
  2. Read last run timestamp from Data Store
  3. Node.js code step: Compare current time to last run time
  4. Filter step: If difference < 15 minutes, exit
  5. Update Data Store with current time
  6. Twilio call step

Example Node.js Code for Data Store Throttle

Add a Data Store prop to your step. Here’s a sample code step you can use:

export default defineComponent({
  props: {
    data: { type: "data_store" },
  },
  async run({ steps, $ }) {
    const now = Date.now();
    const lastRun = await this.data.get("last_run") ?? 0;
    const diffMinutes = (now - lastRun) / 1000 / 60;

    if (diffMinutes < 15) {
      $.flow.exit(`Last run was ${diffMinutes.toFixed(2)} minutes ago. Exiting.`);
    }

    // Update the last_run timestamp
    await this.data.set("last_run", now);

    // Continue to Twilio step
    return { message: "Proceeding to Twilio call." };
  },
});
  • Add a Data Store to your workflow (in the UI, add it as a prop to this step).
  • This code will exit early if the last run was less than 15 minutes ago.
  • If enough time has passed, it updates the timestamp and allows the workflow to continue.

Summary

  • Stop using Google Sheets for throttling—use Pipedream Data Stores instead.
  • This method is faster, more reliable, and works on the Free plan.
  • You can find more info in the Pipedream Data Stores documentation.

If you need a step-by-step guide on setting this up in the UI, let me know!

I’m a bot powered by Pipedream and GPT-4. I’m still learning, so please double-check my answers and code! Learn how to build your own.

3

Ok…this is awesome and works with one issue. It Resets the “last run” time on EVERY execution. If the system sees the last run was less than 15 min, I don’t want it to record that run time. As the code sits, if this gets triggered every 5 min, the system would NEVER exit the the time constraint.

So the last run time should be logged in the data store for “successful” runs, however, if the system is triggered and is blocked since it was less than 15 minutes, do NOT record that time in the data store.

4

Figured it out. I edited the code like this. Now it only updates the data store if the time is greater than XX mins.

I also added debug exports so I could see what numbers it is crunching. The code I have that works:

export default defineComponent({
props: {data: { type: “data_store” }, },
async run({ steps, $ }) {
const now = Date.now();
const lastRun = await this.data.get(“last_run”) ?? 0;
const diffMinutes = (now - lastRun) / 1000 / 60;

// Update the last_run timestamp if time is greater than XX mins
if (diffMinutes > 2) {await this.data.set(“last_run”, now);}

// Stop Flow if time is less than xx minutes
if (diffMinutes < 2) {$.flow.exit(Last run was ${diffMinutes.toFixed(2)} minutes ago. Exiting.);}

// Emit debug info 
$.export("debug", {
  now,
  lastRun,
  diffMinutes,
  now_iso: new Date(now).toISOString(),
  lastRun_iso: lastRun ? new Date(lastRun).toISOString() : null,
});


// Continue to Twilio step
return { message: "Proceeding to Twilio call." };

},
});