How do I trigger a cron scheduler workflow multiple times throughout the day?

Not sure if this is entirely related, lmk if I should open a new thread for this instead.

CRON schedules seem to not work for me at all. I’m trying to have it run a few times for a specific schedule. So when a video is scheduled to be uploaded at 8pm, I want to check a few times right after 8pm to catch it asap. So I have a CRON Expression like this 0,30 0-3 20 * * * (check at 20:00:00 and then every 30 seconds until 3 past). That is also what it says when I set the trigger:


However, this seems to just not do anything. When I checked in the configuration it says “Error: Seconds resolution not supported.” next to the CRON Expression.
Tried this in both Chrome and Edge, neither worked.
What do I do?

Hi @NukeOfficial

I think it may be a bug where our backend doesn’t support the extended cron syntax, but the frontend is showing faux-support.

Especially since v1 shows an error for me:

1 Like

Ah, I see. So to achieve what I want right now, I’d have to just make multiple workflows that each check at a specific time, right?

@NukeOfficial that is one way, or if you’re building with the v2 workflow builder you can use multiple Timer triggers for one workflow.

That way you don’t have to copy your workflow and maintain multiple versions of the same workflow. Hope this helps!

1 Like

Hey, I finally got this set up and tested, but it seems like it’s not detecting the additional triggers. I tried setting up the same trigger but with a different time, as well as a time trigger but neither works.
With setting up multiple triggers in one workflow for the same thing but for different times, only the first one actually gets activated.
image
All of these triggers have the exact same configuration (using the same Data API account and getting information from the same channel) except for the time, which is a different hour for each one.
Still only the first trigger actually triggers.

Hi @NukeOfficial

Is it possible that all of these triggers are using the same YouTube Source?

YouTube Sources are emitted on a timer themselves, so if you’re reusing it it will only emit one event in that polling window.

Ah, I’m not sure.
If I click on Results>Details for the triggers, it shows me different sources. The first and last trigger are set to pipedream/new-videos-in-channel.mjs at master · PipedreamHQ/pipedream · GitHub and the 2nd and 3rd trigger are set to https://github.com/PipedreamHQ/pipedream/tree/master/components/youtube/sources/new-videos-in-channel/new-videos-in-channel.js which just goes to a 404 page.
Also, I’m not sure if this means anything but it’s the only thing that’s different between the first and the other triggers, if I go to Results>Exports the first Trigger’s etag (under steps.trigger>event) is different from all the others

but also, what would be the correct way to do what you said earlier

you can use multiple Timer triggers for one workflow.

if I shouldn’t use the Youtube API Upload trigger multiple times?

Hi @NukeOfficial,

Originally the trigger was a Timer trigger, but it looks like you’ve switched to a YouTube sourced trigger instead. Which will definitely work, but it appears you’re running into YouTube rate limits.

YouTube will only allow a certain number of requests to their API in a certain period of time.

If you create multiple YouTube sources, and they are all using the same YouTube account connection, all of those sources are subject to the same rate limit - not separate ones.

If YouTube had a webhook system, that would be perfect for subscribing to instant events whenever new videos are uploaded.

Since you know the general schedule of when the channel’s uploading these videos, I would just try to match it as close as possible with 1 trigger.

Hey,

yeah I’m trying to match the upload times, but I’m running into these issues because there are multiple uploads per day.
Initially I tried this through a CRON Expression at first, but those didn’t seem to trigger properly.
Doing this through multiple separate workflows technically works, but runs into the issue that the workflows don’t know which videos the other workflows already found. So every webhook tries to get all videos of the last 24 hours.
Doing this through one workflow with multiple triggers also didn’t work, though I’m not entirely sure why. Even if each trigger uses the same account, it should only ask the Youtube API once when the specified time comes around, no?

I feel like I’m missing something obvious here, but I’m not sure how to set this up :sweat_smile:

Hi @NukeOfficial

Initially I tried this through a CRON Expression at first, but those didn’t seem to trigger properly.
Doing this through multiple separate workflows technically works, but runs into the issue that the workflows don’t know which videos the other workflows already found.

This is a perfect use case for using the new Data Stores feature.

When a new video is detected by one of your workflows, you can check the Data Store if the video has already been processed.

Here’s our documentation on how to use them:

Here’s a video that uses a Data Store to build a list of Twitter followers, but you can follow it to build a list of YouTube video IDs:

Even if each trigger uses the same account, it should only ask the Youtube API once when the specified time comes around, no?

For app based triggers like YouTube, a new Source is created. Each of your YouTube source has it’s own timer that asks the YouTube API for new videos:

So if you have created several Sources each of them are polling at different intervals for new videos.

Oh I see, thanks for letting me know! I’ll try this out then.

So if you have created several Sources each of them are polling at different intervals for new videos.

yeah, I was just curious why it would be running into timeouts since I didn’t set up the sources to request information on a timer. Each source should only make a request once per day, at the specified time. And since each of the 4 sources are multiple hours apart it should be fine, no?

as a small update: Handling it through the Data Stores appears to be working well! Also the CRON Schedules do work, but only if you act like the Seconds indicator doesn’t exist^^

1 Like