Sorry, you will need to remove and re-add the trigger — there is not currently any way to update the version of sources today, just actions.
merged, you can test right now, please let me know the result for you
Ok. I will reconfigure triggers
Hello. @UMT4G7E5P I have reconfigured triggers, and all new sources now show version 0.0.6. It seems that the workflow was able to pull a failed task from 10:43 am this morning, which was roughly 3 hours ago in the time zone. This is good news. I have also reconfigured the test task to run at 2:30 am and will test the result tomorrow morning. I will let you know the outcome.
I have a couple of important points to discuss. Firstly, I find it quite cumbersome to reconfigure all the sources each time a new version is released. Is there a more efficient way to obtain the updated version? Are there any plans in place to automate this update process?
Secondly, I wanted to explore the possibility of having a single trigger that can check all the schemas in the database. This would greatly simplify our procedures and minimize the number of checks and queries required for Snowflake.
Lastly, I would like to inquire about the progress of the feature request to select specific tasks using wildcard or regex within a schema. Any updates on this matter would be greatly appreciated.
Thank you for your attention and assistance.
Yes, completely hear you on needing to deploy new sources. That’s on our radar. Hopefully you won’t have to do this a lot more if we’ve fixed the core bug. CC I think we could support watching multiple triggers. @U02GNNVAGS0 we’d just set that prop to a string[] and then run the failed task check on each schema in a loop within the run method. Can you add a ticket for that change and work on that whenever you free up?
@U02GNNVAGS0 it look like you also took that ticket for adding regex support — were you working on that?
About multiple schemas check, yeah, this is totally possible. I will start to work on this.
About Regex, It was in QA, but QA returned due some inconsistencies in the solution, You can check the current status here: https://github.com/PipedreamHQ/pipedream/pull/6736
I expect having both ready for review at Monday max
It looks like the fix is working. Thank you.
On another note. Having ability to check for failed tasks in all schemas would catch tasks created in new schemas automatically.
you can find the regex and schemas list in this PR: [Source] Snowflake - Add regex to getFailedTasksInDatabase by vellames · Pull Request #6736 · PipedreamHQ/pipedream · GitHub
Awesome. Thank you.
Hey . The PR was merged
You can use regex and multiple schemas
Thank you. Let me try it later today.
Posted thread to Discourse: Update on Pipedream Missing Tasks After Schedule Change from 15 Minutes to 6 Hours?
Hi @UMT4G7E5P I am trying to test regex for DB and Schema selection. Is there documentation link or a sample I can view? Thank you.
Posted thread to Discourse: Update on Pipedream Not Picking Up Missed Tasks After Schedule Change from 15 Minutes to 6 Hours?
Hello I just realized that some of our previous communication might have been a bit mixed up. Upon further understanding, I now realize that the regex fix is exclusively applied to the names of Snowflake tasks. While this is indeed great, I was actually hoping for the ability to use regex or wildcard selection for any Snowflake schema. My intention was to ensure that if any new schemas are created, this source would be able to detect them. Is it possible to implement this functionality? Thank you.
We can address this, sorry for the misunderstanding. It’s definitely interesting to think of other places we can apply regex matches like this