Would I need to reconfigure triggers?
What version should I be lookin for in config?
It will be deployed some minutes after the PR status change to Merged. It is in QA right now, this is the last step before merged. I can’t precisely give you a deadline, but I imagine that would be good to go in the next days. Version is 0.0.6. You don’t need the reconfigure the whole workflow, just update your trigger as commented before.
Sorry, you will need to remove and re-add the trigger — there is not currently any way to update the version of sources today, just actions.
merged, you can test right now, please let me know the result for you
Ok. I will reconfigure triggers
Hello. @UMT4G7E5P I have reconfigured triggers, and all new sources now show version 0.0.6. It seems that the workflow was able to pull a failed task from 10:43 am this morning, which was roughly 3 hours ago in the time zone. This is good news. I have also reconfigured the test task to run at 2:30 am and will test the result tomorrow morning. I will let you know the outcome.
I have a couple of important points to discuss. Firstly, I find it quite cumbersome to reconfigure all the sources each time a new version is released. Is there a more efficient way to obtain the updated version? Are there any plans in place to automate this update process?
Secondly, I wanted to explore the possibility of having a single trigger that can check all the schemas in the database. This would greatly simplify our procedures and minimize the number of checks and queries required for Snowflake.
Lastly, I would like to inquire about the progress of the feature request to select specific tasks using wildcard or regex within a schema. Any updates on this matter would be greatly appreciated.
Thank you for your attention and assistance.
Yes, completely hear you on needing to deploy new sources. That’s on our radar. Hopefully you won’t have to do this a lot more if we’ve fixed the core bug. CC I think we could support watching multiple triggers. @U02GNNVAGS0 we’d just set that prop to a string[] and then run the failed task check on each schema in a loop within the run method. Can you add a ticket for that change and work on that whenever you free up?
@U02GNNVAGS0 it look like you also took that ticket for adding regex support — were you working on that?
About multiple schemas check, yeah, this is totally possible. I will start to work on this.
About Regex, It was in QA, but QA returned due some inconsistencies in the solution, You can check the current status here: [Source] Snowflake - Add regex to getFailedTasksInDatabase by vellames · Pull Request #6736 · PipedreamHQ/pipedream · GitHub
I expect having both ready for review at Monday max
It looks like the fix is working. Thank you.
On another note. Having ability to check for failed tasks in all schemas would catch tasks created in new schemas automatically.
you can find the regex and schemas list in this PR: https://github.com/PipedreamHQ/pipedream/pull/6736
Awesome. Thank you.
Hey . The PR was merged
You can use regex and multiple schemas
Thank you. Let me try it later today.
Posted thread to Discourse: Update on Pipedream Missing Tasks After Schedule Change from 15 Minutes to 6 Hours?
Hi @UMT4G7E5P I am trying to test regex for DB and Schema selection. Is there documentation link or a sample I can view? Thank you.