Is logging data to the Workflow instead of job steps beneficial for easier summary access and debugging?

1

This topic was automatically generated from Slack. You can find the original thread here.

I was just doing some debugging and a feature thought. I’m curious if others think this is a good idea and worth requesting. Basically, it is the idea to log data to the Workflow and not to the job step. Now, each run of a workflow has as log for each step. What I’m thinking though is it would be helpful to log some items to the top of the workflow so when I’m viewing the past runs I have a summary of different info at the top. I’ve got a couple long workflows, so it was a pain scrolling and then correlating across a couple steps. Where if I could log to an overall workflow log and it is display at the top line the execution time and details. What do you think?

2

Hi Tim, we love hearing feedback!

Did you also see the new Event History area in the dashboard that centralizes your logs?

3
4

noticed this on the side menu, let me click on it

5

OK, I see event history. I think the idea of being able to log something to a run of a workflow, such that it is displayed at the top of a workflow details is helpful rather than in each job step.

I think i should say I use python code blocks a bit. And I like breaking them into different steps. So being able to log a few essential details such that it is Workflow log and not workflow step log would be nice. (not sure if I’m using the exact terms you use, feel free to correct me).

6

Can you describe the difference between a workflow log vs a step log?

7

Basically overall status that I’d want to see at the top of a workflow, not having to pick through each step to find the details.

8

Maybe I could shoot a screenshot video in a short bit

9

Right, just curious what other status you’d like to see other than the overall status that’s currently available when you select a specific event

10

There’s a banner that describes if the workflow executed successfully, time spent, steps executed, etc for an overall summary

11

Right - I’d like to add some other messages to that.

12

Quick example, I’ve got a workflow that runs daily that archives some data if it old enough. In the Job Step I have a log message for that. In my mind though this info is an overall result of the entire workflow, and I thought if I’d log it to the top of the workflow for easy consumption if I am reviewing workflows or debugging, that would be nice. In this case, it might be “0 items archived” or “10 items archived”.

another example, is I use data stores, so at the end of a workflow I may want to log the state of the data store. Just so if I need to reference the info for debugging. I recently had a bug with my code updating a google sheet, I think i could have found the problem quicker if I wasn’t clicking through a bunch of job steps and their logs individual logs.

13

It’s a hack but you could use $.flow.exit(${items} items archived) in an arbitrary last step to use the exit early feature to show that message at the top of the page

14

But that makes a lot of sense, basically summary metadata for the workflow

15

Good idea. I do use this exit capability when prematurely exiting for various reason, I could use it as a success message too.

16

Definitely encourage posting this as a feature request though: Sign in to GitHub · GitHub

Maybe something like $.flow.result() or $.flow.summary() could be added to track the overall results of the flow.