Is There a Persistent Datastore Accessible Across Workflow Steps to Improve Code Efficiency?

This topic was automatically generated from Slack. You can find the original thread here.

I wonder if I’m missing something. Is there an object/datastore/location that is persistent only for a workflow run and accessible across steps? If I have a workflow, and it has say three python block in it, because I like to have my code split appropraite for the work it is doing, i feel like I’m doing a bit of boiler plate to copy the results of the last step into a variable at the top the next step and then return at the bottom of that step. I guess what I’d like is an object I could build across each step and keep adding/taking away from that object (JSON) across steps without the reloading, so to speak. Make sense?

A data store is for persistance storage across workflows. So kind of like that object, but just within a workflow.

, I imagine what you need is the global state within a workflow. Unfortunately, Pipedream doesn’t offer it natively. There’re some workarounds, but the method making sense to me is to group your Python code to one action (with multiple functions), as they’re related to each other

Hi , yes, a global state within a workflow, that is a nice description.

For ease of working, debugging, and since some of the python blocks I copy to other workflows, I’ll just continue with the pattern I have. Placing it all into a single large block, maybe it helps in execution and cleanliness, but it also makes a very large code chunks to review in the online editor.