I could build a data structure export in a step, as you noted in a linked article. Next I could try to edit/add to that structure from a different step. I was always copying the previuos step into the current step, editing and then exporting in the current step. So in my example above what if I built the structure and exported in Step 1. Then all steps read and write to that Step 1 data structure, would that work?
Are you familiar with the concurrency and throttling controls? We recommend setting a concurrency of 1 to workflows where you need data stores to have a consistent state across runs.
ah-ha! Thank you! I’ll read that.
I’ve been in Pipe Dream a couple days now and going deeper. Thank you for showing me more
Step exports are immutable, so if you wanted to change those values within the workflow execution, I would recommend a data store. You can always clear the contents of the data store at the start of the run with the Delete All Records action, so you have a fresh data store for each.
Happy you are going deeper! These more advanced features are where Pipedream really shines.
Right, I like Data Store, and now that I see the article on Concurrency and Throttling, this clears up what I was stumbling upon in my long post above. So, agree, Data Store is best.
Yes, these advance features are why I’m here, that and able to code in Python. I’ll be moving a handful of automations here I have at another vendor once I get this new one I’m cooking up working.
awesome! Let us know if you hit any other issues / have any questions