Hi,
I have a scheduler running on a daily basis (weekdays only), which involves getting some data from Firestore and APIs and then inputting it into a Google Spreadsheet. However, there seems to be an inconsistency when it runs or gets deployed, in the sense that it fails and throws the following error: ‘Layers consume more than the available size of 262144000 bytes’. I’m not entirely sure where to look to address this error.
Appreciate any suggestions or advice. Thanks in advance!
Hi @chowdud,
Welcome to the Pipedream community, happy to have you here!
That’s an interesting error, could you perhaps share the code from the workflow steps, or share the workflow itself with me? My email address is pierce@pipedream.com.
To share a workflow, there’s a blue “Share” button in the top right of the workflow.
Hi Pierce, I have invited/shared the Pipedream workflow with the provided email.
Xiao
Hi @chowdud, got the invite thanks.
It appears that your Firebase SDK step is hitting a timeout limit intermittently due to the latency of the multiple API calls you’re making in 1 step.
My recommendation is to break this up into multiple steps, that way the execution time needed to perform all of these tasks are split across steps.
After each task is performed, you can share the data it retrieved or transformed or uploaded downstream to other steps.
Here’s a link to how to share data between steps in a Pipedream workflow: What are steps?
Hi Pierce,
Thanks for the update. I will refactor the workflow based on your suggestion!
@chowdud sure thing, I hope it solves your problem. At the very least it will narrow down which of your API calls steps might be the source of the problem.
Pretty confident that splitting up the steps will solve it, but best of luck.