This topic was automatically generated from Slack. You can find the original thread here.
in my workflow, when I test step by step for an incoming event, everything works fine
but when running the whole workflow for the event, I have an error saying it exceeds default memory of the workflow
can you help me how to figure out this?
It sounds like you’re encountering an “Out of Memory” error when running your entire workflow. Here are some steps you can take to address this issue: 1. Increase Memory Limit: You can increase the memory available to your workflow. This can be done in your workflow’s settings. Pipedream allows you to adjust the memory up to a certain limit, and this can help if your workflow is processing a lot of data. Keep in mind that increasing memory will affect the number of credits charged per execution. You can find more information on this in the Pipedream documentation. 2. Optimize Workflow Steps: If your workflow has many steps, it might be consuming more memory than necessary. Each step in a workflow runs as a separate process, which incurs memory overhead. Try to optimize your workflow by reducing the number of steps or combining steps where possible. 3. Stream Data: If you’re handling large files or data objects, consider streaming the data instead of loading it entirely into memory. This can significantly reduce memory usage. You can use techniques like streaming files to/from disk, as described in the Node.js code examples. 4. Debugging: Since your workflow works fine when testing step by step, try to identify which step or combination of steps is causing the memory issue when running the entire workflow. You can add logging to your steps to monitor memory usage and identify bottlenecks. If you continue to experience issues, you can visit Pipedream Support for further assistance.
can you help me with this?
Have you tried increasing the memory of the workflow?
ok let me try that
I increased memory to 1024mb, but same
Did you see the link to the docs about resolving memory issues? You may need to keep increasing and testing, to find the right configuration.