This topic was automatically generated from Slack. You can find the original thread here.
Nacho Caballero : Is there any way to know which step triggered this error? “Workflow Payload Limit Exceeded”
Dylan Sather (Pipedream) : Unfortunately not right now. Do you have any candidate steps with large logs / step exports?
Nacho Caballero : I don’t. I think this is a false positive related to the OOM error I PMed you about
Giao Phan : The heuristic for this is pretty unique, are you handling large requests in that workflow?
Nacho Caballero : I’m using fs.readFileSync(/tmp/myfile.mp3).toString() on a 30MB file. I need to include the raw content as part of the request and it doesn’t work if I stream it
Dylan Sather (Pipedream) : are you returning the data from that file in a step export, or just using it within a particular step?
Nacho Caballero : Doh! I didn’t think was returning it, but since it’s part of the this.request, I am 
Dylan Sather (Pipedream) : Are you able to re-write your workflow to avoid passing that specific data as a step export, and instead writing the contents of that file to /tmp , then reading it later, where you need it?
Dylan Sather (Pipedream) : Also, I think the OOM error may be related to the fact that you copied the workflow. IIRC we increased the memory of the original workflow because you were working with so many large files that it required an increase. I’ll raise the memory of this copy, as well, to try to alleviate that issue
Dylan Sather (Pipedream) : I’m not 100% sure but I believe this.response will still contain the original request data. do you know if that’s the case? Either way I’d still suggest trying to avoid passing the data via step exports if at all possible - I know it’s a bit onerous but it should avoid these issues.
I raised the copy’s memory btw, let me know if you get any more OOM errors
Giao Phan : Yeah, I would expect response to have all the data, I would probably use streams if possible so everything doesn’t get buffered into memory.
Giao Phan : Logging it will give you the same problem
Nacho Caballero : Thanks, Giao. I can’t use streams in this case because the API doesn’t allow it
Nacho Caballero : Same error
Giao Phan : this is the oom or the payload size error?
Nacho Caballero : payload size
Nacho Caballero : I also tried re-sending the event and re-emiting a previous event. All I’m exporting now are two small strings and I’m not logging anything. Same result: Workflow Payload Limit Exceeded
Dylan Sather (Pipedream) : if you disable that step completely, does the workflow run?