Hello expert,
My workflow runs hundreds of time every day, it has several steps where it gets data via http requests.
There is one particular step which takes between 3 and 60 seconds to execute.
5 times it runs ok within a few seconds, then comes to long one.
The http request is always the same, and i checked in detail with the remote system and there the response was always created in less than 1 second.
How can the same step in pipedream have such a variability in runtime?
thanks,
Johannes
Hmm strange, could you please share some details/code?
Thanks
Hi, the concerned step is just an http request using the pipedream functional UI.
The http request queries order item details from antoher system, the response is always of the same length, it contains the order item fields.
Normal execution: under 1 second:
but very often this happens: Execution more than 5 seconds or up to 10 or more seconds:
I found that by increasing the memory of the workflow from 256 to 512, i can keep the runtime always in the range of 5-8 seconds. With 256 MB memory, it increases to sometimes 40-60 seconds, but there are fast runs as well in 2.-3 seconds.
I did a thorough check on the other system and there, all requests are processed in under 1 seconds, so this is not the causing the issue.
With 5 seconds, the issue is more or less under control, though I do not like to have double invocations now because of the memory increase. It would be helpful to know the root cause.
Thank you,
Johannes
Okay, may be could check the timings when the request was dispatched from pipedream vs when it reached your API server vs you can get a response back. Not sure if that would be of any help but it may help you identify or get a hint of the issue.
Thanks