How to prevent timeout errors and determine their cause in Pipedream?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Hey team, I’m still pretty new to Pipedream but I’m loving it for the most part. My biggest issue is running into Timeout errors pretty regularly. I keep increasing the timeout limit with each error, but it doesn’t seem to reduce the chance of a timeout. Here’s an example of a recent Node step that timed out. Please let me know if you have any suggestions to prevent Timeout errors. If there are better ways to determine what caused the Timeout, that would be great too.

Note, I’ve obscured parts of the URL in the example for security. This code will not work if you try to run it.

import axios from 'axios';

export default defineComponent({
  async run({ steps, $ }) {
    let comment = steps.trigger.event.message
    let externalLeadId = steps.trigger.event.externalLeadId
    const url = `https://import.prod.velocify.com/Update.aspx?Provider=Provider&Client=Client&LeadId=${externalLeadId}&ActionId=131`;

    const headers = {
      'Content-Type': 'application/x-www-form-urlencoded'
    };

    try {
      const response = await [axios.post](http://axios.post)(url, { Comment: comment }, { headers });
      console.log("Response:", response.data);
    } catch (error) {
      console.error("Error:", error);
    }
  },
});
2

I’m sorry to hear you’re experiencing timeout issues. One way to handle timeouts is to use try...catch blocks in your code and implement a retry mechanism. You can leverage $.flow.rerun within a try...catch block to retry a failed API request. Here’s an example of how you can modify your code to include retries: ```
import { axios } from “@pipedream/platform”; export default defineComponent({ async run({ steps, $ }) { let comment = steps.trigger.event.message; let externalLeadId = steps.trigger.event.externalLeadId; const url = https://import.prod.velocify.com/Update.aspx?Provider=Provider&Client=Client&LeadId=${externalLeadId}&ActionId=131; const headers = { “Content-Type”: “application/x-www-form-urlencoded”, }; try { const response = await axios($, { method: “POST”, url: url, data: { Comment: comment }, headers: headers, }); console.log(“Response:”, response.data); } catch (error) { console.error(“Error:”, error); // Retry the request after a delay (e.g., 1000ms) $.flow.rerun(1000); } }, });

3

My code already includes a try…catch block. Shouldn’t that at least log any errors that are encountered? The flows that have timed out do not show any log information. How can I update my code to ensure that any errors are displayed even if the step times out?

4

Hi Anthony - are you iterating over records and applying an async action to each record in your workflow with this issue?

5

Hi Pierce, no iterating, just catching an http post request, checking some values to see if we should end flow then using axios to forward the request. Only async is the axios call.

I’ve enabled support visibility for the workflow in question: https://pipedream.com/@verse/cs-lisa-tate-open-sky-lead-activity-to-velocify-p_pWCd2ad/inspect

6

Have you determined if the API response is performant with a script outside of Pipedream? Or viewed the timing within the workflow?

7

We do offer higher timeout limits on Advanced and above plans, but the only other workaround I can think of is splitting your workflow and moving the API call that might be the issue into it’s own workflow and using a $.send.http to send the request asynchronously

8

This workflow was previously on Zapier using the webhook action. We never received any timeout issues. I will do some testing outside of Zapier and Pipedream to see if we encounter any issues with response time.

We’re on the advanced plan and have tried increasing the timeout duration on other workflows but I’m not sure if that’s really helping anything since we do still see timeout issues even with 60 or 120 second timeouts.

Thank you for the information about $.send.http. I’ll look into that. Thanks again!

9

Sure happy to help.

That long of a timeout on an axios call is suspicious. I would make sure that one step isn’t locking up async control too.

Just curious - is this issue happening in just the workflow builder or in production too?

10

The issue is actually pretty rare overall, it never shows up when testing in the workflow builder, only in production. Overall it is rare enough to not be a huge issue, and typically replaying the run does work.

11

That is strange. Maybe perhaps logging timestamps throughout the steps might help identify where the timeout is. Best of luck!

12

Also you should be able to use the retry feature in the workflow settings to retry these failed cases automatically.

13

Thanks again, I appreciate all your help!