What Causes the TimeoutError in Puppeteer When Trying to Capture a Page?

This topic was automatically generated from Slack. You can find the original thread here.

Getting the following error in Puppeteer while trying to capture a page

Erro:     at Timeout.null (file:///tmp/__pdg__/dist/code/a8445dc059c303e0027d027740ba2e74491e6deca8d346c91f2b1a3e6f78ed07/node_modules/.pnpm/puppeteer-core@19.8.0/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:60:32)
    at null.listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)
 {
  name: 'TimeoutError'
}

I’m sorry you’re experiencing this issue. The error you’re encountering is a TimeoutError, which typically occurs when a Puppeteer operation takes longer than the allowed timeout. To help you further, I’ll need more information about your code and the specific operation causing the timeout. Can you please provide the relevant code snippet or the step in your Pipedream workflow where you’re using Puppeteer?

The code I’m receiving the previously mentioned error is this:

import { puppeteer } from '@pipedream/browsers';
import fs from "fs";

export default defineComponent({
  key: "puppeteer-screenshot",
  name: "Take a Screenshot",
  version: "0.1.8",
  type: "action",
  props: {
    puppeteer: {
      type: "app",
      app: "puppeteer",
    },
    url: {
      type: "string",
      label: "URL",
      description: "Enter the URL you'd like to take a screenshot of here",
      default: "{{steps.data_processing_discord.URL}}",
    },
    filename: {
        type: "string",
        label: "Name File",
        description: "Enter the filename",
        default: "{{steps.data_processing_discord[\"Source+Symbol\"]}}-{{steps.data_processing_discord.Data}}",
      },
    data: { type: "data_store" }

  },
    async run({ steps, $ }) {
    const { filename, url } = this;
    try {
      const browser = await puppeteer.browser();
      const page = await browser.newPage();
      await page.setViewport({ width: 1920, height: 1080 });

      const sessionidValue = await this.data.get("sessionid");
      const cachecValue = await this.data.get("cachec");
      const cookieSessionID = {
        name: 'sessionid',
        value: sessionidValue,
        domain: '.[tradingview.com](http://tradingview.com)',
        path: '/',
      };
      const cookieCachec = {
        name: 'cachec',
        value: cachecValue,
        domain: '.[tradingview.com](http://tradingview.com)',
        path: '/',
      };

      await page.setCookie(cookieSessionID, cookieCachec);
      await page.setDefaultNavigationTimeout(0);
      await page.goto(url, { waitUntil: 'networkidle2' });

      const elementChart = 'body > div.js-rootresizer__contents.layout-with-border-radius > div.layout__area--center';
      await page.waitForSelector(elementChart, { timeout: 100000 });

      const elementWidgetBar = 'body > div.js-rootresizer__contents.layout-with-border-radius > div.layout__area--right > div > div.widgetbar-tabs > div > div > div > div > button.button-I_wb5FjE.isActive-I_wb5FjE.apply-common-tooltip.common-tooltip-vertical.accessible-I_wb5FjE';
      await page.waitForSelector(elementWidgetBar, { timeout: 5000 });
      await page.click(elementWidgetBar);

      console.log('Executando screenshot.');
      const screenshot = await page.screenshot({
        clip: { x: 56, y: 42, width: 1810, height: 995 },
        timeout: 100000
      });
      console.log('Screenshot executado.');

      await browser.close();
      console.log('Browser fechado.');

      const fn = `${filename.replace(/[&\/\\#, +()$~%.'"":*?<>{}]/g, "-")}.png`;
      fs.writeFileSync('/tmp/' + fn, screenshot);

      $.export("filename", fn);
      $.export("PathFL", "/tmp/" + fn);
    } catch (error) {
      console.error('Erro:', error);
    }
  },
});

I see that you’re using Puppeteer to take a screenshot of a webpage. The TimeoutError you’re encountering might be caused by the page.waitForSelector calls with specific timeouts. I suggest increasing the timeout values or removing them to use the default values. Also, the CSS selectors you’re using seem to have HTML entities like > instead of >. Please update the selectors with the correct characters. Here’s the modified code with updated selectors and removed timeout values: ```
import { puppeteer } from ‘@pipedream/browsers’; import fs from “fs”; export default defineComponent({ // …props and other configurations async run({ steps, $ }) { const { filename, url } = this; try { const browser = await puppeteer.browser(); const page = await browser.newPage(); await page.setViewport({ width: 1920, height: 1080 }); // …cookie and navigation setup const elementChart = ‘body > div.js-rootresizer__contents.layout-with-border-radius > div.layout__area–center’; await page.waitForSelector(elementChart); const elementWidgetBar = ‘body > div.js-rootresizer__contents.layout-with-border-radius > div.layout__area–right > div > div.widgetbar-tabs > div > div > div > div > button.button-I_wb5FjE.isActive-I_wb5FjE.apply-common-tooltip.common-tooltip-vertical.accessible-I_wb5FjE’; await page.waitForSelector(elementWidgetBar); await page.click(elementWidgetBar); // …screenshot and browser close } catch (error) { console.error(‘Erro:’, error); } }, });

I updated the code to the one generated above, and I’m still getting the same error.

I apologize for the inconvenience. Let’s try to debug the issue further. Can you please provide more details about the error message you’re encountering? Additionally, sharing the URL you’re trying to take a screenshot of would be helpful in identifying any potential issues with the selectors or the webpage itself.

I’m getting the following error:

and I want to capture the page:

I apologize for the confusion earlier. It seems that the error message and the URL you’re trying to capture were not provided in your message. To help you better, please provide the error message you’re encountering and the URL of the webpage you’d like to take a screenshot of. This will help me identify any potential issues with the selectors or the webpage itself.

Hi Higor, I can see that you’ve manually specified a timeout for the screenshot method, have you checked to make sure that the timeout value you provided is lower than your workflow execution time limit?

I’m getting the following error:

Erro:     at Timeout.null (file:///tmp/__pdg__/dist/code/5413b73ce51d9a849eb5beec6281905da34bf504153b61df7dae99cddb7c7e12/node_modules/.pnpm/puppeteer-core@19.8.0/node_modules/puppeteer-core/lib/esm/puppeteer/common/WaitTask.js:60:32)
    at null.listOnTimeout (node:internal/timers:569:17)
    at process.processTimers (node:internal/timers:512:7)
 {
  name: 'TimeoutError'
}

and I want to capture the page:

I apologize for the issues you’re facing. It seems like the waitForSelector function is timing out while waiting for the elements on the page. Let’s try using waitForTimeout to add a fixed delay before taking the screenshot. This will give the page enough time to load all the elements. Here’s the updated code: ```
import { puppeteer } from ‘@pipedream/browsers’; export default defineComponent({ props: { filename: { type: “string”, label: “Filename for the screenshot”, }, url: { type: “string”, label: “URL to take screenshot of”, }, }, async run({ steps, $ }) { const { filename, url } = this; try { const browser = await puppeteer.browser(); const page = await browser.newPage(); await page.setViewport({ width: 1920, height: 1080 }); // Navigate to the URL await page.goto(url); // Wait for a fixed delay (e.g. 5000ms) to allow the page to load await page.waitForTimeout(5000); // Take the screenshot const screenshot = await page.screenshot({ path: /tmp/${filename}, fullPage: true }); // Close the browser await browser.close(); // Return the screenshot file path return { screenshot_path: /tmp/${filename} }; } catch (error) { console.error(‘Error:’, error); } }, });

Hello Pierce! Yes, I was previously encountering other timeout errors, and by manually adding a timeout, I resolved the issue. However, now I’m once again experiencing this error in a different way without having made any changes.

I see several timeouts in your puppeteer script, it could be any one of those waiting for selectors or network activity that is the issue.

I suggest placing console.log() statements before these actions that have timeouts, the problem has to be one of these causing the issue.

The total number of timeouts in your code needs to be less than your workflow’s total execution overall timeout limit.