How to get a file downloaded when clinking a button using Puppeteer?

Help
1

Hi,
I’m using pipedream puppeteer’s option do go to a website, login with username and password and then click on this button:

logo 

When clicking this button in a regular browser an excel file is downloaded to my PC.

I want to use puppeteer to get the file and to return it and send it to a webhook - But I don’t really know how to do it.

I did something similar in the past with such a code:
// Convert the response body to a base64-encoded string
const buffer = await response.arrayBuffer();
const uint8Array = new Uint8Array(buffer);
const base64 = uint8Array.reduce(
(data, byte) => data + String.fromCharCode(byte),
‘’
);
return btoa(base64);
});

// Close the browser
await browser.close();

// Return the base64 representation of the PDF file
return result;

And then in another action in pipedream I used this code in order to return the data to a webhook that initiated the code:

// To return a custom HTTP response, use $.respond() [requires HTTP trigger]
export default defineComponent({
async run({ steps, $ }) {
await $.respond({
status: 200,
headers: {},
body: {data:steps.puppeteer.$return_value}
})
},
})

So now I want to do something similar - but this time get the excel file that being download when clicking the button - I don’t think there is a need to convert it to base64 - but maybe I’m wrong and in order to rerun it to the webhook I have to do it?

Can someone please help?
Regards,
Ram

2

Hi @optimus.nga,

To achieve your goal of downloading an Excel file using Puppeteer and sending it to a webhook, you can follow these steps:

  1. Use Puppeteer to navigate to the website, log in, and click the button to download the file.
  2. Capture the download and save it to the /tmp directory.
  3. Send the downloaded file to the webhook.

Here’s how you can do it:

Step 1: Puppeteer Component to Download the File

import { puppeteer } from '@pipedream/browsers';
import fs from 'fs';
import path from 'path';

export default defineComponent({
  async run({ steps, $ }) {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();

    // Navigate to the login page
    await page.goto('https://example.com/login');
    
    // Perform login
    await page.type('#username', 'your-username');
    await page.type('#password', 'your-password');
    await page.click('#login-button');
    await page.waitForNavigation();

    // Click the button to download the file
    const [download] = await Promise.all([
      page.waitForEvent('download'), // Wait for the download to start
      page.click('#download-button'), // Replace with the actual button selector
    ]);

    // Save the downloaded file to /tmp directory
    const downloadPath = path.join('/tmp', download.suggestedFilename());
    await download.saveAs(downloadPath);

    // Close the browser
    await browser.close();

    // Read the file content
    const fileContent = fs.readFileSync(downloadPath);

    // Return the file content
    return fileContent;
  },
});

Step 2: Send the File to the Webhook

export default defineComponent({
  async run({ steps, $ }) {
    const fileContent = steps.puppeteer.$return_value;

    // Send the file content to the webhook
    await $.send.http({
      method: 'POST',
      url: 'https://webhook.example.com',
      headers: {
        'Content-Type': 'application/octet-stream',
      },
      body: fileContent,
    });

    return { success: true };
  },
});

Explanation

  1. Puppeteer Component:
    • Navigate to the login page and perform the login.
    • Click the button to download the file and wait for

I’m a bot powered by Pipedream and GPT-4. I’m still learning, so please double-check my answers and code! Learn how to build your own.

3

While running this code I got

TypeError

page.waitForEvent is not a function

DETAILS

    at Object.run (file:///tmp/__pdg__/dist/code/b30680f56aec27a5a09baa331e356c88e8869380e4979e8dcd99f360f7bbb03a/component.mjs:26:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at null.executeComponent (/var/task/launch_worker.js:288:22)
    at MessagePort.messageHandler (/var/task/launch_worker.js:788:28)
4

Anyone?