Help with GTMetrix Download PDF Report

rafael · April 29, 2024, 11:14am

Hello!

I’m trying to create a Workflow that I can run a test with GTMetrix and save the reports in my Google Drive but I’m struggling with an HTTP authentication they require for that.

I’ve successfully configured the trigger and actions of GTMetrix run a test and report.

I’ve added an Google Drive action of ‘Upload file’ so I can provide an URL to send this file to my Google Drive.

GTMetrix report provides an URL like this “https://gtmetrix.com/api/2.0/reports/MY_REPORT_ID/resources/report.pdf” so I could use it on Google Drive step. But when we access this URL, it requires an authentication.

I know I could use this URL to authenticate: “https://USERNAME:PASSWORD@gtmetrix.com/api/2.0/reports/******/resources/report.pdf” and this would work.

But how could I modify this field I’ve got in the Pipedream Workflow to use in the action?

I appreciate any suggestion. Thank you.

pi · April 29, 2024, 11:19am

Hi @rafael,

To achieve the task of downloading a GTMetrix report and uploading it to Google Drive with the required authentication, you can use a custom code step in your Pipedream workflow. This step will handle the authentication to GTMetrix, download the report, and then upload it to Google Drive. Here’s how you can do it:

Download the GTMetrix report using HTTP authentication: You’ll first need to download the report from GTMetrix using the provided URL and your GTMetrix API key for authentication.
Upload the downloaded file to Google Drive: Once you have the report, you can then upload it to Google Drive using the Google Drive API.

Here’s an example of how you can structure this custom code step in your workflow:

import { axios } from "@pipedream/platform";
import fs from "fs";
import path from "path";

export default defineComponent({
  props: {
    gtmetrix: {
      type: "app",
      app: "gtmetrix",
    },
    google: {
      type: "app",
      app: "google",
    },
  },
  async run({ steps, $ }) {
    // Define the GTMetrix report URL
    const reportUrl = `https://gtmetrix.com/api/2.0/reports/MY_REPORT_ID/resources/report.pdf`;

    // Download the GTMetrix report
    const reportResponse = await axios($, {
      url: reportUrl,
      method: "GET",
      responseType: "arraybuffer",
      auth: {
        username: `${this.gtmetrix.$auth.api_key}`,
        password: ``,
      },
    });

    // Save the report to a temporary file
    const tempFilePath = path.join("/tmp", "gtmetrix_report.pdf");
    fs.writeFileSync(tempFilePath, reportResponse, "binary");

    // Upload the report to Google Drive
    const driveResponse = await axios($, {
      method: "POST",
      url: "https://www.googleapis.com/upload/drive/v3/files?uploadType=media",
      headers: {
        Authorization: `Bearer ${this.google.$auth.oauth_access_token}`,
        "Content-Type": "application/pdf",
      },
      params: {
        name: "gtmetrix_report.pdf",
      },
      data: fs.createReadStream(tempFilePath),
    });

    return driveResponse.data;
  },
});

Note: This example assumes you

I’m a bot powered by Pipedream and GPT-4. I’m still learning, so please double-check my answers and code! Learn how to build your own.

rafael · May 24, 2024, 5:50pm

Unfortunately, this isn’t really clear what should I do in Pipedream.

I already have a Workflow with the steps:

Trigger on every monday.
Perform a GTMetrix test on a website
Generate the GTMetrix report from this test.
Stores some values on a Google Spreadsheet.

Now I would like to save the PDF Report from this last report to my Google Drive.

What kind of action in Pipedream I can create to include the custom code to download the report and upload to my Google Drive?

andrew · May 24, 2024, 6:02pm

You can use a Custom Node.js code step and copy Pi’s suggestion:

import { axios } from "@pipedream/platform";
import fs from "fs";
import path from "path";

export default defineComponent({
  props: {
    gtmetrix: {
      type: "app",
      app: "gtmetrix",
    },
    google: {
      type: "app",
      app: "google",
    },
  },
  async run({ steps, $ }) {
    // Define the GTMetrix report URL
    const reportUrl = `https://gtmetrix.com/api/2.0/reports/MY_REPORT_ID/resources/report.pdf`;

    // Download the GTMetrix report
    const reportResponse = await axios($, {
      url: reportUrl,
      method: "GET",
      responseType: "arraybuffer",
      auth: {
        username: `${this.gtmetrix.$auth.api_key}`,
        password: ``,
      },
    });

    // Save the report to a temporary file
    const tempFilePath = path.join("/tmp", "gtmetrix_report.pdf");
    fs.writeFileSync(tempFilePath, reportResponse, "binary");

    // Upload the report to Google Drive
    const driveResponse = await axios($, {
      method: "POST",
      url: "https://www.googleapis.com/upload/drive/v3/files?uploadType=media",
      headers: {
        Authorization: `Bearer ${this.google.$auth.oauth_access_token}`,
        "Content-Type": "application/pdf",
      },
      params: {
        name: "gtmetrix_report.pdf",
      },
      data: fs.createReadStream(tempFilePath),
    });

    return driveResponse.data;
  },
});

Click on Run custom code when creating a new action step:

rafael · May 24, 2024, 6:59pm

All right, thank you Andrew.

Looks like it worked.

I changed this line so it uses my last report generated in a previous Action in my Workflow:

// Define the GTMetrix report URL
    const reportUrl = steps.get_performance_report.$return_value.links.report_pdf;

But I would like to use the Action Google Drive - Upload File to upload the file. I tried to use the path of the file “/tmp/gtmetrix_report.pdf” but I’ve got the error that is no such file.

Maybe I wrote something wrong or the file was lost after some time? I couldn’t find a proper way to use that field.

I appreciate if you have any suggestions. Thank you.

andrew · May 24, 2024, 7:04pm

Oh cool, yeah that’ll make it more dynamic!

I assume you this part of the code in your action step:

    // Save the report to a temporary file
    const tempFilePath = path.join("/tmp", "gtmetrix_report.pdf");
    fs.writeFileSync(tempFilePath, reportResponse, "binary");

Files in the /tmp folder are deleted from time to time. See this docs. Try testing the download step again and then testing the upload_file action right after. In a deployed environment you shouldn’t have any issues.

rafael · August 7, 2024, 11:40am

Hey, I managed to make my workflow run correctly for some weeks.

Here is what I did in the last steps.

My code was this:

import { axios } from “@pipedream/platform”;
import fs from “fs”;
import path from “path”;

export default defineComponent({
props: {
gtmetrix: {
type: “app”,
app: “gtmetrix”,
},
/*
google: {
type: “app”,
app: “google”,
},
*/
},
async run({ steps, $ }) {
// Define the GTMetrix report URL
const reportUrl = steps.get_performance_report.$return_value.links.report_pdf;
// Download the GTMetrix report
const reportResponse = await axios($, {
  url: reportUrl,
  method: "GET",
  responseType: "arraybuffer",
  auth: {
    username: `${this.gtmetrix.$auth.api_key}`,
    password: ``,
  },
});

// Save the report to a temporary file
const tempFilePath = path.join("/tmp", "gtmetrix_report.pdf");
fs.writeFileSync(tempFilePath, reportResponse, "binary");
},
});

And I add an Action of Google Drive to upload the temp file with this configuration:

Google Drive Account:
My account

Drive:

Folder:
My folder in my Google Drive

File URL:

File Path:
/tmp/gtmetrix_report.pdf

Name:
GTMetrix Report {{steps.trigger.event.timezone_configured.date.year}} {{steps.trigger.event.timezone_configured.date.month}} {{steps.trigger.event.timezone_configured.date.day}} - {{steps.trigger.event.timezone_configured.time.hour}}{{steps.trigger.event.timezone_configured.time.minute}}{{steps.trigger.event.timezone_configured.time.second}}.pdf

But now I’ve got an error that I couldn’t figure out what is the problem. My workflow runs correctly and generate the report but is failing in these final steps to save the PDF.

I’ve got this error:

AxiosError - Request failed with status code 503

{“type”:“Buffer”,“data”:[123,34,101,114,114,111,114,115,34,58,91,123,34,116,105,116,108,101,34,58,34,80,68,70,32,83,101,114,118,105,99,101,32,117,110,97,118,97,105,108,97,98,108,101,34,44,34,99,111,100,101,34,58,34,69,52,53,48,51,48,51,34,44,34,115,116,97,116,117,115,34,58,34,53,48,51,34,44,34,100,101,116,97,105,108,34,58,34,80,68,70,32,83,101,114,118,105,99,101,32,116,101,109,112,111,114,97,114,105,108,121,32,117,110,97,118,97,105,108,97,98,108,101,46,34,125,93,125,10]}

Details
    at null.settle (/pipedream/dist/code/2090592c85afb4c34cc16c9080bd8bc724c73a261b65c9aba0742d9ec93abefc/node_modules/.pnpm/axios@1.7.2/node_modules/axios/dist/node/axios.cjs:1983:12)
    at IncomingMessage.handleStreamEnd (/pipedream/dist/code/2090592c85afb4c34cc16c9080bd8bc724c73a261b65c9aba0742d9ec93abefc/node_modules/.pnpm/axios@1.7.2/node_modules/axios/dist/node/axios.cjs:3085:11)
    at IncomingMessage.emit (node:events:531:35)
    at null.endReadableNT (node:internal/streams/readable:1696:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at null.callAxios (/pipedream/dist/code/2090592c85afb4c34cc16c9080bd8bc724c73a261b65c9aba0742d9ec93abefc/node_modules/.pnpm/@pipedream+platform@3.0.0/node_modules/@pipedream/platform/dist/axios.js:110:26)
    at Object.run (file:///pipedream/dist/code/2090592c85afb4c34cc16c9080bd8bc724c73a261b65c9aba0742d9ec93abefc/component.mjs:23:28)
    at null.executeComponent (/var/task/launch_worker.js:292:22)
    at MessagePort.messageHandler (/var/task/launch_worker.js:792:28)

I appreciate any idea to review what could be this error.

Thank you.