How do I download a PDF from a URL to attach to a mailgun email?

Help
, ,
1

This topic was automatically generated from Slack. You can find the original thread here.

Theo Lemay : Hey, I’m struggling to get this to work. I’m pretty new at node but I’m not sure why this keeps happening. In terms of XY question, I’m trying to download a PDF from a url to attach to a mailgun email. All the info should be in the image, I’ll also include the code here:

const fetch = require('node-fetch');
const fs = require('fs').promises;

const response = await fetch(params.pdf)
await fs.writeFile("/tmp/file.pdf", response.body);

Thanks in advance!

image.png
image.png1476×735 81.3 KB

2

Dylan Sather (Pipedream) : could you try using writeFileSync instead and see if that works? https://docs.pipedream.com/workflows/steps/code/nodejs/working-with-files/#writing-a-file-to-tmp

3

Theo Lemay : thanks for the reply, the following code still gives the same error:

const fetch = require('node-fetch');
const fs = require('fs');

const response = await fetch(params.pdf)
await fs.writeFileSync("/tmp/file.pdf", response.body);
4

Theo Lemay : I’ve also tried with axios and other fetch libraries, and still have that error

5

Theo Lemay : Scratch that, I seem to be having luck with

const axios = require('axios');
const fs = require('fs');

const response = await axios.get(params.pdf)
await fs.writeFileSync("/tmp/file.pdf", response.data);
6

Dylan Sather (Pipedream) : awesome. You should also be able to remove the await in front of fs.writeFileSync . Let me know if you’re seeing any other issues

7

Theo Lemay : humm, so the only problem is that it seems like the pdf file is corrupted.

const axios = require('axios');
const fs = require('fs');

const url = params.pdf;

const response = await axios({
    url,
    method: 'GET',
    responseType: 'stream'
  })

await fs.writeFileSync("/tmp/file.pdf", response.data);

Seems to have the same “step was still trying to run” error

8

Dylan Sather (Pipedream) : can you try downloading it without the responseType: 'stream' property? How large is the PDF?

9

Theo Lemay : Sure, and it’s about 250kb

10

Dylan Sather (Pipedream) : Thanks. Streaming the contents of a URL directly to a file can be useful in situations where you have very large files that exceed the available memory of your workflow (roughly 200MB), but should be unnecessary here. The reason the file looks corrupted is because axios returns a ReadableStream object when you specify responseType: 'stream', so you’d need to handle the response a bit differently. Removing that property and writing the data directly to the file should work.

11

Dylan Sather (Pipedream) : for a PDF you may need to add:

responseType: 'blob',

though

12

Dylan Sather (Pipedream) : sorry, that’s a browser-only option

13

Dylan Sather (Pipedream) : responseType: 'arraybuffer'

1 Like
14

Dylan Sather (Pipedream) : try that, instead

15

Dylan Sather (Pipedream) : that’s the option axios uses to download binary content

16

Theo Lemay : Yes, thank you!

const axios = require('axios');
const fs = require('fs');

const url = params.pdf;
const response = await axios({
    url,
    method: 'GET',
    responseType: 'arraybuffer'
  })

await fs.writeFileSync("/tmp/file.pdf", response.data);

Works great!

1 Like
17

Dylan Sather (Pipedream) : :fire:

18

Theo Lemay : As an aside, how could I get the PDF name metadata/headers to use as the filename?

image.png
image.png918×248 40.7 KB

19

Theo Lemay : If it’s something along the lines of reading the headers of the response, I might be able to figure it out.

20

Dylan Sather (Pipedream) : yeah if this hosting service provides that metadata in the headers, that should be available in response.headers for you to parse. I’m not familiar with the PDF format in enough detail to know whether that would be communicated in other metadata of the file itself, but if so you may also be able to parse the file to retrieve that