How do I fix a 403 error when downloading a file using got?

This topic was automatically generated from Slack. You can find the original thread here.

Helio Freitas : I’m trying to download (to /tmp) a json from an endpoint that requires authentication. But I’m getting “HTTPErrorResponse code 403 (Unauthorized)” message when submitting the get. When I do the same thing via a get using Axios I get it. How solve this?

DOWNLOAD URL
https://login:password@mydomain.com/rest/accomplishedVisit?currentPage=0&pageSize=500&startDate=2021-08-01&endDate=2021-08-12&zfVisitTypeIds=5107,9774&zfVisitResultIds=4793

STEP CODE
const fs = require(“fs”);
const { default: got } = await import(“got”);
const stream = require(“stream”);
const { promisify } = require(“util”);

const pipeline = promisify(stream.pipeline);
await pipeline(
got.stream(params.downloadURL),
fs.createWriteStream(params.filePath)
);

Dylan Sather (Pipedream) : I’d recommend looking at the got docs on GitHub. Technically passing in the URL with the user, password, and query string params should work, but you can also pass those as the username and password in the options you pass to got.

Helio Freitas : Thanks. :slightly_smiling_face:

Helio Freitas : I am getting 403 not 401 from the server.

Dylan Sather (Pipedream) : Since each server implements how they return 403s differently, I’d recommend reaching out to the target service to ask why you might be getting that.

Alternatively, you can keep using axios and download the file to /tmp using a technique like this: Axios — Download Files & Images in Node.js

Helio Freitas : No sucess again. Today is not my day

Helio Freitas : ‘use strict’

const Fs = require(‘fs’)
const Path = require(‘path’)
const Axios = require(‘axios’)

async function downloadImage () {
const url = ‘https://unsplash.com/photos/AaEQmoufHLk/download?force=true
const path = Path.resolve(__dirname, ‘tmp’, ‘code.jpg’)
const writer = Fs.createWriteStream(path)

const response = await Axios({
url,
method: ‘GET’,
responseType: ‘stream’
})

response.data.pipe(writer)

return new Promise((resolve, reject) => {
writer.on(‘finish’, resolve)
writer.on(‘error’, reject)
})
}

downloadImage()

Helio Freitas : Internal error from Pipedream

Dylan Sather (Pipedream) : remove this line:

const path = Path.resolve(__dirname, 'tmp', 'code.jpg')

and modify this line to read:

const writer = Fs.createWriteStream('/tmp/code.jpg')

Helio Freitas : No more internal error but the file was not saved

Dylan Sather (Pipedream) : await downloadImage() at the end

Helio Freitas : Works. Thansk

Helio Freitas : I have talked to the guys of target service. According they, the rest api implementation is different from stream. So its not compatible try download like I would like

Helio Freitas : ```
I thought it was just a Nginx setting to allow or not stream