What is causing the API response to be exported as an undefined string?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Not sure if this is a PD issue or JS (my ignorance of) issue. A third-party API I use has started returning garbled responses instead on a nice JSON object. (Specifically it is weirdly pre-prending the body of the request it receives to the result it provides!).

The garbled result is being exported to the following step as an string, but is type undefined. In the subsequent step I can sanitise the result using string operations, but cannot turn the sanitised string back into an object because the typeof the string is undefined.

2

If it’s a string, you could do

return JSON.parse(string)

replace string with the variable of your JSON string

3

Yeah, you’d think so… but in this case JSON.parse also returns a result with typeof undefined

4

Could you tried setting an explicit Accept-Encoding header? It might be returning a different content type without it:

const { data } = await axios.get(url, {
  headers: {
    'Accept-Encoding': 'application/json'
  }
})
5

I can’t readily try that since I’m using the HTTP request action to generate the POST request to the API which returns the result, and I’m not sure how you specify Accept-Encoding on that. (and Postman tells me the encoding is application/json)

But in fact I don’t think that’s the problem, I think it’s to deal with the string passed to the next step. When the API works it returns and object { … }, but when it doesn’t, it returns to concatenated objects {…}{…} (!)

I figured I’d just be able to slice off the first { … } and then JSON.parse the rest back into the object I expected. However, my fancy regex replace doesn’t work on the {…}{…} string at all unless - and you’ll love this - I JSON.stringify it first. However, other string operations work ok so I’ll slice it that way.

Then all I have to work out is how to get JSON.parse to recognise {…} or even ‘{…}’, both of which it refuses.

TL;DR Looks like a JS (ignorance of), rather than a PD issue

6

Just as a follow up… how does the PD HTTP request action handle ill-formed responses in terms of passing them on, e.g. specifically, a response ostensibly encoded application/json, but isn’t. What does the result look like downstream in PD?

7

PD will attempt to serialize the response into JSON.

What format is the API returning?

8

Thanks . The API purports to return a JSON object (i.e. { … } using application/json, but is erroneously sending two such objects concatenated (i.e. {…}{…}) so it is probably being passed to the following step as a string. I’ve managed to successfully munge it now, but it took lots of trial and error. Fixing the API would have been easier!! (assuming I could access it)

9

Yikes, yup invalid JSON would cause the serialization to break. Glad you were able to figure that out! Sounds like the service needs a ticket opened about that

10

Indeed , but this is one of those (handy) zombie services which appears to be no longer supported (but still seems to work!)

I guess a broader point is that it might be prudent for PD to able to signal/catch what could be a problem that might pop up from time to time ?

11

Interesting, sounds like an undocumented deprecated API. Those are the fun kind.

You can definitely subscribe to errors and customize how to handle them.

This short video shows how to pipe them into Cloudwatch, but the same technique can be used to send your errors into whichever notification system you prefer: