Why I am getting null?

[slishnevsky]

Hello.

I am building a workflow, RSS aggregator for Twitter that posts to Telegram.
I am using Telegram bot to post feeds items (Twitter posts from multiple channels) to Telegram.
Furthermore, I created multiple RSS triggers for each channel.
Since Twitter does not provide RSS feeds, I am using Google News that can create RSS for Telegram channels.
Google News generates an HTML response that contains JavaScript code that contains a direct link to the Twitter post. When launched in browser, it simply redirects to a Twitter post.
In the pipedream workflow, I catch these HTML responses as text and then, use Node.js code to extract those link strings.
This Node.js code collects all such Twitter links from all feeds in an array.
The result should be an array that contains HTTP URLs that I can use to post to Telegram that would be direct links to Twitter posts collected in all the RSS triggers.

Here is the code:

// To use any npm package, just import it
// import axios from "axios"

export default defineComponent({
  async run({ steps, $ }) {
    // Convert the current text to lowercase
    const text = steps.custom_request.$return_value;
    const words = ['ChanelRion', 'ACTBrigitte', 'OANN', 'AnIllarionov'];
    // Regular expression to match a string in double quotes
    const regex = /"([^"]*?)/g;
    let match;
    let results;
    let lowerCaseMatch;

    // Iterate through all matches
    while ((match = regex.exec(text)) !== null) {
      // Convert the current match to lowercase
      lowerCaseMatch = match[0].toLowerCase();
      // Iterate through each word in the array
      for (const word of words) {
        // Convert the current word to lowercase
        const lowerCaseWord = word.toLowerCase();
        // Check if the match contains the current word
        if (lowerCaseMatch.includes(lowerCaseWord)) {
          results.push(lowerCaseMatch);
        }
      }
    }
    // Return the matched string
    // return results;
    $.export("results", results);
  },
})

And here is the workflow:

My question:

Why do I get this warning message (you can see it on the screenshot)?

Warning

• UNSERIALIZABLE_EXPORT: this.results could not be serialized.

And it returns null, but should return something, since I am scanning returned HTML responses for words that are actual usernames from RSS feeds that have been generated for these users.

I know it’s a bit complicated, but hope someone could actually assist.
Here I am sharing with you my workflow, so that you can run it in your environment.
I don’t care about posting to Telegram at this moment, so you can ignore the last action in the workflow.

Sign Up - Pipedream

Thank you.

[pi]

Hi @slishnevsky,

The warning message UNSERIALIZABLE_EXPORT: this.results could not be serialized. and the fact that it returns null likely occur because the results variable is not initialized before you start pushing matches into it. In JavaScript, before you can push items into an array, you need to initialize the array. Otherwise, when you attempt to push an item into an undefined variable, it will not work, leading to the inability to serialize the results variable for export.

Here’s a corrected version of your code with the results array initialized:

export default defineComponent({
  async run({ steps, $ }) {
    // Convert the current text to lowercase
    const text = steps.custom_request.$return_value;
    const words = ['ChanelRion', 'ACTBrigitte', 'OANN', 'AnIllarionov'];
    // Regular expression to match a string in double quotes
    const regex = /"([^"]*?)/g;
    let match;
    let results = []; // Initialize the results array
    let lowerCaseMatch;

    // Iterate through all matches
    while ((match = regex.exec(text)) !== null) {
      // Convert the current match to lowercase
      lowerCaseMatch = match[0].toLowerCase();
      // Iterate through each word in the array
      for (const word of words) {
        // Convert the current word to lowercase
        const lowerCaseWord = word.toLowerCase();
        // Check if the match contains the current word
        if (lowerCaseMatch.includes(lowerCaseWord)) {
          results.push(lowerCaseMatch);
        }
      }
    }
    // Return the matched string
    // return results;
    $.export("results", results);
  },
})

In this corrected version, results is initialized as an empty array before it’s used in the loop. This should resolve the warning and allow the results array to be serialized and exported properly.

I’m a bot powered by Pipedream and GPT-4. I’m still learning, so please double-check my answers and code! Learn how to build your own.

[slishnevsky]

Thanks!
Yes, I totally forgot.
Now the warning message is gone.

Now I am getting the array back, but it’s empty.

Is there anything I am missing in my code?

I understand that each trigger sends the response according to the interval defined.

So, basically it is one by one, separately.

For simplicity’s sake, it could be just one RSS trigger for any channel.

I have 4 triggers for 4 channels: [‘ChanelRion’, ‘ACTBrigitte’, ‘OANN’, ‘AnIllarionov’]

So, then the response comes back from a trigger to the code, it must contain that javascript with a link to a user’s channel, so it must contain one of these usernames.

That’s what I am checking in my code.
So, using Regex, I am grabbing all strings from the returned HTML response.
I get an array of all possible strings.
And I have an array of all my usernames.
So by looping through those strings, I find the one that contains one of these usernames.
And that is the link that I need, so I adding it to the array of links I want to return.
Then, I return this array to the next step.

I am not getting why the returned array “results” is empty, it should contain something.

P.S. Maybe I am misunderstanding the process, and should not use array at all, since, my assumption, that there is a separate process for each post of every feed.

So, if multiple users on Twitter created several posts, when the workflow triggers are triggered, they return a separate HTML response for each post.

For example, here is the RSS for Andrei Illarionov’s Twitter account (https://twitter.com/AnIllarionov):
"site:twitter.com/AnIllarionov" - Google News

There are lots of items in it.
When my custom_request send an HTTP request to: "site:twitter.com/AnIllarionov" - Google News

it returns something like this:

where you can see a string containing his username (AnIllarionov).

That is the string I am grabbing. It is a direct link to his post.
One post, despite that RSS returns a bunch of items.
Looks like Google News returns only one most recent item.

So the Node.js code should grab that link, but the array returned is empty.
I am definitely missing something here.