How do I iterate through an array of results from a step?

This topic was automatically generated from Slack. You can find the original thread here.

Frank : Hello - can someone point me into the right direction on how I can iterate through an array of results?

For example, I run a query on Airtable and get a response with [2] records. I now want the next step in my workflow to iterate through both those results. How do I do this?

image.png

Dylan Sather (Pipedream) : Hi , to do this right now, you’ll need to write just a bit of Node.js code. Here’s a good reference on loops.

Within the step where you want to iterate, you’ll need to modify the code of that step - just click into the step and start typing. At the top of the step, you’ll need to iterate over the array of results (steps.airtable_get_tasksOutstanding.$return_value.records), running the original code of the step for each element of the array:

for (const record of steps.airtable_get_tasksOutstanding.$return_value.records) {
  // original code here
}

It looks like each record is an object, so you can reference record.fields to grab the fields for each item in the array, for example.

We’re tracking the ability to loop through steps natively in the Pipedream UI, if you’d like to follow that issue for updates.

Let me know if this helps or if you have any other questions.

Frank : Ok thanks will give this a try :slightly_smiling_face:

@pierce any development on Pipedream iterator? Today I checked Helper Functions but haven’t found an iterator helper.
I tested the suggested code like:

for (const record of steps.advanced_search.$return_value.records) {
  var tempArray = steps.advanced_search.$return_value[record].entities;
  ...
}

but I got back error:

TypeError, steps.advanced_search.$return_value.records is not iterable

despite it is an array of 100 objects.
Pls note
steps.advanced_search.$return_value
doesn’t have a visible “records” property ( it might be a hidden system property though )

If I remove “records” property from code like:

for (const record of steps.advanced_search.$return_value) {
  var tempArray = steps.advanced_search.$return_value[record].entities;
  ...
}

I got error message of:

TypeError, Cannot read property ‘entities’ of undefined

I don’t want to bump or OFF the OP, but my issue is just the same as OP:
“How do I iterate through an array of results from a step?”

and as I see there’s still no solution in this.

Hi @vanetreg

That’s correct, unfortunately we don’t have a built in array iterator at this time, and that Github Issue you linked to is the best place to subscribe to future updates.

Improving control flow is a top item on our roadmap and we know this would be super impactful to making workflows even easier to work with and improve productivity.

That said, the approach you’re taking now is the correct one.

The only issue is that you’ll need to tweak the pathing to your step exports a bit.

Can you share a screenshot of the advanced_search exports result? That would help us understand the structure of your data and help guide you on how to iterate it.

1 Like

Hi @pierce

I attach the screenshot you asked.

Pls. note screenshot doesn’t show the whole structure properly (buttons are hiding info), so here I copy the path of the 0th object of the array:

steps.advanced_search.$return_value[0]

The next Node codestep, which drops error, contains this code:

export default defineComponent({
  async run({ steps, $ }) {
    // creating a Loop
    for (const record of steps.advanced_search.$return_value) {
      var tempArray = steps.advanced_search.$return_value[record].entities;
      if (typeof tempArray.hashtags !== 'undefined' && tempArray.hashtags.length > 0) {
        var hashtagsString = tempArray.hashtags.map(hashtag => hashtag.text).join(',');
        // console.log("Hashtags: " + hashtagsString);
      } else {
        var hashtagsString = "";
      };
      if (typeof tempArray.symbols !== 'undefined' && tempArray.symbols.length > 0) {
        var symbolsString = tempArray.symbols.map(symb => symb.text).join(',');
        // console.log("Symbols: " + symbolsString);
      } else {
        var symbolsString = "";
      };
      if (typeof tempArray.user_mentions !== 'undefined' && tempArray.user_mentions.length > 0) {
        var userMentionsString = tempArray.user_mentions.map(item => item.screen_name).join(',')
        // console.log("User mentions: " + userMentionsString);
      } else {
        var userMentionsString = "";
      };
      if (typeof tempArray.urls !== 'undefined' && tempArray.urls.length > 0) {
        var urlsString = tempArray.urls.map(url => url.expanded_url).join(',');
        // console.log("URLs: "+  urlsString);
      } else {
        var urlsString = "";
      };
      if (typeof tempArray.media !== 'undefined' && tempArray.media.length > 0) {
        var mediaString = tempArray.media.map(med => med.expanded_url).join(',');
        // console.log("Media: "+ mediaString);
      } else {
        var mediaString = "";
      };

      $.export('hashtagsString', hashtagsString);
      $.export('hashtagsString', hashtagsString);
      $.export('symbolsString', symbolsString);
      $.export('userMentionsString', userMentionsString);
      $.export('urlsString', urlsString);
      $.export('mediaString', mediaString);
    }
  }, // END async run function
}) // END Component

Thank you!

Hi @vanetreg

Thanks for the whole context, very helpful. I didn’t realize this was a Twitter - Advanced Search action.

You have the step export pathing correct, but the issue is how you’re accessing the individual record within the loop.

You defined the record variable in your for loop definition:

CleanShot 2022-07-13 at 09.38.29

You don’t need to access the value of the record in a temp array, you can just reference it directly:

Here’s a very simple code sample:


// To use previous step data, pass the `steps` object to the run() function
export default defineComponent({
  async run({ steps, $ }) {
    for(const record of steps.advanced_search.$return_value) {
      console.log(record.full_text);
    }
  },
})

I hope this helps!

1 Like

@pierce

OK, thanks a lot! :slight_smile:

1 Like

Sure thing @vanetreg happy to help :slight_smile:

1 Like

Hey everbody,
I am also waiting for pipedream to add loops/iterations features.
But I think the solution offered here is a very nice workaround:
https://github.com/PipedreamHQ/pipedream/issues/193#issuecomment-1203934676

1 Like

Hello. What’s the latest on this? How does Pipedream suggest that we iterate through a step’s array to do X, such as add rows of that data to another tool? Thanks.

I’m considering using Pipedream instead of Zapier for a project, and this lack of loops and conditional logic is a considerable showstopper.
I’ve seen the discussion in the Git issue where it’s suggested to use additional workflows, but this exponentially increases the credit consumption and totally undermines the pricing model of it being time based as opposed to task based.
Sure, can code loops in node, but then what’s the point of the whole framework if subsequent calls based upon the loop data have to be in code instead of the UI. I may as well just code the whole thing.
What am I missing here?

1 Like