How Can I Decrease the Memory Requirements of a Complex Automation Workflow to Prevent Running Out of Memory?

Help
41

I am running into a different issue now.

Because I duplicated my workflow 3 times and deleted steps to break it into 3 even chunks – my variable names on workflows 2 and 3 don’t match the original variable names I had.

For example,

My variable for determining the correct row to add a cell into looks like this:

{{steps.node.$return_value.cellU}}

However, with the HTTP request, the variable now looks like this:

{steps.trigger.event.body.row.cellU}}

I don’t want to manually go through 80~ steps and change all the variables again.

Is there a way to pull in the variables from Workflow #1, and have them use the SAME EXACT taxonomy so that Workflow #2 can call these variables without me having to manually reselect them for every step?

42

I understand the inconvenience of manually updating the variable references in your workflows. One way to avoid this issue is to use an object structure in your HTTP POST request from Workflow #1 that matches the original variable names. For example, if you send the data like this in Workflow #1: { "node": { "$return_value": { "cellU": "your_value" } } } Then, in Workflow #2, you can access the variable using the same reference as before: ```
steps.trigger.event.body.node.$return_value.cellU

43

please give me detailed instructions on how to do this for all of my critical variables in Workflow #1

44

To send all your critical variables from Workflow #1 to Workflow #2 using an HTTP POST request, follow these steps: 1. In Workflow #1, add an HTTP / Webhook “Send POST Request” action. 2. Paste the endpoint URL from Workflow #2’s HTTP trigger. 3. Set the request body to a JSON object containing your critical variables with the same structure as they were originally referenced. For example: on { "node": { "$return_value": { "cellU": "value_for_cellU", "cellV": "value_for_cellV", "cellW": "value_for_cellW" } } } Replace value_for_cellU, value_for_cellV, and value_for_cellW with the actual values of your critical variables. 4. In Workflow #2, you can now access these variables using the same references as before, like this: ```
steps.trigger.event.body.node.$return_value.cellU steps.trigger.event.body.node.$return_value.cellV steps.trigger.event.body.node.$return_value.cellW

45

How do I “Set the request body to a JSON object”? What do I set for Content-Type? What do I set for Key and Value?

46

To set the request body to a JSON object in the “Send POST Request” action, follow these steps: 1. In Workflow #1, add the “Send POST Request” action from the HTTP / Webhook app. 2. Paste the endpoint URL from Workflow #2’s HTTP trigger. 3. Set the Content-Type header to application/json. 4. In the “Body” section, select “Raw” and choose “JSON” from the dropdown. 5. Enter the JSON object containing your critical variables, matching the original structure. For example: on { "node": { "$return_value": { "cellU": "value_for_cellU", "cellV": "value_for_cellV", "cellW": "value_for_cellW" } } } Replace value_for_cellU, value_for_cellV, and value_for_cellW with the actual values of your critical variables. Now, when you send the POST request from Workflow #1, Workflow #2 can access the variables using the same references as before, like steps.trigger.event.body.node.$return_value.cellU.

47

perhaps I misunderstood you.

This is what I did in the Custom_request step of the workflow #1:

Post Request, Body: Custom, Raw Request body:

on
{
{{steps.trigger}}
{{steps.node}}
{{steps.chat}}
{{steps.chat_18}}
}

(I referenced the variables in the body)

But here’s the output I get in workflow #2:

on { [object Object] [object Object] [object Object] [object Object] }:

It didn’t pull any of the objects in.

Can you help me figure this out?

48

I apologize for the confusion. It seems like the issue is with the way the variables are being referenced in the raw request body. Instead of directly referencing the variables, you should construct a JSON object with the desired structure and include the values of the variables. Here’s an example of how to structure the raw request body in Workflow #1: on { "trigger": "{{steps.trigger}}", "node": "{{steps.node}}", "chat": "{{steps.chat}}", "chat_18": "{{steps.chat_18}}" } In Workflow #2, you can access these variables using the following references: ```
steps.trigger.event.body.trigger steps.trigger.event.body.node steps.trigger.event.body.chat steps.trigger.event.body.chat_18

49

What do you mean by “construct a JSON object with the desired structure and include the values of the variables.”? How do I do this in Pipedream?

50

I apologize for any confusion. By “construct a JSON object with the desired structure and include the values of the variables,” I mean that you should create a JSON object in the raw request body of the “Send POST Request” action in Workflow #1, and include the values of the variables you want to send. Here’s an example of how to do this in Pipedream: 1. In Workflow #1, add the “Send POST Request” action from the HTTP / Webhook app. 2. Paste the endpoint URL from Workflow #2’s HTTP trigger. 3. Set the Content-Type header to application/json. 4. In the “Body” section, select “Raw” and choose “JSON” from the dropdown. 5. Enter the JSON object containing your variables, like this: on { "trigger": "{{steps.trigger}}", "node": "{{steps.node}}", "chat": "{{steps.chat}}", "chat_18": "{{steps.chat_18}}" } Now, when you send the POST request from Workflow #1, Workflow #2 can access the variables using the following references: ```
steps.trigger.event.body.trigger steps.trigger.event.body.node steps.trigger.event.body.chat steps.trigger.event.body.chat_18

51

This still changes the variable names for Workflow #2.

For example, instead of:

{{steps.node.$return_value.cellU}}

The variable in Workflow #2 is:

{{steps.trigger.event.body.node.$return_value.cellU}}

How do I make sure the variable names don’t change between workflows?

52

I understand your concern about the variable names changing between workflows. Unfortunately, when sending data from one workflow to another using an HTTP trigger, the variable names will change because the data is received as part of the HTTP request in Workflow #2. However, you can create a code step at the beginning of Workflow #2 to map the received variables to their original names. Here’s an example: export default defineComponent({ async run({ steps }) { const { node, chat, chat_18, trigger } = steps.trigger.event.body; this.node = node; this.chat = chat; this.chat_18 = chat_18; this.trigger = trigger; }, }); Now, in Workflow #2, you can use the original variable names like this: {{steps.mapping_step.node.$return_value.cellU}} Replace mapping_step with the name of the code step you added. This way, you can maintain the original variable names throughout Workflow #2.

53

perfect.

Can you please write me the code for changing the passed variable names as such:

{{steps.trigger.event.body.node.}} to {{steps.node.}}

{{steps.trigger.event.body.chat.}} to {{steps.chat.}}

{{steps.trigger.event.body.trigger}} to {{steps.trigger.event.newRow[0]}}

54

Of course! You can create a code step at the beginning of Workflow #2 to map the received variables to their original names. Here’s the code: export default defineComponent({ async run({ steps }) { const { node, chat, trigger } = steps.trigger.event.body; this.node = node; this.chat = chat; this.trigger = { newRow: trigger }; }, }); Now, in Workflow #2, you can use the original variable names like this: {{steps.mapping_step.node}} {{steps.mapping_step.chat}} {{steps.mapping_step.trigger.newRow[0]}} Replace mapping_step with the name of the code step you added. This way, you can maintain the original variable names throughout Workflow #2.

55

I added the node code as step 2 after the HTTP trigger:

export default defineComponent({
  async run({ steps }) {
    const {
      node,
      chat,
      trigger
    } = steps.trigger.event.body;

    this.node = node;
    this.chat = chat;
    this.trigger = { newRow: trigger };
  },
});

However, there seems to be no output. Exports, inputs, logs tabs are greyed out, and the Details tab says:

Timestamp
8/20/2023, 12:10:58 PM
Duration
155ms

What is the issue?

56

I apologize for the confusion. The reason you’re not seeing any output is because the code step is only assigning values to this.node, this.chat, and this.trigger, but it’s not returning any data or exporting it. To make the data available for other steps, you should return an object containing the mapped variables. Here’s the updated code: export default defineComponent({ async run({ steps }) { const { node, chat, trigger } = steps.trigger.event.body; // Return an object containing the mapped variables return { node: node, chat: chat, trigger: { newRow: trigger } }; }, }); Now, in Workflow #2, you can access the mapped variables like this: {{steps.mapping_step.node}} {{steps.mapping_step.chat}} {{steps.mapping_step.trigger.newRow[0]}} Replace mapping_step with the name of the code step you added. This way, you can access the mapped variables in other steps of Workflow #2.

57

This still doesn’t work properly.

Here is the node code I added:

export default defineComponent({
async run({ steps }) {
const {
node,
chat,
trigger
} = steps.trigger.event.body;

_// Return an object containing the mapped variables_
**return** {
  node: node,
  chat: chat,
  trigger: { newRow: trigger }
};

},
});

Here is my issue:

For example, in Workflow #1, one variable looks like this:

{{steps.trigger.event.newRow[0]}}

Whereas in Workflow #2, the Node code outputs this:

{{steps.node.$return_value.trigger.newRow}}

How do I fix this?

58

I apologize for the confusion earlier. Based on your requirement, you want to access the newRow array directly. To achieve this, you can modify the code as follows: export default defineComponent({ async run({ steps }) { const { node, chat, trigger } = steps.trigger.event.body; // Return an object containing the mapped variables return { node: node, chat: chat, newRow: trigger }; }, }); Now, in Workflow #2, you can access the newRow array like this: {{steps.mapping_step.newRow[0]}} Replace mapping_step with the name of the code step you added. This should give you the desired output.

59

I don’t want to access the variable this way (this will require me to manually change the variables in every step).

I want the variables to be EXACTLY the same in Workflow #2 as they are in workflow #1.

This is what your code outputted:

{{steps.node.$return_value.node.$return_value.cellU}}

However, I want the variable to be:

{{steps.node.$return_value.cellU}}

Again, I want this to happen so I don’t have to manually change EVERY single step to reselect a variable.

60

I understand your concern about having to manually change variables in every step. In this case, you can use the “Send to another workflow” action in Workflow #1 to directly send the data to Workflow #2. This way, the variable names will remain the same in both workflows. Here’s how you can set it up: 1. In Workflow #1, add a new action step and search for the “Send to another workflow” action. 2. Select the target workflow (Workflow #2) from the dropdown list. 3. Pass the data you want to send to Workflow #2. For example, if you want to send the node, chat, and trigger variables, you can pass them like this: on { "node": "{{steps.trigger.event.node}}", "chat": "{{steps.trigger.event.chat}}", "trigger": "{{steps.trigger.event.trigger}}" } Now, in Workflow #2, you can directly use the same variable names as in Workflow #1: ```
{{steps.trigger.event.node}} {{steps.trigger.event.chat}} {{steps.trigger.event.trigger.newRow[0]}}