What is the Reason Behind Trello and Google Sheets Not Experiencing Infinite Looping When Set Up for Bidirectional Syncing?

1

This topic was automatically generated from Slack. You can find the original thread here.

Hey all! So I setup a bidirectional sync between Google Sheets + Trello (It’s basically two workflows), like in the attached diagrams.

Originally I thought that there may be some issue with “infinte looping” when one update is initiated that it would trigger the other workflow, and this would continue on and on. However, it doesn’t seem to be the case. Is there a reason this is? I’m assuming the Google Sheets on Update (Instant) component hooks into the onEdit event, so does this only respond to “human” users interacting with a given sheet? A little puzzled how this works by default:

2

Hello , thanks for the cool diagram.

Why do you want it to be an infinite loop anyway? Assuming the infinite loop is the correct behavior, then which trigger is breaking the loop? Is it the Trello trigger or Google Sheet trigger not emitting expected events on your side?

3

I absolutely don’t want there to be an infinite loop. All the events are looking good and there are no “extra” events, which surprised me considering I didn’t add a break trigger or check anywhere in the the two workflows.

I thought I would need to create some kind of break trigger that would either reference a modified date or check and see if it’s an API or a human creating the modification and break accordingly, but doesn’t seem to be the case.

That’s why I was curious if, by default, either the Trello or Sheets Instant Pipedream integrations do some kind of check like this before activating the rest of the workflow. Not sure if that makes more sense or not…

4

Hi , another thread was started about this kind of syncing architecture:

I left a comment on how to design a database that can prevent the infinite recursion.

But to answer your last question, there is event deduplication within each custom source that powers your triggers. You can investigate and even modify the underlying source code if you need to.

Here’s a link to the Google Sheets sources on Github: pipedream/components/google_sheets/sources at master · PipedreamHQ/pipedream · GitHub

And Trello’s:

1 Like
5

this is really cool stuff, thanks for sharing! Looked through the source code a bit and this actually answered my question why Google Sheets updates were coming in as “batches” in my workflow.

So if I’m looking at this correctly, I can see there is a getMeta method inside of processSpreadsheet that is part of an emit event up to some parent, but I’m not sure exactly where the parent is or how that handles the dedup process from there.

6

I see there’s this parent object here with the dedupe value “unique”, but still not sure what/how this is working. Just for my own learning, no need to rush replying :slightly_smiling_face:

7

Hi you really went for a deep dive :slightly_smiling_face: super cool.

Yes you’ve found the built in deduping strategy definition, here’s the full explaination: Component API Reference

It just picks a strategy for this source to use when deciding if the event is new by the event’s id defined by the this.$emit function: Component API Reference

8

yep, totally makes sense. So essentially each row becomes a unique hashed ID based on content.

I guess my only question then (this might be a syntax thing I’m not familiar with, I’m more of a Python guy) is why the event name is reversed in new-row-added.mjs, cause in the documentation you sent above :point_up_2: looks like this

this.$emit(event, {
  id,
  name,
  summary,
  ts,
});
9

But in the new-row-added

this.$emit(
            {
              newRow,
              range,
              worksheet,
              rowNumber,
            },
            this.getMeta(spreadsheet, worksheet, rowNumber, newRow),
          );

So not sure if this is something you can reverse in JS or something

10

Hi Doug, the format looks a bit different but both methods return two arguments that are objects.

The event can be an object (aka dict in Python speak), and the metadata is also an object containing the keys id, name , summary and a timestamp (ts).

The getMeta returns this object and passes it as a parameter to this.$emit

11

If you haven’t read this yet, I highly recommend reading this free book to learn Javascript:

https://eloquentjavascript.net/

I used it myself to learn a synchronous server side language like Python, it’s a great introduction to the syntax and how asynchronous functions work in Node/Javascript