Hoping anyone could shed some insight on how they may of used Pipedream to implement bi-directional support and data reconciliation of records between 2 different systems / platforms?
Is it a case of implementing 2 workflows treating either platform as the source & destination within each workflow and then have it work off the same data store record?
High-level scenarios are as follows;
Record A exits in both System A and B. Record A is manipulated or updated via a mobile app which sends it’s changes up to System A. Pipedream Workflow 1 see’s that Record A has been updated, kicks off workflow and either creates or updates the corresponding Record A in System B.
Record B is created in System B, saved and Pipedream Workflow 2 picks up the changes and pushes it to System A where Record A is created.
Any thoughts or feedback on this matter would be much appreciated?
Thanks
Rory
First off, welcome to the Pipedream community. Happy to have you!
I’m not sure what System A & B are exactly, but I think your approach is sound.
You can use a Data Store to record the state between transfers, otherwise you may run into issues of an infinite recursion when a new record in System B causes a new record in System A and vice versa.
The Data Store needs to be the source of truth to make sure that the same record isn’t added over and over again.
PS - regarding tags, they’re only for specific apps. We use a workflow to sync these issues to our app marketplace pages.
To add some context to my scenario(s). System A would be the product / platform that I work on which is a SaaS company in the Field Operations & Management space.
System B would be like your classic ERP’s / CRM’s like NetSuite or Salesforce.
The Data Store needs to be the source of truth to make sure that the same record isn’t added over and over again.
So would I be right in thinking then that the data store record or instance created when written to by Workflow 1 performing a Sync from System A->B should not be referenced or written over by Workflow 2 performing a Sync from System B-> A?
Instead Workflow 2 should create new record of the same data store instance and have the data captured in the sync or writeback from System B-> A?
Anyhows that’s my brain dump for today, thanks for feedback and insights.
Unfortunately, I have to refer to the classic engineering response of “Well, it depends”.
If you need to only insert new records across systems, that’s a much easier operation.
You can have two workflows that are responsible for listening to only new records in their respective systems, then your data store or database could look something like this:
app_inserted
crm_inserted
app_record_id
crm_record_id
1
0
user_1
null
This record signals that the CRM has yet to be updated with the user’s record from the app.
If you need records to also update in addition to creation, think an upsert operation. Then you can use the updated_at timestamp on each record as the source of truth to keep them in sync:
app_updated
crm_updated
app_record_id
crm_record_id
2022-11-28
2022-10-01
user_3
contact_14
Just a suggestion, the above is a table whereas a Data Store is a simple key/value pair.
