Following is my analysis when choosing a tool to use to integrate data from various systems with data in our CRM system.
Notable Criteria
When developing dependable integrations you will likely want to hand-craft HTTP requests, as HTTP requests are what you see when reading the API provider’s documentation. For example, the Salesforce REST API has facilities for creating a single record, but it also provides facilities for sending one HTTP request to create many records, called a "composite resource". The API for creating a single record is arguably pretty easy to hide behind a graphical user-friendly interface (GUI), but it’s a little more different to do that for the composite resource, as each data entity you put in the HTTP request can reference other data entities in the same request. Taking it a step further, Salesforce also supports receiving a “data graph” in an API endpoint, which requires an even more complex GUI to painlessly develop programs against it.
When developing multi-step integrations, it can be useful to have the ability to customize error-handling. For example, consider the situation in which we fetch a Customer entity from System B and it only has an ID. We might want to enrich this entity with more details, such as username, email, or team members, by requesting them separately from System B. If the privacy of this entity in System B changed such that System B reports the entity has been deleted, then we may need to decide how to handle this unexpected situation. An integration platform which doesn’t support customizable error-handling will likely treat the entire process as a failure, and perhaps won’t communicate useful failure reasons.
Automate.io
Like IFTTT and Zapier, the interface language is “Select Trigger”, then “Select Action App” and “Configure Action”.
Documentation is sparse: Help Center | Automate.io
The Action Configuration language is generally picklists, which populate holes/parameters of the Action App. If you know the API used by an Action App is capable of more than the Action App provides, you need to forget about the easy-to-use Action App and use the Webhooks Action App instead. The Webhooks Action App will likely allow you to create the HTTP request you expect, but at great pains.
The Webhooks Action App has the following issues:
- No authentication manager. Manually getting an auth token from an OAuth 2.0 provider is a huge pain.
- No support for creating a complex JSON string for an HTTP body.
The Salesforce Action App has the following issues:
- Unable to write an arbitrary SOQL query to execute, instead you must use their query builder interface.
- Unable to use the more interesting REST API methods, like the Composite Resource or Autolaunched Flows.
The available actions have the following issues:
- No clear facilities of handling a collection of entities, such as a list of Muse customers.
- No clear facilities of defining named parameters of a workflow, such as Salesforce Org name.
n8n.cloud
The interface language follows the ideas of Dataflow Programming as an attempt to make API integrations easy to create and maintain. It consists of a set of nodes, each of which can have multiple arrows pointing at it and multiple arrows pointing out of it. This adds some nice flexibility of adding multiple ways of triggering a workflow, such as by email, HTTP request, or schedules.
Documentation is pretty good: Reference Overview | Docs (n8n.io)
The Node Configuration consists of picklists and free-text fields, as well as a way of inserting “mergefields” to insert data from another node in the workflow. These mergefields are called Expressions, documented here Expressions | Docs (n8n.io), which is any expression in the JavaScript language.
Issues with Node Configuration:
- It appears Expressions don’t allow you to import JavaScript modules from a module/package registry, such as NPM. This is understandable, because as a platform it’s good to limit the runtime and side effects of huge programs/expressions, but as a developer of integrations, this is very limiting.
Issues with the HTTP Request node:
- Unclear execution semantics. When I wrote an HTTP request to insert a single Lead and pushed the “Execute Node” button, it created 4 identical leads and the n8n GUI shows that it took this action.
Useful Nodes:
-
Function, Function Item nodes: Add/remove items from a collection, or edit items in a collection. Supports JavaScript expressions to do complex collection manipulation.
-
Set node: Define a primitive-typed variable and set its value. Support JavaScript expressions to define non-trivial values.
Integromat
The interface language follows the ideas of Dataflow Programming, like n8n.cloud. It consists of a set of modules, each of which has an arrow specifying which module should be executed next.
Documentation is pretty good, but it’s difficult to assess any gaps: Integromat Support
The Module Configuration consists of picklists and text fields. When a text field is in focus, a panel appears which lists names of other values in the workflow you can use as well as functions to further modify values.
Issues with JSON creation:
- Unable to easily design a specific JSON structure. It supports defining an entity schema to use to build a JSON value, but the only way to create a JSON value which fits this schema is to use the mapping interface to specify the data to place into it. Doing this will create one JSON value for each entity which was used to map, rather than one JSON value containing two entities. It’s important to be able to easily build the latter to efficiently use HTTP requests. The current design builds many “bundles” and use many module-executions. This seems to be designed to suit Integromat’s billing system, which counts each module-execution and HTTP requests.
Useful Modules:
- HTTP (Make an OAuth 2.0 request): Handles authentication for the request.
- Salesforce (Make an API Call): Can use any Salesforce REST resource, and Integromat will re-use authentication from other Salesforce modules.
Pipedream
The interface language is a list of steps which execute when a trigger occurs.
Documentation is pretty good: What are workflows? (pipedream.com)
The Step Configuration consists of a simple graphical interface, such as picklists, text fields, and radio/check boxes, to specify the parameters the step is designed to accept.
Dynamic values can be specified to enrich a parameter’s value by selecting a value from the toolbox panel which appears when a text field is in focus. This adds an expression which references the selected value in braces, like {{ … }}.
If the graphical interface to the parameters of a step are insufficient, you can open the “code” section to see the JavaScript code which implements the step, then edit it and save it as a custom version of the step. This JavaScript code has the ability to freely import modules from the NPM module repository to use. This is appealing because it enables you to easily handle complex requirements that an integration might require.
Useful Steps:
- Inject authentication to a NodeJS step: Build a new NodeJS step, then click the ‘+’ icon underneath the green NodeJS logo on the left side of the step to inject an authentication token for a specific web service into the step.
Answers to relevant concerns:
- Log retention - Longer retention for event source logs? - Help - Pipedream
- High-availability
-
As you saw on our status page, this system is highly available, but we currently make no guarantees on uptime or have SLAs for that service. Occasionally, we may return a 5XX HTTP error to the client making the webhook request. In that case, we haven’t stored the details of the incoming request, and we expect the client to retry the request at a later time.
-
– Is a Pipedream webhook listener Highly Available (HA)? - Dev - Pipedream