What Causes the 'Error Communicating with Data Store' in a High Priority Workflow and Should We Build a Code Retry?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Hello, we are receiving this error occasionally ‘Error communicating with data store’. Previously had posted here about this and the theory was that the key we were trying to store included a bad character but that is not the case and now we see this error pretty often on a frequently run, high priority workflow.

Any ideas on why this is happening? Should we just build in a code retry into the node step that we are performing datastore actions in?

2

Hi , may I ask how often do you call the datastore and how big is each key?

3

Among all workflows we probably call this datastore 200 times daily. The keys are phone numbers in the format +0000000000

4

I see. Let me try to ask Pipedream Engineer to see if they can help

5

Hi Tom!
Thank you for reporting. shared this with the team.
Data Stores are currently in beta, so there are still some limits. Nonetheless, would you mind sharing the link for a specific event that encountered the error?

6
7

Posted thread to Discourse: What Could Be Causing Frequent 'Error Communicating with Data Store' Messages in High Priority Workflow Despite No Bad Characters in Key?

8

any updates on this?

9

Data stores will soon be prioritized! If you’re encountering too many issues, I suggest using a 3rd party database like MySQL for example

10

Or optimize your data stores keys so get/set doesn’t get called too often