Search Data Store by Value

Help
1

We want to import a list of products from an e-commerce platform and store them locally in the Pipedream data store. The issue is, the Product ID is unique, but there is a possibility that two Product Names will be the same.

Therefore, the question is twofold:

  • Can we store with a key of product ID and value of product name, and if we do, can we search the Data Store by the value?

or

  • Do we store with a key of product name (and hence have duplicate key values) and set a value of the product ID, and if we call get() on the data store, will it return all rows with that key?
2

Hi @mckltech

Great question. In general, a resource (product, order, customer, product variant) should be stored by a unique ID in any type of database.

If you need to perform more complex queries like searching by value, that’s your first sign that you need to use more robust database.

Data Stores are much like a cache, it’s very good at looking up items by a key name, but it lacks the query support to query by value.

I recommend you look into storing this data into a different database, we support connections with MySQL, Postgres and MongoDB.

1 Like
3

Thanks Pierce, appreciate it!

Yeah, the more I think about it, the more I figure we need something with just a little more ommpphh in this case. Data Stores is awesome, but like you said, not a good fit here.

Thanks again!