Can I build a cache for an API using Pipedream?

This topic was automatically generated from Slack. You can find the original thread here.

Raymond Camden : I guess this could be in help, but it felt very generic so asking here. :wink: I’d like to hit an API and cache the results for some time, let’s say a day. I could use checkpoint, but has anyone built a simple utility to abstract: if foo in cache and not expired return else execute custom code {}

Dylan Sather (Pipedream) : Take a look at this workflow. It saves data to a cache in $checkpoint where items are stored by the key passed in the body of the HTTP request:

curl -d '{ "key": "1" }' https://endpoint.m.pipedream.net

In this example I’m hitting swapi.dev to fetch data on characters, so you can pass a key of 1, 2, etc. to fetch data for various characters.

If the data for key has been cached in the last 24 hours, we return the data from the cache. Otherwise, we make an API request to swapi.dev for the data for the given character ID and refresh the cache.

I also use DynamoDB for a lot of use cases like this. You can set the TTL / expiry for data at a given key. When the expiry is reached, the key will expire, so when you get the data at that key, you’ll get undefined data back if it expired. Dynamo is nice when you need to have the cache available across applications / workflows.

Let me know if that helps!

Raymond Camden : that’s a great example. also, ive never used DynamoDB, but that one feature alone may get me to check it out