Questions about limitations in the SQL Service

This topic was automatically generated from Slack. You can find the original thread here.

Liam Manderson : Hi all, I am trying to understand the use case for SQL, but am unsure what the limitations are:

  1. Do SQL services persist, and for how long?
  2. Is there a size limit for SQL tables
  3. I understand that the SQL service does not support update or delete rows, but can a whole table be overwritten with new data?

Nghia Nguyen : the built-in pipedream SQL service? Docs say it’s an append-only service that apparently mutates schema as needed (assuming it just adds more columns as needed, never deleting them, nulling out columns that aren’t utilized in new appends)

Nghia Nguyen : You’d most likely be better off with a free tier RDS postgres SQL instance… or you could also do a free cluster MongoDB Atlas NoSQL DB.

Liam Manderson : Thanks for answering. My use-case is to pull data from a google-sheet and store it, so that I don’t need to re-query the Google sheet on every query, so all I need is a single table with very little content. Is this do-able with SQL service or would it absolutely require a separate database?

Nghia Nguyen : To be honest, I’d still use something other than the built-in SQL service as it is.

Dylan Sather (Pipedream) : I’d recommend checking out our workflow state API for this. You can store any JSON-serializable data there and read it on future invocations of the workflow. I think that might work for your use case

Liam Manderson : That looks like a good possibility! Thanks. I will check it out.