Hello Pipedream experts,
I am using the built-in step to add a row to a google sheet.
Works fine most of the time, but every day I get some RATE_LIMIT_EXCEEDED errors.
The error (seel below) says, that a limit of 300 write requests per minute (!) has been exceeded. But this cannot be that case, I checked the sheet and it has around 250 lines added PER DAY not per minute.
Is this a known error?
What can I do?
I tried to find (within my google account) some usage analytics, but could not find. I did not create a âgoogle development projectâ, the error has this: project_number:342853642011
What is this number?
Or maybe this is a pipedream-owned API project and this reaches the limit (because of usage by other workflows?)
Thank you for your support,
Johannes
Great question, I recommend using the throttling settings of your workflow to guarantee that bursts of events donât cause the workflow to exceed the rate limit on the API youâre integrating with.
For example, you can set your throttle to 59 events per minute to be safe.
Iâm surprised the error message is saying 300 requests per minute, in my experience Google Sheets throttles writes to 60 per minute.
Hello @pierce
I do not think that the problem is the number of write requests from my workflows. As described, my workflow adds around 250 entries to the google sheet PER DAY on average. this happens between 10am and 7pm. It is the count per day, not per minute. So I wonder how a rate limit of 300 per minute can be reached, or even a limit of 60, as you mention.
I rather thought that in the background, a collective API from pipedream to google sheets might be used (by different workflows/users) and this causes the limit to be reached. Can you confirm this?
thank you,
Johannes
@johannes1 your suspicion may be correct, that the rate limit was applied to the Pipedream Google Sheets app. We are investigating now and have requested a quota increase from Google in the meantime.
Hi @danny
I did not get any write request errors due to quota limits any more.
BUT today I received two erros for Read requests to Google Sheets:
Error â Quota exceeded for quota metric âRead requestsâ and limit âRead requests per minute per userâ of service âsheets.googleapis.comâ for consumer âproject_number:342853642011â.
Could you check the quota limit there, and eventually increase this as well?
I am not sure why I get a read request error because I do not read, I just add rows with my workflows.
Hi @danny the google sheets quota problem is probably back, i had many of those errords today at around 1pm CET. Errors occur when calling âadd one rowâ to google sheet - raises âResource exhaustedâ, see screenshot. I am not sure wether this is the same as the quota issue or something new.
Or was there a general issue on the pipedream platform today around 1pm? I got so many âtimeoutâ, âout of memoryâ and âresource exhaustedâ errors.
Hm, interesting. I donât believe we had any incidents on 2/27, but there could have been transient issues with Googleâs API perhaps. Are you still getting these errors? Can you send a screenshot?
Hello @danny today again many errors âresource exhaustedâ with google API to add one row to a g-sheet. see this screenshot.
can you do something about this? It gets more and more unreliable.
this time itr was a big order with 65 line items coming in, and then it hit the google api almost in parallel with around 30 requests in the same second. Can this be a problem?
â update: I had another incoming order with 113 items, a good test case. without throttling and unlimited concurrency, it hit the limits every time i tested it.
Now with throttling to 1 every 3 secs and a concurrency of 3 it finished successfully. I hope this solves the issue.
