Are Pipedreamers Experiencing Another Round of Google API Quota Limits for Actions Like Google Drive?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Pipedreamers, anyone getting another round of Google API quota limits for things like Google Drive actions?

2

image.png
image.png1229×267 18.7 KB

3

cc

4

Hi Shawn, can you share what types of actions you are using where you’re hitting this limit? I’m looking into the quota now, and I see that the queries per minute per user is set by Google to 6000 queries per minute per user - I’m wondering what operations might be causing you to hit this?

5

Let me have a look. Deets inc.

6

Ok so the WF in question is p_pWCWnRD

7

I’ve toggled the PD visibility switch on

8

It’s one of our first workflows: for each new customer, create a Customer folder in Google Drive (with several subfolders)

9

If you have a look inside you’ll see it’s very lightweight and straightforward: 1 base folder, 6 subfolders

10

And while I would definitely be lying if I didn’t say we are struggling with PMF scale related issues, this one still feels very laid back. I mean…16 calls so far today during business hours.

It’s not one of those 100/sec type WFs.

11

I typically never hear from this WF. It always…just runs.

12

Shawn, can you try two things here:
• Limit concurrency - it’s possible that some of the create folder actions in your workflow are running concurrently and running up against some internal queries-per-second limits on Google’s side.
• Auto-retry errors - the steps that failed due to quota issues should be retried
If you have other sources that are watching Google Drive for example, these can be taking up a lot of the 6000/user/min quota that is in place.

13

Is that project number (342853642011) ours or PD’s? If ours, perhaps I can request a quota increase?

14

(from Google)

15

the project is Pipedream’s in this scenario

16

Shawn, a potential option if you are continuing to run up against the per-user API rate limit would be to configure your own Google Drive app in GCP, and we would provide the redirect URI and handle all the authentication - that way you can set the per-user quota to the maximum and request additional headroom from Google as well. let me know if that’s something that you’d like to explore, and I set up the configuration on the Pipedream side - this would be very similar to the Gmail (Developer App) setup.

17

Got it. :+1: Will review and assess next step