This topic was automatically generated from Slack. You can find the original thread here.
Anyone using the HubSpot Sources? We’re seeing a continuous “You have reached your secondly limit.” message but none of our workflows would come near the rate limit. We’re trying to determine if it’s us, or potentially being triggered by HubSpot seeing a requests from the Pipedream IP range.
Hi Colin, which Hubspot sources are you seeing this limit specifically? Are they polling sources? If so, what interval are they configured to?
Contact Property Change, every 15 mins.
Seems to have started about 2 days ago, but we’ve made no changes. Likewise I spent the morning adding some delays to other HubSpot workflows to space out the API calls, and we’re still getting it.
It could be something else with my client I’m not aware of, but I wanted to rule out Pipedream first,
Yea that’s a 429 from HubSpot directly, not from Pipedream’s system.
You might be able to leverage the Event History section to at least narrow down workflow executions. But either you’ll need to find a way to increase HubSpot’s rate limit for your account with them, or decrease your usage on Pipedream or any other app using the same HubSpot token that’s using it.
Yes, I know it’s from HubSpot, but I’m wondering if HubSpot is seeing multiple requests from the Pipedream IP range and blocking us as a result.
I know it’s a stretch but I need to try rule it out.
The Pipedream IP address range is the entirety of the us-east-1 region in AWS.
It’s a pretty large block.
The only other possibility is that they’re throttling based on the User Agent header. But that’s highly unusual for an API performing actions on behalf of an authenticated user.
That kind of IP address/user agent throttling is usually for public routes or webscraping because you can’t throttle based on an account.
Yeah, I had the same thoughts. Guess I can discount that theory.
I’ll keep digging, thank you.
Yes of course. Sorry I wish I could point to more tooling to help you diagnose this.
The only other thing I can think of is making custom sources to proxy all HubSpot requests so you can centralize and throttle usage.
From my reading of HubsSpot’s docs, it doesn’t look like they’re rate limiting the Pipedream OAuth app either, it seems account-specific HubSpot APIs | Usage guidelines
Thanks Danny. Super weird on our end, we can’t find a reason we’d be hitting it.
Ok, this is somewhat related to Pipedream I think.
Turns out my client did a bulk update (~1,700) contacts. The contacts/search has a lower rate limit. batchGetContacts() makes a HTTP request, if there are 1,700 contacts and it can process a max of 50 per iteration, that’s 34 requests, which depending on the speed could hit the 15 calls/second limit, which would trigger this error.
Ah
I’m trying a bad ‘sleep’ in the loop to see if that fixes it.
If you hit the timeout limit for the workflow, it might help to try $.flow.delay instead, or split the HubSpot call into it’s own workflow and adjust the concurrency settings to make sure it’s under the rate limit
Thanks Pierce, the problem is that Pipedream component generates the list of Contacts, so the delay has to be in the component, not the workflow.
Oh got it. Perhaps it would make more sense to use a code step here instead then. So you have that flexibility
Yeah, I think I’ll try that next week but for now I’m going to try bodge together a solution to get the client back online.
Thanks again.
Sure thing.
Feel free to request a new prop on the component as well to optionally delay between calls
