I’m encountering 429 errors even though I’ve throttled workflow executions to 29 requests per 3 seconds (I believe their limit is 30/3s but please correct me if I’m wrong).
I’ve found this article which seems to make sense. Do I add a new step (“Python”) after my 2nd step in the workflow (“Generic API Call”) and paste this code exactly as is? I’m very inexperienced with coding so please ELI5. Thanks!!
import requests
def handler(pd: 'pipedream'):
MAX_RETRIES = 3
# 10 seconds
DELAY = 1000 * 10
run = pd.context['run']
print(pd.context)
# pd.context.run.runs starts at 1 and increments when the step is rerun
if run['runs'] == 1:
# pd.flow.rerun(delay, context (discussed below), max retries)
pd.flow.rerun(DELAY, None, MAX_RETRIES)
elif run['runs'] == MAX_RETRIES + 1:
raise Exception("Max retries exceeded")
else:
# Poll external API for status
response = requests.get("https://example.com/status")
# If we're done, continue with the rest of the workflow
if response.json().status == "DONE":
return response.json()
# Else retry later
pd.flow.rerun(DELAY, None, MAX_RETRIES)
I think throttling is quite a complex task to complete with an inexperienced programmer. I think to best save your time, you might need dedicated support for your usecase. For this you can subscribe to Pipedream team plan to be invited to a dedicated support channel, or you can hire a Pipedream expert using this link: Connect with a Pipedream Partner
Will this work? “{{steps.trigger.context.id}}” is configured as a parameter in step 2 (Generic API Call) to Kucoin Futures that identifies “clientOid” on every trade execution.
import requests
def handler(pd: 'pipedream'):
MAX_RETRIES = 5
# 10 seconds
DELAY = 500 * 10
run = pd.context['run']
print(pd.context)
# pd.context.run.runs starts at 1 and increments when the step is rerun
if run['runs'] == 1:
# pd.flow.rerun(delay, context (discussed below), max retries)
pd.flow.rerun(DELAY, None, MAX_RETRIES)
elif run['runs'] == MAX_RETRIES + 1:
raise Exception("Max retries exceeded")
else:
# Poll external API for status
response = requests.get("https://api.kucoin.com/api/v1/orders/{{steps.trigger.context.id}}")
# If we're done, continue with the rest of the workflow
if response.json().status == "isActive":
return response.json()
# Else retry later
pd.flow.rerun(DELAY, None, MAX_RETRIES)