How to Edit Python Code in Custom Step Using REST API?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

editing code in custom python step via REST API

hey there - apologies in advance if this is the wrong forum. im using the REST API to create/update workflows. the issue im having is that there doesn’t seem to be an option to edit the python code via api - am i missing a trick? this functionality would really help scale our pipedream usage as we could integrate with our ci/cd process

using a template workflow with http trigger and python step - im able to create a duplicate workflow via api. however, it’s unclear to me what i need to do in the api request to create/update the python code

2

we currently don’t expose an API for editing the code of existing workflows, but I’m sharing this feedback with the team. Can you share how your CI / CD setup works and more about the use case for editing the code for existing workflows?

3

It’s a clunky workaround, but you could create a new workflow via API with your change, then deactivate the old workflow using the update endpoint.

But I’d imagine your constraint is that the URL for triggering the workflow is hardcoded, or not easily updated.

4

thanks that could be a step in the right direction. re. new workflows: how do i edit my workflow definition to allow for custom python?

workflow_definition = {
  "project_id": pipeline_project_id,
  "org_id": pipeline_org_id,
  "template_id" : 'tch_7Jf2bL',
  "steps": [
  ],
  "triggers": [
    {
      "props": {
        "emitShape": "ERGONOMIC",
        "responseType": "default",
        "discardAutomatedRequests": None,
        "domains": []
      }
    }
  ],
  "settings": {
    "name": "dev_property_condition",
    "auto_deploy": True
  }
}
5
  • thanks for the quick response. we’re using databricks notebooks (python) to deploy custom dashboards and web pages into our product. and pipedream endpoints would allow for more interactivity on these pages. for example, submitting form data to our product database, retrieving data based on user feedback, etc.
6

Really good question.

I think you’re hitting a limitation of what the programmatic workflow API offers.

You can modify the props injected into the new workflow, but not modify the steps structure yet.

7

But I’m double checking on that with the team to make sure that is in fact a constraint

8
  • id be happy to have a 15-minutes zoom call over the next couple of weeks with you or someone in your team to discuss the use case in more detail.

main REST API gaps for ci/cd integration are:
1/ listing existing workflows by name so that we only create the workflow (endpoint) when it doesnt already exist for that environment
2/ ability to edit custom python code step via api both for new and existing workflows; editing python code for existing workflow is equally important since there is no delete option via api

not sure if that’s a big lift for your team, so perhaps showcasing the use case in more detail may help decide if the juice is worth the squeeze

9

Just circling back, I was correct. At this time you can only create workflows with dynamic props , not steps.

And we also noticed that Python steps and data stores aren’t being included in this API request shown in the dashboard. We’ll fix that.

10

That’s awesome thanks for the detailed feedback! @U052C338AQK is the PM for this API so I’ll let the two of you connect

11

We pretty much have the exact same requirements for workflow API.

It should basically allow us to do absolutely everything without touching the UI.

12
  • great to hear that im not alone in this user story
13
  • im based in london, given the time difference, i can meet between 5pm and 6pm (GMT) on tues to thurs this week. 20-mins should suffice. let me know if there’s a time that suits
14

Hi - will DM you!