This topic was automatically generated from Slack. You can find the original thread here.
is there a way to automatically check in all Components to a Git repository? I want my components to have version control.
Currently, Pipedream does not provide a built-in feature for automatically checking in all components to a Git repository for version control. However, you can manually manage your Pipedream components with Git by following these steps: 1. Write your components locally using your preferred code editor. 2. Initialize a Git repository in your local project directory. 3. Commit your component code to this repository. 4. Push the repository to a remote Git service like GitHub, GitLab, or Bitbucket for version control. Remember to use the Pipedream CLI to publish your components to Pipedream after making changes locally. This way, you can maintain a version-controlled history of your components in Git while also keeping them updated in Pipedream. If you need further assistance or have more questions about this process, please visit Pipedream’s support page.
To tack onto this, you could also set up a GitHub action or Pipedream workflow to automatically publish component changes to your Pipedream workspace from deploys to the main branch of your component code.
is there a way to download all the current component sources from the command line?
Great question, I don’t believe so exactly because the current endpoints to retrieve a component don’t include the code:
You can use the list current sources to iterate over active sources on your account for example, but not retrieve the code.
It might be worth a shot using pd describe: CLI Reference
But again, because I don’t believe the REST API includes these details it won’t include the code itself.
It would make a great feature and allow a company to do the backing git store with a script.
Agreed, but the “push” method I described also works as well. It’s exactly how our open source components work: https://github.com/PipedreamHQ/pipedream/blob/master/.github/workflows/publish-components.yaml
Can you tell me more about your current development flow? Are you publishing private actions right from the workflow builder with Node.js steps? Or are you still developing locally and pushing via the Pipedream CLI?
We have several components defined using the Pipedream interface. I want to move them to version control as you described.
Got it, and these are sources or actions?
Hmm. They are found by clicking the “Components Beta” button on the left nav.
Sorry, I’m just trying to understand what you mean by “published by the Pipedream interface”.
That listing contains all components, it doesn’t filter by if they were published from Node.js code steps in the workflow builder, or if they were published from the Pipedream CLI.
I might be confused, but as we built workflows, we built components and they show up here. I guess they are actions?
Sounds like perhaps someone else on your team authored these components?
Nice. Yes all actions.
Yes, as part of building complex workflows, we built these. The start of this question was that I want to somehow get these under version control.
Ok, perhaps asking your team member who authored these actions to add the Github action as I described to their local code.
I suspect the team member that developed these actions probably published them from their own computer, and has the code locally.
So having them check it into a Github Repository and publishing over a Github action should automate that backing up process.
That’s what I recommend in the short term, but you can also request for new features here:
That will make an issue on our GitHub Repository where we can then notify you when they’re available.
OK, I was just trying to see if this was currently “scriptable” by being able to download all the current components from Pipedream.