Github Sync

When Github Syncing is enabled on your project, Pipedream will serialize your workflows and synchronize changes to a GitHub repo.

Capabilities include:

• Bi-directional GitHub sync (push and pull changes)
• Edit in development branches
• Track commit and merge history
• Link users to commits
• Merge from Pipedream or create PRs and merge from GitHub
• Edit in Pipedream or use a local editor and synchronize via GitHub (e.g., edit code, find and replace across multiple steps or workflows)
• Organize workflows into projects with support for nested folders

Getting Started

Create a new project and enable GitHub Sync

A project may contain one or more workflows and may be further organized using nested folders. Each project may be synchronized to a single GitHub repo.

• Go to https://pipedream.com/projects
• Create a new project
• Enter a project name and check the box to Configure GitHub Sync
  • To use OAuth
    • Select a connected account, GitHub scope and repo name
    • Pipedream will automatically create a new, empty repo in GitHub
    • 
  • To use Deploy Keys
    • Create a new repo in GitHub
    • Follow the instructions to configure the deploy key
    • Test your setup and create a new project
    • 

Create a branch to edit a project

To edit a git-backed project you must create a development branch by clicking Edit > Create Branch

Next, name the branch and click Create:

To exit development mode without merging to production, click Exit Development Mode:

Your changes will be saved to the branch, if you choose to revisit them later.

Merge changes to production

Once you've committed your changes, you can deploy your changes by merging them into the production branch through the Pipedream UI or GitHub.

When you merge a Git-backed project to production, all modified resources in the project will be deployed. Multiple workflows may be deployed, modified, or deleted in production through a single merge action.

Merge via the Pipedream UI

To merge changes to production, click on Merge to production:

Pipedream will present a diff between the development branch and the production. Validate your changes and click Merge to production to complete the merge:

Create a Pull Request in Github

To create a pull request in GitHub, either choose Open GitHub pull request from the git-actions menu in Pipedream or in GitHub:

You can also review and merge changes directly from GitHub using the standard pull request process.

Commit changes

To commit changes without merging to production, select Commit Changes from the Git Actions menu:

You can review the diff and enter a commit message:

Pull changes and resolve conflicts

If remote changes are detected, you'll be prompted to pull the changes:

Pipedream will attempt to automatically merge changes. If there are conflicts, you will be prompted to manually resolve it:

Move existing workflows to projects

First, select the workflow(s) you want to move from the workflows listing page and click Move in the top action menu:

Then, select the project to move the selected workflows to:

Use the changelog

The changelog tracks all git activity (for projects with GitHub sync enabled). If you encounter an error merging your project, go to the changelog and explore the log details to help you troubleshoot issues in your workflows:

Local development

Projects that use GitHub sync may be edited outside of Pipedream. You can edit and commit directly via GitHub’s UI or clone the repo locally and use your preferred editor (e.g., VSCode).

To test external edits in Pipedream:

1. Commit and push local changes to your development branch in GitHub
2. Open the project in Pipedream's UI and load your development branch
3. Use the Git Actions menu to pull changes from GitHub

Known Issues

Below are a list of known issues that do not currently have solutions, but are in progress:

• Project branches on Pipedream cannot be deleted.
• If a workflow uses an action that has been deprecated, merging to production will fail.
• Legacy (v1) workflows are not supported in projects.
• Self-hosted GitHub Server instances are not yet supported. Please contact us for help.
• Workflow attachments are not supported

Github Enterprise Cloud

If your repository hosted on an Github Enterprise account, you can allow the static Pipedream IP address to sync your project changes.

Follow the directions here to add an IP address.

Then add this static IP address 3.214.142.179 to allow Pipedream to sync changes.

Frequently Asked Questions

How are Pipedream workflows synchronized to Github?

Pipedream will serialize your project's workflows and their configuration into a standard YAML format for storage in Github.

Then Pipedream will commit your changes to your connected Github account.

Do you have a definition of this YAML?

Not yet, please stay tuned!

Can I sync multiple workflows to a single Github Repository?

Yes, projects are synced to a single Github Repository which allows you to store multiple workflows into a single Github Repository for easier organization and management.

Can I use this feature to develop workflows locally?

Yes, you can use the Github Syncing feature to develop your workflows from YAML files checked into your Pipedream connected Github Repository.

Then pushing changes to the production branch will trigger a deploy for your Pipedream workflows.

Why am I seeing an error about "private auth mismatch" when trying to merge a branch to production?

This error occurs when both of the below conditions are met:

1. The referenced workflow is using a connected account that's not shared with the entire workspace
2. The change was merged from outside the Pipedream UI (via github.com or locally)

Since Pipedream can't verify the person who merged that change should have access to use the connected account in a workflow in this case, we block these deploys.

To resolve this error:

1. Make sure all the connected accounts in the project's workflows are accessible to the entire workspace
2. Re-trigger a sync with Pipedream by making a nominal change to the workflow from outside the Pipedream UI (via github.com or locally), then merge that change to production

Can I sync an existing GitHub Repository with workflows to a new Pipedream Project?

No, at this time it's not possible because of how resources are connected during the bootstrapping process from the workflow YAML specification. However, this is on our roadmap, please subscribe to this issue for the latest details.

How does the production branch work?

Anything merged to the production branch will be deployed to your production workflows on Pipedream.

From a design perspective, we want to let you manage any branching strategy on your end, since you may be making commits to the repo outside of Pipedream. Once we support managing Pipedream workflows in a monorepo, where you may have other changes, we wanted to use a branch that didn't conflict with a conventional main branch (like main or master).

In the future, we also plan to support you changing the default branch name.