Flutter CI/CD: GitHub Actions vs. Bitrise

What’s Flutter?

Flutter is an open-source framework by Google for building beautiful, natively compiled, multi-platform applications from a single codebase. You can build, test, and deploy beautiful mobile, web, desktop, and embedded apps from a single codebase.

Flutter CI/CD 

There are a number of continuous integration (CI) and continuous delivery (CD) options available to help automate the delivery of your Flutter application. In this article, we will compare both options in GitHub Actions and Bitrise. 

Sounds interesting? Let’s get started!

The CI/CD Infrastructure

Stacks

As we know, to be able to build, run, and test Flutter applications, you need to have macOS VMs to build both Android and iOS apps. At Bitrise we always have the latest macOS versions so you can always use the latest stacks in your team. 

On GitHub Actions, they are still working on supporting macOS 11. Windows VMs are also supported with different versions like Windows Server 2022 [beta], Windows Server 2019 and 2016.

With Bitrise you can easily switch between VM types and versions to meet your requirements. All the stacks include both Android and iOS tools, SDKs, and everything needed for your apps. With GitHub Actions, sometimes you need to install different tools, such as Java. Here you can find the stack of Bitrise's latest macOS machines. 

Parallelization

With GitHub Actions, you can run the same workflow with different configurations in parallel easily, such as the OS using the Matrix feature:

On Bitrise, to be able to run parallel builds you need to create sub-workflows and run them at the same time with the Bitrise Start Build Step and configure the workflows names with it. 

Source Code Management

Bitrise is integrated with different source code management platforms like GitHub, Bitbucket, or GitLab. GitHub Actions only integrates with GitHub.

Workflows and Steps

On Bitrise, once you connect your code repository, the Bitrise Scanner detects the application type and gives you the suitable stack with the default workflows, and then you can edit these workflows.

Your workflows/pipelines are easy to configure using the online Workflow Editor, where you can use any of our 330+ Steps and Integrations (all of them are open-source!)

With GitHub Actions, you need to build the YAML file by yourself and there is no visual UI for the steps. On the other hand, one of the biggest features of GitHub Actions is that you don’t need to clone the code repository to any other platform. Your repository and workflow are already at the same place, so you don’t need to have authenticated access to the repo. Bitrise will use the actions/checkout step. This action checks out your repository under $GITHUB_WORKSPACE, so your workflow can access it.

YAML file

Both Bitirse and GitHub Actions use YAML files for the workflows, but as we mentioned above Bitrise uses the Workflow Editor to design and implement CI/CD workflows easily. It also has the option to auto-generate the YAML file and you can modify it through the website or store it in the app repository. 

With GitHub Actions, it’s only a YAML file. Developers have to write it from scratch or select a template but also they need to add other steps to complete the workflow.

Trigger Builds

Bitrise supports the Gitflow process, so you can specify which workflow runs with the trigger map. For example, on every PR you can run the primary workflow, and with tags or releases, you can also run the deploy workflow and also there is an option to trigger workflows manually. 

On GitHub Actions, you can only specify it inside the YAML files. There is no manual trigger, it should only be via new PR or commit to an existing one.

Build

With Bitrise, you can build both iOS and Android projects at the same time or separately, each using its own Workflow. You can set this up in the Platform input of the Flutter Build Step at any time. By default, the Step is configured according to the platform (or platforms) that the scanner detected when the app was added on Bitrise.

In this benchmark, I’ve used a simple Flutter application with the same Steps in the workflow. On Bitrise’s Velocity Tier 2, the build takes only 6m 29s and on GitHub Actions, it takes between 10-11 mins. 

If the Build fails for any reason, with Bitrise, you can rebuild the workflow with remote access using SSH to be able to debug the failure issues.

But with GitHub Actions, there is no option to do that, only to rebuild the workflow. 

Testing & Test Reports

Bitrise has multiple Steps to easily run Unit, Integration, UI, and Security tests on emulators and simulators on a Bitrise virtual machine or cloud devices, such as SauceLabs, BrowserStack, Genymotion, Perfecto.

On Github Actions, you have different steps for testing as well but as we mentioned above you need to add and configure them in the YAML file.

The test results for Flutter tests are integrated with our Test Report Add-on by default. After the build is successfully finished, you can check the results of the test, including the unit tests, widget tests, integration, and UI tests.

On GitHub Actions, you need to add a step in the YAML file to extract the test results.

Deployment & Code Singing

If you have platforms specified in your repository, a deploy Workflow will be automatically generated when adding the app on Bitrise. The content of this Workflow depends on the platforms. For example, if your app contains only an iOS project, the Workflow will contain the Certificate and profile installer and the Xcode Archive & Export for iOS Steps, but if there is an Android version too, it’ll contain an Android Sign Step as well.

Apps and Artifacts 

On Bitrise, the Deploy to bitrise.io Step uploads all the artifacts related to your build into the APPS & ARTIFACTS tab on your Build’s page. Any number of files under 2 GB deployed are stored for an indefinite time.

On GitHub Actions, you need to add this Step as well in the YAML file to be able to download them from the build. 

Pricing

GitHub Actions:

• You pay for the amount of time your CI/CD pipelines are running per billing period. 
• A free tier can be used indefinitely without any cost as long as you stay below the threshold. GitHub Actions gives 2K action minutes per month for free. 

For example, if your organization uses GitHub Team and allows unlimited spending, using 15,000 minutes could have total storage and a minute overage cost of $56 USD, depending on the operating systems used to run jobs.

Jobs that run on macOS runners hosted on GitHub consume minutes at 2 and 10 times the rate that jobs on Linux runners consume. For example, using 1,000 macOS minutes would consume 10,000 minutes included in your account.

• 10,000 Linux minutes at $0.008 USD per minute = $80 USD.
• 10,000 macOS minutes at $0.08 USD per minute = $800 USD.

GitHub calculates your storage usage for each month based on your hourly usage.

Bitrise (compare plans):

• You can pick from different options based on your build needs, each of which will cost a different amount of credits per minute of usage. 
• You will have access to our fastest build platform ever, Gen2 the next generation of Bitrise Build Infrastructure.

• It has a free hobby plan capped at one user, up to 5 concurrent builds, and 90 min./build timeout, for up to 300 credits per month.
• It has a single-user developer plan, which is discounted for students, open-source projects, and Bitrise contributors.
• It offers a trial period of 14 days with all advanced features.
• Its Teams Plan starts at $31.50 per month with unlimited users, with up to 10 (macOS) and 30 (Linux) concurrent builds. You will have a 2-week free trial for Teams and 500 credits to start building. When the trial ends, we will suggest a credit bundle that will fit your needs.
• It has a Velocity Plan with tiers for teams focused on performance starting at $2,500 with unlimited users from 20 (macOS) and 80 (Linux) concurrent builds.
• It provides custom pricing for the Enterprise Build Platform — the virtual private cloud offering.

Happy building!