Flutter CI/CD (GitHub Actions-Codemagic-Shorebird)
In this article, I will demonstrate how to design a sample CI/CD pipeline for a Flutter application. The concepts I’ll cover are based on my own experiences as well as the projects I have seen and researched.
Before diving into the CI/CD pipeline, it’s important to emphasize the significance of version management and branch structure. A proper branch structure and versioning allow developers to manage the process in a secure and organized way. When branch protection rules, PR processes, and version tagging are integrated into the CI/CD process, both development speed and quality increase.
Branch Structure
In a Flutter project, you can manage your development process by setting up your git branches around different “flavors” (environments). We can structure it like this:
- Master (Production): The branch that contains the live version of our app.
- Staging: The branch where testing and QA processes take place.
- Dev (Development): The branch actively used by the development team, where new features and bug fixes are processed.
Each developer creates a new branch from the dev branch to work on. Once a feature is completed or a bug is fixed, these changes are added to the dev branch by opening a pull request (PR). After the PR is approved and merged, the related branch can be deleted. This approach supports continuous development.
Branch Protection Rules
How can we prevent direct code pushes to the master and staging branches? Also, how can we ensure that PRs are not merged before workflows are completed?
By using branch protection rules, we can prevent direct code pushes to the master and staging branches. Here’s how to do this on GitHub:
- Go to the GitHub repository settings to create branch protection rules.
- Under the “Target branches” section, select the master and staging branches.
- Enable the “Require a pull request before merging” option. This will prevent direct pushes without a PR.
- Under “Required approvals,” you can specify how many approvals are needed to merge PRs.
- Enable the “Require status checks to pass” option, and specify which workflows must be completed in the “Status checks that are required” section.
- Lastly, the enforcement status should be set to “Active.”
With these settings, direct code pushes to the master or staging branches are prevented, and PRs cannot be merged until workflows are completed.
Versioning and Release Process
When we want to release a new version to the Staging or Production environment, we can do this by opening a pull request (PR) from the dev branch to the staging or master branches. Once the PR is opened, the related GitHub Actions workflows are triggered, and versioning is done once these workflows are completed successfully.
Release and Patch
-
Staging Environment: When releasing a new version in the staging environment, we increase the build number of our Flutter app and release it. This means we only update the build number while keeping the same version number.
- 1.0.0+1 -> 1.0.0+2
-
Production Environment: In the production environment, we need to increase both the build number and the version number according to the type of change we made. Changes at the major, minor, or patch levels should be reflected in production releases.
- 1.0.0+5 -> 1.0.1+6 (patch)
- 1.0.1+6 -> 1.1.0+7 (minor)
-
Patch Versions: When we want to apply a patch, we add a “patch” label to the build number and specify which patch it is. This method allows us to release small fixes to the production environment quickly.
- 1.0.1+2 -> 1.0.1+2-patch1
- 1.0.1+2 -> 1.0.1+2-patch2
In mobile applications, the maximum version code we can have is 2,100,000,000.
GitHub Tags and Releases
Every time we release a new version, we push the related version tag to GitHub. These tags allow us to mark and track specific versions. Additionally, whenever we release a version for the production app, we can create a GitHub Release to track the changes between the previous version and the new version. This way, we can easily see what has been updated between versions.
GitHub Release Steps:
- Tag Creation: When a new release is made, a git tag with the relevant version number is created and pushed to GitHub.
- Release Creation: On GitHub, a release is created by selecting the created tags. This release includes the release notes and changes (changelog).
- Viewing Changes: On GitHub Release, all changes between the previous version and the new version can be listed. This makes it easier for developers and users to access the release notes and track updates.
This process ensures that versioning and release operations are orderly and trackable. Especially for changes in the production environment, using GitHub Tags and Releases is essential for careful tracking and documentation.
CI/CD Pipeline Configuration
At this stage, we can configure our pipeline using GitHub Actions, Codemagic, and Shorebird.
- GitHub Actions: Manages workflows that are automatically triggered when code is pushed or PRs are opened.
- Codemagic: Manages the build and distribution processes of our Flutter applications.
- Shorebird: By using Shorebird CodePush, we can deliver code changes instantly to end users without releasing a new version to the app stores.
Let’s go through the detailed steps of the pipeline:
- GitHub Actions Workflow: When code is pushed or a PR is opened, lint, test, and build checks are performed.
- Codemagic Build & Release: The build process is carried out for both iOS and Android versions of the app using Codemagic. Distribution to Google Play Store or App Store can be automated via Codemagic.
- Shorebird CodePush: Shorebird can be integrated to instantly deliver small code changes to users’ devices without sending a new build.
This pipeline minimizes manual tasks for developers while offering a more secure and faster development cycle through automated tests and processes.
How to Build a Sample CI/CD Pipeline?
In this section, I will explain how to build a CI/CD pipeline for our Flutter project using GitHub Actions, Codemagic, and Shorebird integration. The focus here is on the operations carried out on the master and staging branches. The workflows are designed to be automatically triggered when a PR is opened for these branches and after the PR is approved.
GitHub Actions Workflows
1. Workflow to Run When a Pull Request is Opened:
This workflow is triggered when a pull request is opened. Its purpose is to check if the current version matches the latest tag.
name: Version Check
on:
pull_request:
branches:
- master
- staging
jobs:
check-version:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0 # Fetch all history for all branches and tags
- name: Get the last version from git tags
id: get_last_version_tag
run: |
# Determine the target branch of the pull request
branch_name="${{ github.base_ref }}"
echo "Branch name: $branch_name"
# Get all tags sorted by creation date
all_tags=$(git tag --sort=-creatordate)
# Find the last tag in all tags
last_version_tag=$(echo "$all_tags" | grep -oP '(?<=v)\d+\.\d+\.\d+.*' | head -n 1)
echo "Last version tag: $last_version_tag"
echo "Extracted last tag version: $last_version_tag"
# Export the last version for use in later steps
echo "last_version_tag=$last_version_tag" >> $GITHUB_ENV
- name: Read current version from pubspec.yaml
id: read_pubspec
run: |
current_version=$(grep '^version:' pubspec.yaml | sed 's/version: //')
echo "current_version=$current_version" >> $GITHUB_ENV
echo "Current version: $current_version"
- name: Compare versions
id: compare_versions
run: |
last_version_tag="${{ env.last_version_tag }}"
current_version="${{ env.current_version }}"
# current version is not the same as the last version, so finish the workflow
if [[ "$last_version_tag" != "$current_version" ]]; then
echo "Versions are different: $last_version_tag != $current_version"
exit 1
fi
echo "Versions are the same: $last_version_tag == $current_version"This workflow retrieves the last version tag and compares it with the current version in the pubspec.yaml. If the versions do not match, the PR is stopped, and an error is thrown. This is important for ensuring consistency in the versioning process. You can also add additional checks, like lint and test, at this stage.
2. Workflow to Run When a Pull Request is Approved:
This workflow is triggered when a pull request is approved and pushed to the master or staging branches. At this stage, the Codemagic API is used to decide which workflow to trigger, and the build process is started.
name: Publish Codemagic
on:
push:
branches:
- master
- staging
jobs:
publish:
name: Publish Codemagic
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0 # Fetch all history for all branches and tags
- name: Get the last version on pubspec.yaml
id: get_current_version
run: |
current_version=$(grep '^version:' pubspec.yaml | sed 's/version: //')
echo "Current version: $current_version"
echo "current_version=$current_version" >> $GITHUB_ENV
- name: Get current branch name
id: get_branch_name
run: |
branch_name=${GITHUB_REF##*/}
echo "Branch name: $branch_name"
echo ::set-output name=branch_name::$branch_name
- name: Get workflowId
id: get_workflow_id
run: |
current_version="${{ env.current_version }}"
echo "Current version: $current_version"
echo "ref_name: ${{ github.ref_name }}"
workflowId=""
if [[ "$current_version" == *"-patch"* ]]; then
if [[ "$current_version" == *"-android"* ]]; then
workflowId="patch-android"
echo "patch-android"
elif [[ "$current_version" == *"-ios"* ]]; then
workflowId="patch-ios"
echo "patch-ios"
else
workflowId="patch"
echo "patch"
fi
else
workflowId="release"
echo "release"
fi
if [[ "${{ steps.get_branch_name.outputs.branch_name }}" == "staging" ]]; then
workflowId="${workflowId}-staging"
fi
echo "workflowId: $workflowId"
echo ::set-output name=workflowId::$workflowId
- name: Publish Codemagic
run: |
RESPONSE=$(curl -H "Content-Type: application/json" -H "x-auth-token: ${{ secrets.CM_AUTH_TOKEN }}" --data '{"appId": "${{ secrets.CM_APP_ID }}","workflowId": "${{ steps.get_workflow_id.outputs.workflowId }}","branch": "${{ steps.get_branch_name.outputs.branch_name }}"}' https://api.codemagic.io/builds)
echo "Response: $RESPONSE"
echo "post workflowId: ${{ steps.get_workflow_id.outputs.workflowId }}"This workflow determines which Codemagic workflow to run based on the branch name and current version.
The selected workflow is triggered via the Codemagic API. The appropriate build process is initiated based on whether it is a production or staging environment.
Setting Up and Adding Secrets
In this section, I’ll explain step-by-step how to set up the necessary secrets and add them to GitHub.
1. Required Secrets
Some of the secrets used in our example pipeline are:
- CM_AUTH_TOKEN: Authentication key used to access the Codemagic API.
- Teams > Personal Account > Integrations > Codemagic API > Show
- CM_APP_ID: Codemagic application ID
- Apps > App Settings (https://codemagic.io/app/**appId**/settings)
2. How to Add GitHub Secrets
To add GitHub Secrets to your project, follow these steps:
- Go to GitHub Repository Settings: Navigate to your GitHub repository and click on the Settings tab in the upper right corner.
- Access the Secrets Section: On the settings page, click on Secrets and variables > Actions > Secrets in the left menu.
- Add a New Secret:
- Click the “New repository secret” button.
- Enter the secret name in the Name field (e.g., CM_AUTH_TOKEN).
- Paste your secret value in the Secret field.
- Click the “Add secret” button to add it.
Codemagic and Shorebird Workflows
We are creating a codemagic.yaml file to manage the CD (Continuous Delivery) process of your Flutter application’s CI/CD pipeline. This file contains the necessary settings and steps for build and release processes.
definitions:
prod_env_versions: &prod_env_versions
flutter: 3.22.3
xcode: latest
java: 17
android_signing:
- exampleCiCd-android-key
ios_signing:
distribution_type: app_store
bundle_identifier: com.hasankarli.exampleCiCd
groups:
- shorebird
- google_credentials
vars:
GOOGLE_PLAY_TRACK: "internal"
staging_env_versions: &staging_env_versions
flutter: 3.22.3
xcode: latest
java: 17
android_signing:
- exampleCiCd-android-key
ios_signing:
distribution_type: app_store
bundle_identifier: com.hasankarli.exampleCiCd.stg
groups:
- shorebird
- google_credentials
vars:
GOOGLE_PLAY_TRACK: "internal"
cache: &cache
cache_paths:
- $HOME/.pub-cache
- $FLUTTER_ROOT/.pub-cache
- $HOME/.gradle/caches
- $HOME/Library/Caches/CocoaPods
scripts:
- &get_flavor
name: Get flavor
script: |
CURRENT_BRANCH=${CM_BRANCH}
if [[ "$CURRENT_BRANCH" == "master" ]]; then
FLAVOR="production"
elif [[ "$CURRENT_BRANCH" == "staging" ]]; then
FLAVOR="staging"
else
FLAVOR="development"
fi
echo "FLAVOR=$FLAVOR" >> $CM_ENV
- &extract_version
name: Extract version
script: |
#!/bin/sh
# Extract the version line from pubspec.yaml
VERSION_LINE=$(grep '^version:' pubspec.yaml)
if [ -z "$VERSION_LINE" ]; then
echo "Error: Version not found in pubspec.yaml"
exit 1
fi
# Extract build name (e.g., 1.0.4)
BUILD_NAME=$(echo $VERSION_LINE | grep -o '[0-9]\+\.[0-9]\+\.[0-9]\+')
# Extract build number (e.g., 13)
BUILD_NUMBER=$(echo $VERSION_LINE | grep -o '\+[0-9]\+' | tr -d '+')
echo "Starting build with version: $VERSION_LINE"
echo "BUILD_NAME=$BUILD_NAME"
echo "BUILD_NUMBER=$BUILD_NUMBER"
# Write variables to CM_ENV
echo "BUILD_NAME=$BUILD_NAME" >> $CM_ENV
echo "BUILD_NUMBER=$BUILD_NUMBER" >> $CM_ENV
- &set_keystore
name: Set up keystore.properties
script: |
cat >> "$CM_BUILD_DIR/android/key.properties" <<EOF
storePassword=$CM_KEYSTORE_PASSWORD
keyPassword=$CM_KEY_PASSWORD
keyAlias=$CM_KEY_ALIAS
storeFile=$CM_KEYSTORE_PATH
EOF
- &get_flutter_packages
name: Get Flutter packages
script: |
flutter packages pub get
echo "Flutter packages get done"
- &flutter_analyze
name: Run static code analysis
script: flutter analyze
ignore_failure: true
- &setup_local_properties
name: Set up local.properties
script: echo "flutter.sdk=$HOME/programs/flutter" > "$CM_BUILD_DIR/android/local.properties"
- &xcode_project_use_profiles
name: Use Xcode profiles
script: xcode-project use-profiles
- &ios_pod_install
name: Pod install
script: |
cd ios && pod install --repo-update
- &ios_build_ipa
name: Build IPA
script: flutter build ipa --release --flavor $FLAVOR -t lib/main_$FLAVOR.dart --export-options-plist=/Users/builder/export_options.plist
- &android_build_aab
name: Build AAB
script: flutter build appbundle --release --flavor $FLAVOR -t lib/main_$FLAVOR.dart --obfuscate --split-debug-info=./build/app/outputs/bundle/$FLAVORRelease/symbols
- &shorebird_install
name: Install Shorebird CLI
script: |
# Install the Shorebird CLI
curl --proto '=https' --tlsv1.2 https://raw.githubusercontent.com/shorebirdtech/install/main/install.sh -sSf | bash
# Set Shorebird PATH
echo PATH="/Users/builder/.shorebird/bin:$PATH" >> $CM_ENV
- &shorebird_android_release
name: Build with Shorebird for Android release
script: shorebird release android --flutter-version=3.22.3 --flavor $FLAVOR -t lib/main_$FLAVOR.dart -- --obfuscate --split-debug-info=./build/app/outputs/bundle/$FLAVORRelease/symbols
- &shorebird_ios_release
name: Build with Shorebird for iOS release
script: shorebird release ios --flutter-version=3.22.3 --flavor $FLAVOR -t lib/main_$FLAVOR.dart -- --export-options-plist=/Users/builder/export_options.plist
- &shorebird_android_patch
name: Build with Shorebird for Android patch
script: shorebird patch android --release-version "$BUILD_NAME"+"$BUILD_NUMBER" --flavor $FLAVOR -t lib/main_$FLAVOR.dart -- --obfuscate --split-debug-info=./build/app/outputs/bundle/$FLAVORRelease/symbols
- &shorebird_ios_patch
name: Build with Shorebird for iOS patch
script: shorebird patch ios --release-version "$BUILD_NAME"+"$BUILD_NUMBER" --flavor $FLAVOR -t lib/main_$FLAVOR.dart -- --export-options-plist=/Users/builder/export_options.plist
publishing_store: &publishing_store
google_play:
credentials: $GCLOUD_SERVICE_ACCOUNT_CREDENTIALS
track: $GOOGLE_PLAY_TRACK
submit_as_draft: false
app_store_connect:
auth: integration
submit_to_testflight: false
submit_to_app_store: false
publishing_notifications: &publishing_notifications
email:
recipients:
- example@mail.com
slack:
channel: '#channel-name'
notify_on_build_start: true
notify:
success: true
failure: true
release_scripts: &release_scripts
scripts:
- *get_flavor
- *extract_version
- *set_keystore
- *get_flutter_packages
- *flutter_analyze
- *setup_local_properties
- *xcode_project_use_profiles
- *ios_pod_install
- *ios_build_ipa
- *android_build_aab
- *shorebird_install
- *shorebird_android_release
- *shorebird_ios_release
patch_scripts: &patch_scripts
scripts:
- *get_flavor
- *extract_version
- *set_keystore
- *get_flutter_packages
- *flutter_analyze
- *setup_local_properties
- *xcode_project_use_profiles
- *ios_pod_install
- *shorebird_install
- *shorebird_android_patch
- *shorebird_ios_patch
workflows:
release:
name: Release Apps Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *prod_env_versions
cache:
<<: *cache
integrations:
app_store_connect: Codemagic API Key
<<: *release_scripts
artifacts:
- build/**/outputs/**/*.aab
- build/**/outputs/**/mapping.txt
- build/ios/ipa/*.ipa
- /tmp/xcodebuild_logs/*.log
- flutter_drive.log
publishing:
<<: *publishing_store
<<: *publishing_notifications
release-staging:
name: Release Apps Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *staging_env_versions
cache:
<<: *cache
integrations:
app_store_connect: Codemagic API Key
<<: *release_scripts
artifacts:
- build/**/outputs/**/*.aab
- build/**/outputs/**/mapping.txt
- build/ios/ipa/*.ipa
- /tmp/xcodebuild_logs/*.log
- flutter_drive.log
publishing:
<<: *publishing_store
<<: *publishing_notifications
patch:
name: Patch Apps Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *prod_env_versions
cache:
<<: *cache
<<: *patch_scripts
publishing:
<<: *publishing_notifications
patch-staging:
name: Patch Apps Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *staging_env_versions
cache:
<<: *cache
<<: *patch_scripts
publishing:
<<: *publishing_notifications
patch-android:
name: Patch Android Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *prod_env_versions
cache:
<<: *cache
scripts:
- *get_flavor
- *extract_version
- *set_keystore
- *get_flutter_packages
- *flutter_analyze
- *setup_local_properties
- *shorebird_install
- *shorebird_android_patch
publishing:
<<: *publishing_notifications
patch-ios:
name: Patch iOS Workflow
instance_type: mac_mini_m1
max_build_duration: 60
environment:
<<: *prod_env_versions
cache:
<<: *cache
scripts:
- *get_flavor
- *extract_version
- *set_keystore
- *get_flutter_packages
- *flutter_analyze
- *setup_local_properties
- *xcode_project_use_profiles
- *ios_pod_install
- *shorebird_install
- *shorebird_ios_patch
publishing:
<<: *publishing_notificationsWorkflow Definitions and Setup
- At the beginning of the workflow, before defining the workflows, we add definitions that we will use continuously to the definitions section. For Codemagic workflows, we need the following environment groups: shorebird and google_credentials:
- For the Shorebird token, obtain it from this link and add it as an environment group with the name shorebird and the value as the token.
- For Google credentials, follow the steps at this link and add the obtained JSON file with the name google_credentials and the value as the JSON file.
Next, you need to follow the steps in the provided links to complete the configurations for Android signing and iOS signing.
If you wish, you can integrate notifications for failure or successful completion of the workflow via email, Slack.
When a release is successful in Google Play Console, it is sent with the internal test version. For App Store Connect, the version is sent to TestFlight.
For release scripts and patch scripts:
- Obtain the flavor and version, and set which version to patch on Shorebird. Shorebird will handle releases based on the last version released, so no need to specify an additional version.
- Set up the necessary keystore for Android, install the required packages for Flutter, and analyze the code.
- Add the xcode uses profiles script for the iOS profile.
- Then, obtain the app bundle for Android and the IPA for iOS.
- Finally, send the obtained app bundle and IPA to the stores and Shorebird.
Note: To ensure the pipeline runs successfully on Google Play, you must manually upload the app bundle to the Closed Test track for the first time.
Thank you for reading, I hope it is helpful. If you have any questions, feel free to ask on LinkedIn.
You can access the source code of the application from the link below: GitHub - hasankarli/example_ci_cd