CI with GitHub Actions
Use GitHub Action workflows to provide PR feedback, package and publish your product.
GitHub Actions provide a simple way to run custom CI jobs triggered by GitHub events like raising a pull request or pushing to
main. They are well-integrated with the GitHub UI and accessible cross-divisionally to all who have repository access. They are defined in the repository itself as GitHub Action workflows. This allows them to be updated via the same process as source code changes. A high level of customisation is possible, alongside a extensive library of open source and/or 3rd party actions. Jobs that require access to an internal service or environment can use self-hosted workflow runners that are connected to the inter-divisional network (IDN).
Common uses are:
- To run linters, compile & build checks, unit tests and so on to provide fast automated feedback on pull requests. This provides clear UI feedback to both contributor and reviewer. Draft pull requests can be raised for feedback on pre-review work.
- Package and publish release artefacts to other systems for divisional deployment. OpenID Connect can be used to safely authenticate with cloud providers like AWS.
Docs on how to use workflows are available from GitHub. Training courses are also available – contact the Inner Source Team for availability. Jobs can be executed on Linux, macOS or Windows job runners.
Pull Request Validation
To validate the changes in a pull request you can use the
pull_request event. For example to run some unit tests as a status check on any pull request:
# /.github/workflows/unit-tests.yml name: Example Test Runner permissions: contents: read on: [ pull_request ] jobs: unit-tests: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v3 - name: Run unit tests run: mvn test
Package and Publish
Workflows can also be used to package and publish build artefacts. For example to push to a Docker registry whenever a new version is tagged in the repo:
# /.github/workflows/docker-publish.yml name: Publish Docker to GitHub Packages permissions: contents: read packages: write on: push: tags: - 'v*' env: REGISTRY: ghcr.io IMAGE_NAME: $ jobs: build-and-push-image: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v3 - name: Log in to the registry # note how 3rd party provided workflow is pinned to a commit SHA: uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a with: registry: $ username: $ password: $ - name: Build and push Docker image uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 with: context: . push: true tags: user/app:$
Note that in most cases you will need to configure some authentication between the GitHub workflow and the system you ate publishing to:
- OpenID Connect allows your workflows to exchange short-lived tokens directly from your cloud provider like AWS.
- GitHub Packages can use the GitHub workflow permissions and
- GitHub Secrets can be used to safely store authentication tokens for other systems.
Key aspects of GitHub Action workflow security hardening is documented by GitHub. A brief summary:
- Consider in threat modelling that workflows can be created and triggered in the non-default (unprotected) branch: e.g. by anyone with write access. This is especially important for inner source repositories with a wide contributor group.
- This is why if secrets are required, you should consider the use of environments to restrict access to selected protected branches.
- If using a 3rd party action in your workflow, pin the version to a commit SHA to ensure it is immutable.
- Restricting the workflow job permissions to only what is necessary (ideally read-only) also reduces the risk of using 3rd party actions as it reduces the risk of compromised workflow tokens.
- If authenticating to AWS (or any major cloud provider), prefer OpenID Connect to manage the workflow auth within AWS IAM directly rather than saving AWS access tokens in GitHub secrets.
A common requirement in GitHub workflows is accessing other repositories to use scripts, data files or other dependencies. In
Flutter-Global a read-only method is provided to all repository workflows to make this easier for you.
GitHub Action workflows run by default on GitHub managed infrastructure and operating system images which is suitable for most use-cases. If you have internal network access requirements or need more control over the runner infrastructure you can use the Flutter self-hosted runners. These run on AWS infrastructure and with some internal network routing available through the inter-divisional network (IDN).
Consider using self-hosted runners if:
- you need to access internal services like test environments or an internal Artifactory instance.
- your jobs are exceptionally demanding of compute, memory or other specific requirements.
Enabling Self-Hosted Runners
Access to the self-hosted runners are not available to your repository by default. To request access contact the Inner Source Teamor raise a PR directly on the
runners.csv file in the root of the
Using Self-Hosted Runners
To use the self hosted runners simply adjust the
runs-on key in your workflow to:
The base images used for self-hosted runners is based on that provided by GitHub for their managed runners to reduce the differences between running on self hosted vs GitHub managed runners. The base image specification is open to contribution in the Flutter-Global/aws-ec2ami-ghr repository.
Running workflows on GitHub managed runners is billed by minute. The costs are modest and for 99% of use-cases can be ignored as they fall under normal usage billing process and cross-charged to your divisional budget. Please contact the Inner Source Team for billing advice if:
- You intend to use MacOS runners as these are significantly more expensive.
- You usage is likely to be unusually high e.g. regular scheduled jobs and/or hours-long build times.