CI with GitHub Actions

GitHub Actions provide a simple way to run custom CI jobs triggered by GitHub events like raising a pull request or pushing to main. They are well-integrated with the GitHub UI and accessible cross-divisionally to all who have repository access. They are defined in the repository itself as GitHub Action workflows. This allows them to be updated via the same process as source code changes. A high level of customisation is possible, alongside a extensive library of open source and/or 3rd party actions. Jobs that require access to an internal service or environment can use self-hosted workflow runners that are connected to the inter-divisional network (IDN).

Common uses are:

• To run linters, compile & build checks, unit tests and so on to provide fast automated feedback on pull requests. This provides clear UI feedback to both contributor and reviewer. Draft pull requests can be raised for feedback on pre-review work.
• Package and publish release artefacts to other systems for divisional deployment. OpenID Connect can be used to safely authenticate with cloud providers like AWS.

Docs on how to use workflows are available from GitHub. Training courses are also available – contact the Inner Source Team for availability. Jobs can be executed on Linux, macOS or Windows job runners.

Pull Request Validation

To validate the changes in a pull request you can use the pull_request event. For example to run some unit tests as a status check on any pull request:

# /.github/workflows/unit-tests.yml
name: Example Test Runner
permissions:
  contents: read
on: [ pull_request ]

jobs:
  unit-tests:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Run unit tests
        run: mvn test

Package and Publish

Workflows can also be used to package and publish build artefacts. For example to push to a Docker registry whenever a new version is tagged in the repo:

# /.github/workflows/docker-publish.yml
name: Publish Docker to GitHub Packages
permissions:
  contents: read
  packages: write
on:
  push:
    tags:
      - 'v*'

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: $

jobs:
  build-and-push-image:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Log in to the registry
        # note how 3rd party provided workflow is pinned to a commit SHA:
        uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
        with:
          registry: $
          username: $
          password: $

      - name: Build and push Docker image
        uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5
        with:
          context: .
          push: true
          tags: user/app:$

Note that in most cases you will need to configure some authentication between the GitHub workflow and the system you ate publishing to:

• OpenID Connect allows your workflows to exchange short-lived tokens directly from your cloud provider like AWS.
• GitHub Packages can use the GitHub workflow permissions and GITHUB_TOKEN provided.
• GitHub Secrets can be used to safely store authentication tokens for other systems.

Workflow Security

GitHub describe how to security harden GitHub Action workflows. To summarize:

• Consider in threat modelling workflows created and triggered in the non-default (unprotected) branch: e.g. by anyone with write access. This is especially important for inner source repositories with a wide contributor group.
• If using secrets, consider the use of environments to restrict access to selected protected branches.
• If using a 3rd party action in your workflow, pin the version to a commit SHA to ensure it’s immutable.
• Restrict the workflow job permissions (ideally read-only) to reduce the impact of a workflow token leak.
• If authenticating to AWS (or any major cloud provider), prefer OpenID Connect to manage the workflow auth within AWS IAM directly rather than saving AWS access tokens in GitHub secrets.

Cross-Repository Workflows

A common requirement in GitHub workflows is accessing other repositories to use scripts, data files or other dependencies. In Flutter-Global a read-only method is provided to all repository workflows to make this easier for you.

Self-Hosted Runners

GitHub Action workflows run by default on GitHub managed infrastructure and operating system images which is suitable for most use-cases. If you have internal network access requirements or need more control over the runner infrastructure you can use the Flutter self-hosted runners. These run on AWS infrastructure and with some internal network routing available through the inter-divisional network (IDN).

Consider using self-hosted runners if:

• you need to access internal services like test environments or an internal Artifactory instance.
• your jobs are exceptionally demanding of compute, memory or other specific requirements.

Enabling Self-Hosted Runners

Access to the self-hosted runners are not available to your repository by default. To request access contact the Inner Source Team or raise a PR directly on the runners.csv file in the root of the product-inner-source repository.

Using Self-Hosted Runners

To use the self-hosted runners simply adjust the runs-on key in your workflow to:

runs-on: centos-self-hosted

The base images used for self-hosted runners is based on that provided by GitHub for their managed runners to reduce the differences between running on self hosted vs GitHub managed runners. The base image specification is open to contribution in the Flutter-Global/aws-ec2ami-ghr repository.

Managing Cost

Running workflows on GitHub managed runners is billed by minute. The costs are modest and for 99% of use-cases can be ignored as they fall under normal usage billing process and cross-charged to your divisional budget. Please contact the Inner Source Team for billing advice if:

• You intend to use MacOS runners as these are significantly more expensive.
• You usage is likely to be unusually high e.g. regular scheduled jobs and/or hours-long build times.