GitHub Actions is eating the world

Continuous integration (CI) is the automation of integrating code changes from multiple contributors into a central repository where automated builds, tests, and code quality checks run.

For a long time, software projects would typically use standalone CI services like Jenkins, Travis or CircleCI. Some CI services are built into social coding platforms. For example, GitLab has had CI capabilities since 2012. The currently most popular platform, GitHub, announced its own integrated CI service, GitHub Actions, in 2018.

This week’s paper looks at the rise of GitHub Actions and the subsequent fall of its competitors.

Researchers analysed more than 201,000 public repositories on GitHub that are associated with an npm package. Almost 92,000 repositories include or have at some point included files and directories that are used for CI services, such as travis.y(a)ml and .github/workflows.

Travis and GitHub Actions are by far the most popular CI services. Each has been used by more than half of all repositories with CI. Their combined “market share” among repositories with CI exceeds 90%. This is to be expected for Travis, which has been around since 2011, but it’s somewhat surprising that GitHub Actions has managed to become the most popular CI given that it made its entrance just a few years ago, in 2019.

The five next popular CI services, following at quite a distance, are CircleCI, AppVeyor, Azure Pipelines, GitLab CI, and Jenkins. Together, these seven CI services represent 99% of all CI usages, and cover 99.6% of all repositories that have used CI.

Most CI services see either a slight recent decrease in market share (Travis, AppVeyor) or reduced growth (CircleCI, Azure Pipelines, GitLab CI). In the case of Travis, this is due to a combination of fewer repositories adopting Travis from 2020 onwards (side note: Travis raised its prices and limited its free plan.) and existing repositories discontinuing its use. The reduced growth of CircleCI, Azure Pipelines, and GitLab CI is mostly due to fewer new users.

Interestingly, most of the churn starts a few months after the introduction of GitHub Actions, which is the only CI service that has a steadily increasing adoption rate and nearly no discontinuations.

It’s not unusual for a repository to use multiple CI services throughout its lifetime.

In some cases multiple CI services are used simultaneously. Travis, AppVeyor, GitHub Actions, and CircleCI are co-used most often. This is often done to improve fault tolerance or to work around limitations imposed by CI services, like the number of jobs that can be run in parallel or the operating systems that are supported for builds (side note: For a long time, Appveyor was the only service that supported Windows builds. Its decline coincides with the availability of Windows builds in competing CI services.).

Another likely scenario is that a repository migrates to a different CI service, for instance because a maintainer is unsatisfied with their current CI. Most migrations are due to repositories migrating away from Travis and (to a lesser extent) CircleCI. Repositories generally migrate to GitHub Actions and (again, to a lesser extent) CircleCI.

Why do so many repositories migrate to GitHub Actions? That’s not within the scope of this study, but the authors do hypothesise that it’s because GitHub Actions…

• is fully integrated with GitHub, which removes the need to deal with a third-party provider;

• is trendy, very easy to set up and use, and can be directly accessed from any repository via the Actions tab;

• comes with a large marketplace, which allows users to build pipelines using predefined components and templates; and

• provides a generic mechanism to automate various aspects of the GitHub development workflow, including comments, issues and pull requests. This makes it more than just a CI service.