News from the DevOps team: Improving our CI/CD reliability with RunsOn
From Travis CI to RunsOn
OpenProject uses a large test suite for our Rails-based application, which includes a high number of compute-intensive system tests. In the past, we used Travis CI before moving to GitHub Actions with the help of Cyril, creator of RunsOn. He provided us with a basic tool to run the test suite on multiple private hosted runners that were also created on-demand. In that sense, our CI has been running an alpha version of RunsOn for quite some time.
More speed, a higher reliability and lower CI costs
Transitioning to RunsOn for our private runners has been a game-changer for reliability and observability. Not only have we seen a substantial reduction in our daily CI costs, but we’ve also experienced a notable increase in the speed and reliability of our test runs. The ability to leverage AWS instances with spot bidding has allowed us to run extensive test suites at a fraction of the previous cost on faster instances, without sacrificing performance.
The entire test suite now finishes in around 10 minutes, including build, caching, and all other setup times. Previous iterations would easily run in the 20-30 minute range or even longer, depending on the underlying instances.
A 10-minute integration process
The integration process with RunsOn was remarkably straightforward. Within just 10 minutes, we had the system up and running in our private AWS account, seamlessly syncing with our existing workflows. Due to full compatibility with official GitHub runners, RunsOn has allowed us to maintain our established workflows while benefiting from all the improvements mentioned earlier. The deep integration with AWS services, including unlimited fast caches and dynamic instance selection, has provided us with a flexible and scalable solution. We’ve been particularly impressed with the ability to handle an arbitrary number of workflows concurrently without any issues, ensuring our development pipelines remain smooth and uninterrupted.
Since the introduction of RunsOn to our test suite,we have also rolled it out to other parts of the DevOps pipelines, such as containerization of our application for Docker Hub (including faster builds for arm64 images thanks to native arm64 runners) and other related products.
Our sincere recommendation
Overall, RunsOn has transformed our CI/CD process, making it faster, cheaper, and more reliable. We recommend it to all teams using GitHub Actions at scale for their pipelines and testing.
For more information, see https://runs-on.com/
