Our investment in Earthly
Write code, build it, and test it — this is the most foundational feedback loop within software engineering, and the speed at which one can iterate through this process is arguably the single largest factor contributing to developer productivity. Yet, if you ask the average software engineer what they think of their build system, you’re unlikely to hear a very happy answer.
Build management is arguably one of the most despised facets of software engineering, characterized by myriad problems. Builds are often slow, taking 10+ minutes to run. Builds are frequently difficult to reproduce — for example, when an engineer is trying to reproduce a CI failure locally, or run another team’s code locally when they aren’t used to setting up that team’s environment. Furthermore, a complex, brittle mess of build scripts is often the status quo for build environment and dependency management, leading to a maintainability nightmare.
Indeed, it’s not uncommon to see slack channels titled #waiting-for-jenkins, where engineers with nothing to do waiting for their build to complete share memes and commiserate with each other about how much of their time is seemingly being wasted.
The aforementioned problems — which really only scratch the surface — are greatly exacerbated by the fact that modern software systems are becoming increasingly complex. As the typical software application incorporates more languages, more interdependent services, more testing, more linting, more static analysis, more compilation targets, and more artifacts, the “build graph” of an application — essentially, all the myriad operations that need to happen in a given order to compile, test, and run code — becomes exponentially harder to manage.
Recognition of these challenges has led a few large companies to build extremely sophisticated build frameworks like Blaze/Bazel (Google) and Pants (Twitter), the former of which has become particularly popular among modern tech companies due to its rich functionality around native caching, parallelization, and reproducibility of builds. In fact, Google is probably one of the few places where an engineer is likely to love their build system.
Unfortunately, in the case of build systems, great power has traditionally been coupled with great complexity. While systems like Bazel are incredible once they have been implemented and you have learned how to use them, if you ask the average company who has gone through a Bazel migration what the process was like, you might trigger some PTSD. Bazel is so opinionated, so difficult to learn, and requires such a re-architecture of your existing build process, that adopting it as an organization often takes over a year.
Vlad A. Ionescu was one of those lucky Google engineers who got to enjoy an amazing build system, at least until he decided to found his last company, ShiftLeft. As he built out the engineering team there, he began to realize just how lucky he had had it at Google, and just how bad it was for everyone else. Many of the engineers he worked with struggled endlessly with sluggish, difficult-to-reproduce builds, and while he longed for the capabilities of Bazel, he knew it would be completely infeasible for his team to adopt it at ShiftLeft.
Eventually, this got Vlad, and the rest of the team at Earthly, thinking — would it be possible to come up with a build framework that maintains many of the advantages of these more advanced frameworks, but which was more incrementally adoptable, readable, and flexible? After a lot of thinking and exploration, they ended up with Earthly.
Earthly is a build framework that produces containerized, reproducible, and language-agnostic builds by default. Write a build once, and it will run anywhere! Notably, Earthly does not require you to replace language-specific build toolkits like npm. Instead, it sits elegantly on top of your CI and language-specific build tools, giving you a unified layer to manage and define build processes. Earthly is also easy to use; if you are familiar with Docker or Makefiles, the syntax will immediately make sense to you.
Earthly achieves these profound advantages by treating every build step as the combination of a target, a container image, and a set of upstream build dependencies. This declarative approach allows Earthly to systematically understand the build graph — something not possible in the more common imperative & scripting-based approaches to build management. The result is that Earthly can automate a huge amount of previously manual and cumbersome build tasks for developers, such as caching and parallelization of build execution. Indeed, the typical team using Earthly sees their average build speed increase by 2x, with some builds improving by almost 20x.
This elegant combination of repeatability, speed, and simplicity has led Earthly to grow its open-source community dramatically over the past year — over 1300 teams now use Earthly to power the builds on their CI, including notable companies like Workday, Roche, DeepGenomics, Posit, and ExpresssVPN.
However, what’s most exciting for Earthly is what’s next. As of today, Earthly is officially launching Earthly CI. Although Earthly’s open source build framework offers immense benefits on top of existing CI vendors like Jenkins and GitHub Actions, it turns out that if you design a CI around a modern, containerized build automation system, things get even faster! By controlling the CI and build layer, Earthly can optimize disk IO, networking, and more to truly deliver the fastest builds possible. Coupled with its insane speed is an aligned business model — unlike other CIs which charge you per minute of compute, Earthly CI charges you compute at cost, meaning that the company is just as incentivized as you are to make your builds faster.
More broadly, Earthly’s novel approach to build management has laid the groundwork for many core aspects of the software engineering workflow to be completely rethought. The team suspects that numerous other classically annoying issues in builds, testing, and CI — such as dealing with monorepos, artifact management, handling multi-chip builds (x86 and M1), and build security — can be reinvented by Earthly over time.
As such, we couldn’t be more excited to announce our lead investment in Earthly’s seed+ round, and to partner with Vlad and the rest of the Earthly team in their journey to finally make builds something that sparks joy in developers all over the world.
