Designing a Package Manager

Less than three months ago, we launched CircleCI orbs, our package management ecosystem for CircleCI configuration. In that time we have seen over 300 orbs created in just the first 10 weeks and more than 200 organizations using orbs across over 1,000 projects in tens of thousands of builds. We are also featuring more than 20 official partner orbs through our new Technology Partner Program. We’re incredibly grateful for the huge and positive response.

In this piece, I want to take a look back and share the key design decisions that went into creating and launching CircleCI orbs. If you’re using orbs today and want to understand how to get the most from them, or if you’re thinking about designing your own package management system, this post is for you.

What are CircleCI orbs?

Orbs are shareable packages of CircleCI configuration. Like most other code-packaging systems, orbs provide a common set of abstract structural elements, making reuse of tested, encapsulated build components practical across your organization as well as the broader ecosystem of users. Orbs have a domain-specific language (DSL) defined in YAML syntax that expresses the semantics of our build configuration. This makes it straightforward to compose idiomatic interfaces for encapsulated commands, jobs, and executors in your build configuration.

Orbs also provide package management infrastructure. This makes it straightforward to publish clearly versioned orbs and then import them for use in projects.

What problems were orbs designed to solve?

Our job is to help teams animate and accelerate their software delivery. One of the key ingredients to getting the most from CircleCI is our build configuration, a YAML-based semantics to describe the workflows, jobs, and environments of CI pipelines. When we started building orbs we wanted to solve three key problems with regards to the build configuration:

1. Better DRY support The configuration in CircleCI 2.0 is designed to be highly transparent, expressive, and deterministic. However, those traits can also make our base semantics verbose and repetitive. Without nice shortcuts and sugaring, configurations can have lots of boilerplate code, hindering project setup and making the config unwieldy as it grows in sophistication and scope.

2. Code reuse across projects

Many customers want better ways to share configuration across projects. CircleCI configuration was not previously easy to share across teams, creating unnecessary redundancy and slowing teams down. Our customers who have large teams want ways to easily disseminate and enforce common patterns and policies and to help their teams improve through shared practices.

3. Easier paths to common configuration

Developers try to avoid reinventing things or re-solving solved problems. When bootstrapping code, they want to use off-the-shelf, honed encapsulations whenever it’s practical and reliable. In CircleCI, starting from a blank page can be daunting for first-time users, and we wanted to make it easier to get from nothing to something useful.