Flexport’s main backend service is a Ruby on Rails monolith. In the company’s early days, Rails helped us move quickly. However, like many other fast-growing startups, we’ve found it challenging to manage complexity with Rails as the team has grown.
Conveniences that once improved productivity now make it hard to understand what’s going on: many two-way model associations, arbitrary reads and writes through ActiveRecord, a global
app/ directory structure, implicit behavior, etc.
To help untangle this complexity, we’ve started using Rails Engines. Rails Engines are modules within a Rails app that give projects their own separate directories and namespaces. However, the modularity they provide out of the box is mostly cosmetic — nothing prevents engineers from reaching under the hood to directly access an engine’s internals.
To enhance this default behavior, Flexport has developed a few techniques to more strictly enforce engine isolation internally. This blog post explains our approach, including three new engine-related RuboCop cops that we are open sourcing to share with the broader community.
What is Rails Engine isolation?
Isolating a Rails Engine means preventing arbitrary reads and writes to the engine by code outside the engine, and vice versa.
Let’s start with a quick example in code. Imagine we have two engines in addition to the default
app/: ocean and trucking. Our directory structure might look like the following:
Notice the separate Rails subdirectories within each of these three for models, controllers, services, etc.
An engineer on the ocean team wants to know when a shipping container has arrived at its destination port. Within the ocean engine, they add a handler:
Later, the trucking team decides that they also want to know when this happens. A trucking engineer adds some code to the ocean handler to reach into the trucking engine to update its models:
This approach is convenient, but it violates separation of concerns across the engine boundaries. The ocean engine now knows about trucking’s internal models and business processes. And because the ocean engine has a reference to the ActiveRecord model, ocean code may set trucking fields however it wants:
Enforcing engine isolation means programmatically preventing this kind of reaching across engine boundaries.
Benefits of enforcing engine isolation
Our primary goal with engines is separation of concerns: high cohesion within engines and loose coupling across engines. During development, engine isolation enforcement guides us towards these principles even when it might be more convenient to violate them to quickly ship a feature. Tactically, there are several benefits of separating engines:
Prevent arbitrary writes
When an ActiveRecord model is accessed directly, anyone can write to the model in arbitrary ways with
.save from anywhere in the codebase. This makes it hard for teams to centralize write paths, which makes the code harder to reason about:
- A single business process like “change the trucking carrier for this delivery” might be fragmented across validations, callbacks, and external code.
- Models themselves must contain any important cross-model validation. This can have performance implications when validations load associations, sometimes leading to N+1 queries problems.
- Side effects such as sending emails, syncing to third-party systems, emitting events, etc are triggered from several sources and can be hard to debug.
Prevent arbitrary reads
When an ActiveRecord model is accessed directly, other models can be loaded indiscriminately via associations. This means:
- Teams are unsure how other parts of the codebase use their models. This makes it hard to identify clear interfaces and boundaries of ownership, so refactoring is hard, and evolving the product is hard.
- When reading from another team’s section of the data model, engineers need to build and maintain their own
includesdefinition to avoid N+1s. If the other team’s data model changes, the includes must be updated as well. It’s hard to keep these in sync.
Out-of-the-box modularity with isolate_namespace
By default, Rails offers a small degree of engine isolation. There is a method called
isolate_namespace, used like this:
The Rails Engine docs explain that
isolate_namespace is responsible for isolating the controllers, models, routes and other code into the engine namespace, away from similar components inside the main
This means is that in order for
OtherEngine to access
MyModel defined within engine
MyEngine, it needs to use
MyEngine::MyModel instead of the un-namespaced
MyModel. The services and models are still accessible, so any part of the codebase can do arbitrary reads and writes.
While a step in the right direction, this is mostly cosmetic isolation in our experience.
Rail Engine isolation enforcement cops
To extend the default Rails Engine isolation behavior, we’ve written three new RuboCop cops. We love RuboCop — we’ve created 30+ custom cops internally, including a few that we upstreamed earlier this year. Violations of cops cause failures in local pre-commit hooks and in our continuous integration pipeline.
There are two main types of protection needed to isolate a Rails Engine:
- Inbound access: code outside the engine arbitrarily reaching into the engine
- Outbound access: code inside the engine arbitrarily reaching out to other code
If all of our code was in protected engines, then #1 would be sufficient. But we have a large existing
app/ directory to contend with as well, so engine authors generally need to be wary of both directions of coupling. Our cops restrict these two forms of access and encourage the use of engines instead of the main
NewGlobalModel: encouraging engines
The Flexport/NewGlobalModel cop registers a violation when a new model is added to the main
app/models directory. By convention, we call models in this directory “global” models, and we say they are located in the “main app.” Instead of adding models to the main app, engineers are encouraged to add new models in Rails Engines.
GlobalModelAccessFromEngine: limiting outbound access
The Flexport/GlobalModelAccessFromEngine cop prevents code within an engine from directly accessing models in the main app. Consider this violation:
Instead of direct model access, the best practice is to add business-process-centric service classes to what we call the “main app engine API” — a set of files defined in an
engine_api/ directory within
app/. Then in the engine we have a clean interface to the main app:
The use of
MainApp::EngineApi is an idiom we’ve converged upon internally — it is not enforced by the cop. Engine code with this cop enabled can technically access any non-model code in the main app, which is more lenient than the cop protecting against inbound access covered in the next section.
The GlobalModelAccessFromEngine cop inspects associations too:
When an engine model directly associates with a global model, then association walking can make it easy to inadvertently couple modules that should be separate.
We tend to treat data modeling for engines as if we were data modeling for network-isolated services: it’s natural to hold foreign IDs referencing models across engine boundaries, but not to hold strictly enforced foreign database keys or use ORM associations that hide the separation of concerns in the underlying modules.
EngineApiBoundary: limiting inbound access
The Flexport/EngineApiBoundary cop warns when an engine namespace appears in a file located outside the engine directory. Here’s an example protecting
This cop works on model associations as well.
Rails Engine public Ruby APIs
Often, of course, engines do need to communicate to one another in some way. The cop allows engine authors to define an API surface that code outside the engine can use to interact with the engine.
The APIs are somewhat similar to the network APIs a microservice would expose, but engine API calls are typically just synchronous Ruby method invocations. Here’s how
OtherEngine might use
MyEngine in an acceptable way:
Engine authors define the API to their engine in an
api/ directory within their engine. It can be defined in two ways:
- Add files to
api/. Code defined in these files will be accessible outside your engine. For example, adding
api/foo.rbwill allow code outside the engine to invoke eg
- Create a
api/. Modules listed in this file are accessible to code outside the engine. The file must have this name and a particular format:
Files outside the engine are allowed to access modules that begin with any of the whitelisted prefixes, in addition to any code defined directly in the
Engine API best practices
Though not currently enforced, engineers are encouraged to use plain-old Ruby objects or Dry::Struct values instead of ActiveRecords as the API exchange value between engines. If one engine gets a reference to an ActiveRecord object for a model in another engine, it will be able to perform arbitrary reads and writes via associations and
Engineers are also encouraged to use Sorbet signature to type their APIs. We’ve considered writing a cop that ensures (1) that all engine API files have Sorbet signatures and (2) that these signatures do not include ActiveRecord model types.
In addition to the API, the cop also allows engine authors to define a list of “legacy dependent” files. This is a backlog of files that are allowed to do direct access to an engine “under the hood” for whatever reason. We’ve found it super useful while migrating existing code into engines. You can enable the cop with a bunch of legacy dependents and then slowly refactor to isolate.
Use in practice
We’ve been using the engine isolation cops successfully since early 2019. The codebase has 40 engines now, representing 35% of our Ruby code (comparing
cloc app to
cloc engines/**/app). The cops have proven valuable both when refactoring existing code and starting new projects.
When incrementally isolating code according to a bounded context in our domain model, we follow this process:
- Create an empty new engine with the cops disabled.
- Move the code into this engine.
- Attempt to enable the cops on the engine.
- See where RuboCop violations exist to discover dependencies.
- For each inbound violation, pick one: move the file into the engine, expose engine API surface to support the file’s use case, or add the file to the
- For each outbound violation, pick one: remove the dependency, create main app engine API surface area.
We’ve also found the cops useful when creating greenfield engines. With both cops enabled from the beginning, a new engine’s data is modularized from the rest of the system. This allows projects to get up and running quickly in the monolith and then — if needed — later fork out into a separate network-isolated service with lower effort.
Analysis of cops
Like any tool, the cops have strengths and limitations.
- Boundary violations are detected from static analysis rather than runtime, which makes for a quick feedback loop.
- Teams can isolate at their own pace, adopting incrementally.
- For greenfield engines that enable both cops from the start, isolation is quite strong.
- Anecdotally, engineers say that cops lead to better designs by encouraging an interface-first approach to development.
- The “engine API” definition is coarse — nothing (yet!) prevents engines from returning ActiveRecord models.
- Legacy dependents are coarse — you can add a new direct engine access from an existing legacy dependent file without getting a new warning.
- The cops can be circumvented with raw SQL access, metaprogramming, or RuboCop disabling.
Alternative isolation mechanisms
In addition to the new RuboCop cops, we’ve explored several other ways to enforce modularity within the monolith.
Read-only active record
Flexport engineer Kevin Miller has internally proposed extending Rails models with the following:
.api_association: does the same as the existing
.readonlybut also enforces it on any chained associations such that
User.all.api_association.last.company.readonly? == true.This would enforce all writes can only be done through service APIs.
.with_whitelisted_methods: makes all methods on the underlying model error except for those whitelisted. Allows exposing specific methods/columns without all the cruft of the underlying model.
Only load explicitly defined dependencies during tests
The startup Root’s excellent blog post The Modular Monolith: Rails Architecture outlines how they enforce modular isolation among their engines: only loading certain engines during tests.
Let’s say we have three engines: A, B, and C. A depends on B, and B depends on C. When we run our tests for engine C, we only load engine C; we do not load A or B. This ensures that code in C cannot use any code in A or B. When we run the tests for B, we load C (since B depends on C), but we do not load A. When we run the test suite for A, we load B and C.
This seems like it could work well for greenfield engines, but less so for incrementally modularizing an existing codebase.
ActiveRecord save hook
Add a hook in the ApplicationRecord class to block
save! calls from outside the engine that the model is defined in.
Association loader hook
Hook into ActiveRecord’s association loader to block loading associations across engine boundaries. This seems equivalent to removing all associations according to the cops above.
Deploy engines as separate app instances and have them only communicate over network boundaries. This is something we’re starting to do more.
Instrument method calls at runtime
aftersave hooks and metaprogramming to create something like a backlog list for each engine (or each model) that shows the percentage of in-engine and out-of-engine saves or commits in production. Once that number is small enough, then use Sentry warnings with full stack traces.
Aside: open source philosophy
A quick note about our open sourcing philosophy and the status of the cops mentioned in this post:
Flexport prefers to upstream into existing community-backed repositories whenever possible. However, sometimes existing repos aren’t a good match, and makes more sense for us to host the code ourselves.
Based on a discussion with the RuboCop team, that’s the case for these Rails Engine isolation cops. So we’ve started a new repo to house these and other internally developed cops that we suspect might be useful to others but don’t have another upstream home:
Rails Engines have proven to be a useful tool for managing complexity in our monolith. As the company continues to grow, our attention is shifting towards network-isolated backend services with more robust interface-first APIs defined in protobuf. We’ll be using engines as a migration path to fork out some code into new services, and we expect to continue using engines both during the migration and for internal modularity within new services over the long term.
We’re curious to hear from others’ experience in this area as well. Let us know what you think!