Using SPIRE to (Automatically) Deliver TLS Certificates to Envoy For Stronger Authentication

By Andrew Harding

Hello! This is Andrew Harding from Scytale. If you are currently using Envoy to provide secure service-to-service communication, I’d like to show you how to leverage the open-source SPIRE project to dramatically improve your authentication security with automatically delivered and rotated keys and certificates based on multi-factor workload authentication.

While Envoy and SPIRE have both been around for a while now, we’ve recently added support for the Envoy SDS API inside SPIRE, which makes this all much easier to set up. Let’s explore how.

Envoy is a popular open-source service proxy that, among other things, is widely used to provide abstracted, secure, authenticated and encrypted communication between services. Envoy enjoys a rich configuration system that allows for flexible third-party interaction.

One component of this configuration system is the Secret Discovery Service protocol or SDS. Envoy uses SDS to retrieve and maintain updated “secrets” from SDS providers. In the context of authentication, these secrets are the TLS certificates, private keys, and trusted CA certificates Envoy uses to provide secure TLS communication between services.

Short-lived secrets are an important aspect of security, as they reduce the need for revocation list infrastructure, which weakens security and contributes to an increased attack surface. Rotating short lived secrets frequently involves manual auditing and deployment and is generally very burdensome to operators. An SDS provider’s ability to deliver updated secrets to Envoy is a useful step in simplifying secrets management and providing Envoy with up-to-date service identity.

SPIRE (the SPIFFE Runtime Environment) is a toolchain for establishing trust between software systems across a wide variety of platforms. SPIRE supports containerized and elastically scaled environments such as Kubernetes, cloud-hosted infrastructure such as Azure, AWS, and GCP, as well as bare metal deployments on premise. Services in these environments can leverage SPIRE to obtain service identity in the form of X.509 certificates (X509-SVIDs) with the associated private keys, as well as a bundle of trusted CA certificates the service can use to verify other identities.

When a service engages with SPIRE, it undergoes a process called attestation, wherein SPIRE asserts characteristics about the service and its environment. These assertions are matched against operator-defined policy to decide which service identity should be given to the service. (See this video for more details on attestation.) SPIRE automatically rotates the X.509 certificates and keys for the service according to operator-defined policy.

In other words, Envoy can dynamically consume service identity via SDS and SPIRE can provide service identity dynamically. Sounds like a great match!

When Envoy connects to the SDS server the SPIRE Agent attests Envoy and determines which service identities and CA certificates it should make available to Envoy over SDS.

As service identities and CA certificates are rotated, updates are streamed back to Envoy, which can immediately apply them to new connections without interruption or downtime and without the private keys ever having to touch the disk. In other words, SPIRE’s rich methods of defining and attesting services can be used to target the Envoy process, define an identity for it, and provide it with X.509 certificates and trust information that Envoy can use for TLS communication.

A high-level diagram of two Envoy proxies sitting between two services using the SPIRE Agent SDS implementation to obtain secrets for mutually authenticated TLS communication.

Setting up SDS support in SPIRE is as simple as setting the enable_sds = true configuration value in the SPIRE Agent configuration.

Envoy must be configured to communicate with the SPIRE Agent by configuring a cluster that points to the Unix domain socket the SPIRE Agent provides.

For example:

clusters:
- name: spire_agent
connect_timeout: 0.25s
http2_protocol_options: {}
hosts:
- pipe:
path: /tmp/agent.sock

The connect_timeout influences how fast Envoy will be able to respond if the SPIRE Agent is not running when Envoy is started or if the SPIRE Agent is restarted.

To obtain a TLS certificate and private key from SPIRE, you can set up an SDS configuration within a TLS context.

For example:

tls_context:
common_tls_context:
tls_certificate_sds_secret_configs:
- name: "spiffe://example.org/backend"
sds_config:
api_config_source:
api_type: GRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent

The name of the TLS certificate is the SPIFFE ID of the service that Envoy is acting as a proxy for.

Envoy uses trusted CA certificates to verify peer certificates. Validation Contexts provide these trusted CA certificates. SPIRE can provide a validation context per trust domain.

To obtain a validation context for a trust domain, you can configure a validation context within the SDS configuration of a TLS context, setting the name of the validation context to the SPIFFE ID of the trust domain.

For example:

tls_context:
common_tls_context:
validation_context_sds_secret_config:
name: "spiffe://example.org"
sds_config:
api_config_source:
api_type: GRPC
grpc_services:
envoy_grpc:
cluster_name: spire_agent

SPIFFE and SPIRE are focused on facilitating secure authentication as a building block for authorization, not authorization itself, and as such support for authorization-related fields in the validation context (e.g. verify_subject_alt_name) is out of scope. Instead, we recommend you leverage Envoy’s extensive filter framework for performing authorization. Additionally, you can configure Envoy to forward client certificate details to the destination service, allowing it to perform its own authorization steps, for example by using the SPIFFE ID embedded in the URI SAN of the client X.509 SVID.

And that’s how you use SPIRE to improve authentication security in service-to-service communication! Try it out and let us know how it goes! You can reach us on the SPIFFE slack. There are future improvements planned and your experience will be very valuable in shaping Envoy support within SPIRE.

I’d like to acknowledge Scytale’s own Marcos Yacob and Marcos Yedro whose efforts were instrumental in prototyping and developing the SPIRE SDS implementation.