How to Create a AWS Lambda Layer of Your Gemfile / Ruby Gem Dependencies


If you use AWS SAM to build and deploy your serverless Ruby functions, it'll take care of building your dependencies on a function by function basis and package them up for deployment. Great!

But if you don't want to use SAM, using gems with native dependencies (e.g. Nokogiri) is a nightmare! If you prefer to travel light (or, maybe, you like coding throwaway functions in the AWS console), you can instead build your dependencies into a Lambda Layer and then use that layer across whatever Lambda functions you like without worrying about dependencies again.

But how? After an hour of trial and error plus a few tips from this neat article (though that one still used SAM), I figured out the magic spell required to locally build Ruby gems and their dependencies in a local Amazon Linux container and turn them into a layer.

Note: This post assumes you're familiar with the aws client and have the credentials to use Lambda with it. If not, this is too advanced for now. Google the aws CLI client and how to create an IAM user with the right permissions.

The code

Without further ado, here's the magic:

LAYER_NAME="my-ruby-layer"
mkdir $LAYER_NAME && cd $_ bundle init
bundle add http --skip-install
bundle add nokogiri --skip-install
rm Gemfile.lock docker run --rm -v $PWD:/var/layer \ -w /var/layer \ amazon/aws-sam-cli-build-image-ruby2.7 \ bundle install --path=ruby mv ruby/ruby ruby/gems
zip -r layer.zip ruby aws lambda publish-layer-version \ --layer-name $LAYER_NAME \ --region eu-west-1 \ --compatible-runtimes ruby2.7 \ --zip-file fileb://layer.zip

This results in a LayerVersionArn you can use with your Lambda functions. Once you've done this, loading the gems you need in the usual way (e.g. require 'nokogiri') will Just Work™.

If you use the AWS console, it'll let you pick this from a drop down menu which is how I like to do it:

Picking a Lambda layer in the AWS console

How the code works

Here's a quick break down of what the code above does. First, we create a folder with the name of our layer:

LAYER_NAME="my-ruby-layer"
mkdir $LAYER_NAME && cd $_

Next, we create a Gemfile and add some gems to it. If you already have a Gemfile, copy it in at this point instead of doing this:

bundle init
bundle add http --skip-install
bundle add nokogiri --skip-install
rm Gemfile.lock

Note: I delete Gemfile.lock because the version of Bundler in the Lambda container clashes with mine.

Let's now instruct Docker to run the image Amazon provides for building Ruby 2.7 dependencies and do the bundle install there (while saving it to our normal filesystem):

docker run --rm -v $PWD:/var/layer \ -w /var/layer \ amazon/aws-sam-cli-build-image-ruby2.7 \ bundle install --path=ruby

A quick directory name tweak is necessary to match the structure that Lambda expects. We can then zip up the ruby folder:

mv ruby/ruby ruby/gems
zip -r layer.zip ruby

Finally, publish the layer to AWS Lambda:

aws lambda publish-layer-version \ --layer-name $LAYER_NAME \ --region eu-west-1 \ --compatible-runtimes ruby2.7 \ --zip-file fileb://layer.zip

Make sure to set the region to the one you actually want to use the layer from. Every region has its own layers.

Note: If you want to deploy your layer to the public or use it more broadly across all regions, you'll want to search up the AWS Serverless Application Repository.