If you use AWS SAM to build and deploy your serverless Ruby functions, it'll take care of building your dependencies on a function by function basis and package them up for deployment. Great!
But if you don't want to use SAM, using gems with native dependencies (e.g. Nokogiri) is a nightmare! If you prefer to travel light (or, maybe, you like coding throwaway functions in the AWS console), you can instead build your dependencies into a Lambda Layer and then use that layer across whatever Lambda functions you like without worrying about dependencies again.
But how? After an hour of trial and error plus a few tips from this neat article (though that one still used SAM), I figured out the magic spell required to locally build Ruby gems and their dependencies in a local Amazon Linux container and turn them into a layer.
Note: This post assumes you're familiar with the
aws client and have the credentials to use Lambda with it. If not, this is too advanced for now. Google the
aws CLI client and how to create an IAM user with the right permissions.
Without further ado, here's the magic:
LAYER_NAME="my-ruby-layer" mkdir $LAYER_NAME && cd $_ bundle init bundle add http --skip-install bundle add nokogiri --skip-install rm Gemfile.lock docker run --rm -v $PWD:/var/layer \ -w /var/layer \ amazon/aws-sam-cli-build-image-ruby2.7 \ bundle install --path=ruby mv ruby/ruby ruby/gems zip -r layer.zip ruby aws lambda publish-layer-version \ --layer-name $LAYER_NAME \ --region eu-west-1 \ --compatible-runtimes ruby2.7 \ --zip-file fileb://layer.zip
This results in a
LayerVersionArn you can use with your Lambda functions. Once you've done this, loading the gems you need in the usual way (e.g.
require 'nokogiri') will Just Work™.
If you use the AWS console, it'll let you pick this from a drop down menu which is how I like to do it:
Here's a quick break down of what the code above does. First, we create a folder with the name of our layer:
LAYER_NAME="my-ruby-layer" mkdir $LAYER_NAME && cd $_
Next, we create a
Gemfile and add some gems to it. If you already have a
Gemfile, copy it in at this point instead of doing this:
bundle init bundle add http --skip-install bundle add nokogiri --skip-install rm Gemfile.lock
Note: I delete Gemfile.lock because the version of Bundler in the Lambda container clashes with mine.
Let's now instruct Docker to run the image Amazon provides for building Ruby 2.7 dependencies and do the
bundle install there (while saving it to our normal filesystem):
docker run --rm -v $PWD:/var/layer \ -w /var/layer \ amazon/aws-sam-cli-build-image-ruby2.7 \ bundle install --path=ruby
A quick directory name tweak is necessary to match the structure that Lambda expects. We can then zip up the
mv ruby/ruby ruby/gems zip -r layer.zip ruby
Finally, publish the layer to AWS Lambda:
aws lambda publish-layer-version \ --layer-name $LAYER_NAME \ --region eu-west-1 \ --compatible-runtimes ruby2.7 \ --zip-file fileb://layer.zip
Make sure to set the region to the one you actually want to use the layer from. Every region has its own layers.
Note: If you want to deploy your layer to the public or use it more broadly across all regions, you'll want to search up the AWS Serverless Application Repository.