This post is part of our complete guide: How to Build a Serverless Application on AWS using Ruby

This post is a complete guide. That means we are going to create an AWS Lambda in the console first and then move onto using AWS SAM on our local machine to package and deploy the Lambda to AWS. If you’d like to skip the console section, use one of the following links to jump ahead. If not, read on!

If you found this post because you are already trying to build an AWS Lambda and you got one the following errors:

• Build Failed
• Error: RubyBundlerBuilder:CopySource - File name too long

Then maybe skip ahead and try reading these sections first: build, package, and deploy your AWS Lambda Function using AWS SAM.

If you are trying to build an AWS Lambda using a native binary and got the error:

• Error loading the ‘mysql2’ Active Record adapter. Missing a gem it depends on?

Then read: Building Gems with Native Binary Packages in AWS Lambda

If none of that applies, then read on!

Creating a new AWS Lambda using the AWS Console

First, we are going to create a function using the console. This will get us familiar with the AWS Lambda Console and the built in code editor Cloud9.

Go to the AWS Lambda Console and click Create a function

If you already have functions then your console will look like this:

Choose Author from scratch and enter the following values:

• Name: Type a name for your function
• Runtime: Choose Ruby 2.5
• Role: Choose Create a new role from one or more templates
• Role name: Type a name for your role
• Click Create function

Scroll down to the Cloud9 Editor and click Test

In the Configure test event dialog box, choose Create new test event

• Name your test event
• Click Create

Click Test again, Cloud9 will use your test event. Note the output in the console:

Response:
{ "statusCode": 200, "body": "\"Hello from Lambda!\""
}

Great! Our lambda works. Its worth noting that the test event passed in some data. Normally you would configure that payload to something that your lambda function expects, but for this example we didn’t need to.

Let’s go ahead now and require ActiveRecord in our lambda… simple right?

Add the require statement to the top of the lambda_function.rb in the Cloud9 editor:

require 'json'
require 'active_record' def lambda_handler(event:, context:) # TODO implement { statusCode: 200, body: JSON.generate('Hello from Lambda!') }
end

Click Save, then click Test and review the output in the Cloud9 console:

A note on saving code in the Cloud9 editor: ⌘ + S and the Save button are not the same thing! If you simply hit ⌘ + S and test you will not have deployed your latest code changes. You need to click the Save button before you test your changes.

Response:
{ "errorMessage": "cannot load such file -- active_record", // response continued...
}

So what this error means is that our lambda function was unable to load the ActiveRecord gem. Makes sense, we didn’t run bundler or anything like that. However, its a crucial concept for you to understand when working with Lambda functions. A Lambda function is a completely self contained thing. It needs to have everything it needs to run when its deployed. Its not a server. It doesn’t go out and grab code or pull anything down. You need to package your lambda with everything it needs at deployment.

Now its time to develop this Lambda on our machine and use the AWS Serverless Application Model to deploy it.

Creating a new AWS Lambda Function using AWS SAM

You just saw the easy way to create an AWS Lambda Function using Ruby… now you are going to see the real way.

What is AWS SAM?

AWS Serverless Application Model (AWS SAM) is an open source framework developed by Amazon that you use to build serverless applications. Its comprised of a template file that outlines what resources your serverless application will use (AWS Lambda for example) and a command line interface (CLI) that you use to package and deploy your serverless apps to AWS.

Installing the AWS SAM CLI using Python

You are going to need the AWS SAM CLI in order to package and deploy code from your computer. If you already have it installed, you can skip this section.

The AWS SAM CLI can be installed using python’s pip package manager. This means you need to have python installed on your mac to use it.

Let’s get python installed using homebrew:

brew install python

When that’s finished, install the AWS SAM CLI using pip (homebrew will have named your executable pip3):

pip3 install aws-sam-cli --upgrade --user

Once you have finished installing the AWS SAM CLI you might notice that if you type:

sam --version

You will get the error sam: command not found.

This is because the aws-sam-cli package was installed to a location that is not being covered by your PATH variable. We will fix that by adding the location to the PATH variable.

Review the list of packages installed with pip3 by typing into your Terminal:

pip3 list

Notice that the package we just installed (aws-sam-cli) is in the list. Determine the location of the package by typing in your Terminal:

pip3 show aws-sam-cli

Using the output, copy the location up to and including the version number only:

# We simplified the home directory path using ~ instead
__~/Library/Python/3.7__

Open your Terminal shell configuration file ~/.bash_profile on macOS for editing using an editor of your choice and add the following line:

# notice we added /bin to the path - where the sam executable lives
export PATH=~/Library/Python/3.7/bin:$PATH

Close and reopen your Terminal, or run the following to update the PATH variable:

source ~/.bash_profile

Test your AWS SAM CLI Installation by typing:

sam --version
#-> outputs version number now!

Congratulations! You installed the AWS SAM CLI!

Optional: Installing the AWS CLI using Python

If you don’t already have the AWS CLI installed then we are also going to need to install the AWS CLI also. The AWS CLI manages credentials our SAM CLI commands will use when they run. The good news is that you’ve done all the hard work of setting up the PATH variable for the previous command, so all you need to do to install the AWS CLI is run the following command in the terminal:

pip3 install awscli --upgrade --user

Once that’s complete, you need to configure your AWS access key and secret using the command:

aws configure

Follow the prompts and provide your access key id and your secret access key, set your default region (we use: us-west-2) and hit ENTER for the output option.

That’s it! you are ready to go.

Writing your first AWS Lambda Function using Ruby

Now that we have AWS SAM CLI installed, we can use it to generate a starting point for our Lambda function. Run the following command in the terminal to generate an application directory for our app:

sam init --runtime ruby2.5 —-name weapons

This command generates a log of boilerplate, but here’s the most important stuff to pay attention to right now:

template.yaml	declares resources that comprise your serverless application
hello_world/app.rb	Your Lambda function!
hello_world/Gemfile.rb	Gemfile for dependencies
README.md	Sample app documentation

The README.md file documents the commands we are going to use next to package and deploy this lambda, so if you use sam init for future projects, you can use that documentation if you forget the commands.

This command also created a hello world function. Normally we would remove this and replace it with our own code, but we are going to use it in this guide to keep things simpler.

Let’s smoke test this and make sure we can package and deploy.

In order to deploy the Lambda you will need an S3 bucket.

If you have the AWS CLI installed (steps are same as AWS SAM CLI above - just install awscli instead), you can run the following command:

aws s3 mb s3://your-bucketname

Or you can create a bucket using the S3 Console:

Go to the S3 Console and click Create bucket

• Name your bucket and click Next
• Configure Options: Click Next
• Set Permissions: Click Next
• Review: Click Create bucket

build, package, and deploy your AWS Lambda Function using AWS SAM

Remember these commands build, package, deploy. You will run them in that order in order to deploy your Lambda function to AWS.

build	creates your deployment artifacts in .aws-sam/build directory and bundles gems!
package	zips the contents of your build directory and deploys to your S3 bucket
deploy	creates a cloudformation stack based on your template and provisions it

First, lets build the deployment artifacts by running the command:

sam build

Make sure you are running these commands from inside the directory you created using the init command.

OPTIONAL: Some developers have said they get the error:

Build Failed
Error: RubyBundlerBuilder:CopySource - File name too long

Its a bit misleading, but basically what it’s telling you is that it tried to create deployment artifacts out of your root directory. So, try explicitly declaring your source directory with --base-dir.

# tell the command where your source code is if you get that error
sam build --base-dir hello_world

At this point you should have successfully built the project using the build command.

You should take a moment and look at the deployment artifacts by viewing the .aws-sam/build directory. The most important thing to note for Ruby developers is that it bundled your gems into the directory. This is because a Lambda has to have everything it needs to run.

Now you package your application and deploy it to S3 using the package command:

sam package \ --output-template-file packaged.yaml \ --s3-bucket your-bucketname

This will zip up your lambda function and upload it into the S3 bucket named in the command parameters.

Finally, run the deploy command:

sam deploy \ --template-file packaged.yaml \ --stack-name hello-world-app \ --capabilities CAPABILITY_IAM \ --region us-west-2

Now return to the AWS Lambda Console and test your deployed function like before. You will see the output:

Response:
{ "statusCode": 200, "body": "{\"message\":\"Hello World!\"}"
}

Also notice that the Gemfile for this sample app declared httparty and bundled it into a vendors directory inside the Lambda function! This is exactly what we need to do to include ActiveRecord, so let’s go ahead and add that to the Gemfile for this sample application:

# hello_world/Gemfile source "https://rubygems.org" gem "httparty"
gem "activerecord"

Run a build command like before and make sure your ActiveRecord gem gets bundled to the build directory.

One thing you should know is that if you made a typo in your Gemfile and the bundle command fails - the SAM CLI gives you this completely unhelpful Build Failed error. Try it, change your activerecord gem declaration to active_record. See how unhelpful that is!

So if you ever see that error, a good troubleshooting technique is to cd into your source code directory (in this case hello_world) and run bundle yourself. You will get a much more helpful error message.

Now let’s add an ActiveRecord call to our Lambda function.

Get Ready to Level Up

In order to add ActiveRecord to this Lambda and successfully query the Aurora Serverless Cluster we are going to need to bundle the mysql2 gem in the Lambda. Sounds simple right? Its not. Its actually a bit tricky because the mysql2 gem requires a mysql client library to be installed in order for it to work. Similar to how nokogiri needs libxml. So we are going to have to build them gem against the linux distribution used by AWS Lambda AND include the mysql client library required by the gem in the Lambda deployment files.

Deep breath. Let’s get started…

Adding the ActiveRecord code to our Lambda is the easiest thing we will do.

Go ahead and add a reference to the mysql2 gem to the Gemfile:

# hello_world/Gemfile source "https://rubygems.org" gem "httparty"
gem "activerecord"
gem "mysql2"

Let’s make this example feel a bit familiar and add a database.yml file to the source code directory hello_world:

# hello_world/database.yml adapter: mysql2
database: weapons
username: your-db-master-username
password: your-db-master-password
host: your-db-cluster-name.cluster-id.region.rds.amazonaws.com
port: 3306
connect_timeout: 5

Nothing to out of the ordinary there, just fill in the username, password, and host (cluster endpoint). The only thing to note is the connect_timeout. That’s helpful to declare because if your Lambda can’t connect to the cluster it will just hang.

Let’s add a boilerplate model to represent a weapon in our Weapons database by adding a weapon.rb file to the source code directory:

# hello_world/weapon.rb class Weapon < ActiveRecord::Base
end

Now let’s add some ActiveRecord calls to the function handler:

# hello_world/app.rb require 'json'
require 'active_record'
require 'weapon' def lambda_handler(event:, context:) config = YAML::load(File.open('database.yml')) mysqldb_config = config.merge(database: 'mysql') begin ActiveRecord::Base.establish_connection(mysqldb_config) ActiveRecord::Base.connection.create_database(config["database"]) puts 'database created!' rescue => e puts e.message end ActiveRecord::Base.establish_connection(config) unless ActiveRecord::Base.connection.table_exists?(:weapons) ActiveRecord::Base.connection.create_table :weapons do |t| # :id is created automatically t.string :name end puts 'table created!' end @weapon = Weapon.new @weapon.name = 'throwing star' @weapon.save { statusCode: 200, body: { message: "There are #{Weapon.count} weapons in the database.", # location: response.body }.to_json }
end

Here’s the breakdown of this function handler:

1. Load the database configuration
2. Create a mysqldb configuration, which is required to create a database
3. Wrap the create_database call in a rescue block so we can run the lambda repeatedly
4. Re-establish our connection using the original database config
5. create_table if it doesn’t exist, again to allow the lambda to run repeatedly
6. Instantiate a Weapon model, set the name and call save. ActiveRecord!
7. Output the count of Weapons in the database, this will increment each time you run the function

Ok so now that’s done. Its time to build the gem against a Lambda container and get a copy of the mysql library we need into our deployment package.

Building Gems with Native Binary Packages in AWS Lambda

So if you went ahead and tried to build your Lambda now you would get the following error:

Error loading the ‘mysql2’ Active Record adapter. Missing a gem it depends on?

The problem is the mysql2 gem needs to be built against the linux environment it will be running on AND it needs to use a mysql package at runtime. So we are going to need docker.

Installing Docker using Homebrew

What is Docker is way out of scope for this post… but for the AWS Lambda developer, Docker allows you to run the same container on your machine that AWS will use when your Lambda is executed. This is really useful because not only can you build gems against binaries like we are going to, but you can also locally test your Lambdas without deploying them!

Installing Docker with Homebrew is relatively easy, simply run the following command in the Terminal:

brew cask install docker

This command will take some time, near the end it will prompt for your password so it can add the Docker App to your Applications directory (macOS).

When the command completes, locate the Docker App in your Applications directory, double-click it and kick off the setup. Ignore the Docker Hub username and password, you don’t need an account to use the app or download containers.

Once that’s complete you will have an icon in your menu bar and a docker command available in the Terminal!

I know we are flying through a Docker installation here, but it could be an entirely other guide. Also, you will likely only use it when you either compile native binaries or test your lambdas locally. Neither of which will require you to be a Docker expert.

Now let’s use Docker to build our gem and copy the library we need into our source code directory.

Let’s dive right in by running a command in the Terminal. Make sure your current working directory is the source code directory hello_world:

docker run -v "$PWD":/var/task -it lambci/lambda:build-ruby2.5 /bin/bash \
-c "yum -y install mysql-devel ; bundle install --deployment ; bash"

When you run this for the first time, Docker will pull down the lambci/lambda container from Docker Hub. This will only happen the first time. Let’s breakdown what this command is doing:

First, the -v argument is telling our run command to mount the present working directory ($PWD) at /var/task in the container. AWS Lambda’s execute from /var/task. Its called: LAMBDA_TASK_ROOT.

Next we pass the argument -it, this is actually two arguments and basically that’s going to drop us into the container at a command prompt.

The lambci/lambda:build-ruby2.5 argument is telling the run command which container to use. The build-ruby2.5 is unique to the container image and is documented on its Docker Hub page.

Finally we are running a bash command that basically installs the mysql-devel package our mysql2 gem needs and then doing a vendor directory deployment of the gems.

After running that command you should be staring at a bash command prompt inside your container. The mysql-devel package is now installed and you should notice that a vendor directory has been created in your source code directory from the bundler command. All that’s left for us to do is simply copy the mysql library into our source code directory.

You might be wondering now how you would determine what file you even need to copy… good question! In order to determine what library this gem needed, I simply deployed the lambda with the mysql2 gem now compiled against the container. The error it gave basically told me what it was looking for:

libmysqlclient.so.18: cannot open shared object file

So there we go. We need the file: libmysqlclient.so.18. How do we find it?

We can run the following command in the container prompt:

/sbin/ldconfig -p | grep mysql | cut -d\> -f2

The output of this command will give the you full path to the libmysqlclient.so.18 shared object file. Hooray!

Now remember how we mounted our source code directory at /var/task in the container? If you take a look at the documentation for a Lambda Execution Environment, you will see there is a variable named LD_LIBRARY_PATH. Looking at the PATH defined for this variable, you will see $LAMBDA_TASK_ROOT/lib. This means, that if we drop our shared object file into a folder called lib in our source code directory (which will mount as /var/task when your lambda executes) then that will be found when the lambda is executing and the mysql2 gem goes looking for a mysql library.

So let’s simply create a directory called lib and copy our shared object file in there:

mkdir lib
cp /usr/lib64/mysql/libmysqlclient.so.18 lib/

We are in the home stretch!

Make sure the folder and file shows up in your local directory, then type exit to leave the bash prompt and return to your Terminal.

All that is left to do now is build, package, and deploy this Lambda.

A quick note on the build command… since we have a gem that needs to be compiled against the container, we are going to need to pass the --use-container argument along with our build command in order to ensure we don’t compile the mysql2 gem on our machine and then deploy that (you can see all the sam cli commands here). Here’s the new build command:

sam build --base-dir hello_world --use-container

Packaging is the same as before:

sam package \ --output-template-file packaged.yaml \ --s3-bucket your-bucketname

And then deploy:

sam deploy \ --template-file packaged.yaml \ --stack-name hello-world-app \ --capabilities CAPABILITY_IAM \ --region us-west-2

Ok, there is literally only one last thing to do before we run our Lambda. Because we are using Aurora Serverless we need to make sure our Lambda is running in the same VPC as the Aurora Serverless Cluster. Otherwise it won’t be able to connect.

Set the Lambda VPC

• Go to the RDS Console
• Choose Databases
• Click on your Cluster
• Review the Network configuration, identify the VPC id

Now, go to the AWS Lambda Console and choose your Lambda function.

Scroll down to the Network configuration:

• Select the same VPC as your Aurora Serverless Cluster
• Choose at least one Subnet
• Choose the same security group as the Aurora Serverless Cluster
• Scroll to the top of the page and click Save

The moment is finally here! Click Test and run your Lambda. You should see the message we defined in our handler code! Congratulations!

In our upcoming post we will cover creating a Single Page Application in S3 to use this Lambda. You don’t want to miss this series, so sign up for our newsletter and follow us on Twitter!