Deploy Rig to AWS Lambda

Rust agents built with Rig compile to a single small binary, which makes them a natural fit for AWS Lambda: low memory usage and fast cold starts. This guide deploys a simple Rig agent — one that calls the OpenAI API and uses no vector store — to Lambda using the cargo lambda CLI, then walks through the resulting cloud metrics.

For reference, we compare the deployed Rig agent against an equivalent Python agent on the same platform. In summary:

• Lower memory footprint (~26MB vs. ~130MB).
• Faster cold starts (~90ms vs. ~1900ms).
• More consistent performance across memory configurations.

Prerequisites

Before we begin building, ensure you have the following:

• A clone of the rig-entertainer-lambda crate (or your own Rig application).
• An AWS account
• An OpenAI API key

AWS Lambda quick overview

You might deploy your Rust application on AWS lambda if it’s a task that can execute in under 15 mins or if your app is a REST API backend.

AWS and Rust

AWS Lambda supports Rust through the use of the OS-only runtime Amazon Linux 2023 (a lambda runtime) in conjunction with the Rust runtime client, a rust crate.

REST API backend

• Use the lambda-http crate (from the runtime client) to write your function’s entrypoint.
• Then, route traffic to your lambda via AWS API services like Api Gateway, App Sync, VPC lattice, etc …
• If your lambda handles multiple endpoints of your API, the crate axum facilitates the routing within the lambda.

Event based task (15 mins max.)

• Your lambda function is invoked by some event with the event passed as the payload. For example, configure your S3 bucket to trigger the lambda function when a new object is added to the bucket. The function will receive the new object in the payload and can further process it.
• Use the lambda_runtime crate with lambda_events (from the runtime client) to write your function’s entrypoint.
• Then, invoke your function either via lambda invoke command or with integrated AWS triggers (ie. S3 UploadObject trigger).

For both cases, the crate tokio must also be added to your project as the lambda runtime client uses tokio to handle asynchronous calls.

The Rig entertainer agent

The crate rig-entertainer-lambda implements a simple Rust program that is executed via the lambda_runtime. It invokes a Rig agent using the OpenAI API to entertain users with jokes. It is an event-based task, executed with the lambda invoke command.

The main takeaway here is that the app’s Cargo.toml file must include the following dependencies:

1. rig (our rig crate)
2. lambda_runtime
3. tokio

Now let’s deploy it

There are many ways to deploy Rust lambdas to AWS. Some out of the box options include the AWS CLI, the cargo lambda CLI, the AWS SAM CLI, the AWS CDK, and more. You can also decide to create a Dockerfile for your app and use that container image in your Lambda function instead. See some useful examples here.

Here we’ll use the cargo lambda CLI to deploy the code in rig-entertainer-lambda from your local machine to an AWS Lambda function:

# Add your AWS credentials to your terminal
# Create an AWS Lambda function named ‘rig-entertainer’ with architecture x86_64.

function_name='rig-entertainer'

cd rig-entertainer-lambda
cargo lambda build --release # Can define different architectures here with --arm64 for example
cargo lambda deploy $function_name # Since the name of the crate is the same as the lambda function name, no need to specify a binary file

Metrics on the cloud

Deployment package

This is the code configuration of the rig-entertainer function in AWS. The function’s code package (bundled code and dependencies required for lambda to run) includes the single rust binary called bootstrap, which is 3.2 MB.

Memory, CPU, and runtime

The image below gives metrics on memory usage and execution time of the function. Each row represents a single execution of the function. In yellow is the total memory used, in red is the amount of memory allocated, and in blue is the runtime. Although the lambda has many configuration options for the memory ranging from 128MB to 1024MB, we can see that the average memory used by our app is 26MB.

Let’s get more information on the metrics above by spamming the function and calculating averages. I invoked rig-entertainer 50 times for each memory configuration of 128MB, 256MB, 512MB, 1024MB using the power tuner tool and the result of those invocations are displayed in the chart below.

The x-axis is the memory allocation, and the y-axis is the average runtime over the 50 executions of rig-entertainer.

Q. We know that the function uses on average only 26MB per execution (which is less than the minimum memory allocation of 128MB) so why should we test higher memory configurations? A. vCPUs are added to the lambda in proportion to memory so adding memory could still affect the performance.

However, we can see that adding memory to the function (and therefore adding computational power) does not affect its performance at all. Since the cost of a lambda execution is calculated in GB-seconds, we get the most efficient lambda for the lowest price!

Cold starts

Cold starts occur when the lambda function’s execution environment needs to be booted up from scratch. This includes setting up the actual compute that the lambda function is running on, and downloading the lambda function code and dependencies in that environment. Cold start latency doesn’t affect all function executions because once the lambda environment has been setup, it will be reused by subsequent executions of the same lambda.

In the lambda cloudwatch logs, if a function execution requires a cold start, we see the Init Duration metric at the end of the execution.

For rig-entertainer, we can see that the average cold start time is 90.9ms: Note that the function was affected by cold starts 9 times out of the 245 times it was executed, so 0.036% of the time.

Comparison with a Python framework

We replicated the OpenAI entertainer agent using a popular Python LLM framework in this mini python app which is also deployed to AWS Lambda in a function called python-entertainer.

Let’s compare the metrics outlined above.

Deployment package

This is the code configuration of the python-entertainer function in AWS. The function’s code package is a zip file including the lambda function code and all dependencies required for the lambda program to run.

Memory, CPU, and runtime

There are varying memory configurations from 128MB, 256MB, 512MB, to 1024MB for the lambda shown in the table below. When 128MB of memory is allocated, on average about 112MB of memory is used, and when more than 128MB is allocated, about 130MB of memory is used and the runtime is lower.

Let’s get some more averages for these metrics: I invoked python-entertainer 50 times for each memory configuration of 128MB, 256MB, 512MB, 1024MB using the power tuner tool and the result of those invocations were plotted in the graph below.

We can see that by increasing the memory allocation (and therefore computation power) of python-entertainer, the function becomes more performant (lower runtime). However, note that since you pay per GB-seconds, a more performant function is more expensive.

Cold starts

For the Python agent, the average cold start time is: 1,898.52ms, ie. 20x as much as the rig app coldstart. Note that the function was affected by cold starts 6 times out of the 202 times it was executed, so 0.029% of the time.

Next steps

• Deploy Rig with LanceDB on AWS Lambda — add a vector store for RAG on Lambda.
• cargo lambda documentation — full CLI reference for building and deploying Rust Lambdas.