Cutting costs on AI inference using AWS Lambda Containers

AWS Lambda has long been synonymous with small, lightweight, event-driven functions.  For years, the advice has been clear: steer clear of Lambda for containerised workloads due to slow cold starts, opting instead for more traditional services like Amazon ECS (Elastic Container Services).

However, some dramatic advancements in AWS Lambda’s container loading capabilities have changed the game. It’s now a genuinely viable, cost-effective solution for more demanding tasks like AI inference.

The biggest hurdle?

Containers are typically much bigger than traditional lambda functions, meaning a lot more data to transmit. The team at AWS came up with a cunning strategy for handling this, improving cold starts by up to 15 times.

What’s the secret sauce?

It turns out that most containers aren’t as unique as we might think. Many share common foundations, such as Alpine or AWS base images, with some libraries added on top. In fact, only about 1% of each container is truly unique! By breaking containers down into chunks, AWS achieves massive deduplication. Think of it like building with LEGOs. Many projects use the same basic bricks, and you only need to store those bricks once.

The chunks are stored in S3 with multi-tier caching in data centers and local workers. This caching strategy significantly increases the probability that, even during a cold start, the vast majority of the necessary data is immediately available, leading to much faster function initialisation.

Crucially, it turns out that, on average, less than 10% of a container’s data is actually needed at startup. This enables on-demand loading, where AWS Lambda only fetches the specific data required at that moment, dramatically accelerating cold starts for large container images (can be up to 10GB).

Curious about the nitty-gritty? Here’s how AWS pulls it off:

Deterministic Serialisation and Chunking

When you create or update a function, your container images (typically composed of multiple layers) are flattened into a single `ext4` filesystem for consistent processing. This flattened filesystem is then divided into fixed-size 512KiB chunks. This process ensures that identical files or even parts of files consistently always generate the same chunks. Consistency is fundamental, enabling powerful caching and massive deduplication.

Convergent Encryption for Secure Deduplication

AWS Lambda uses a sophisticated technique called Convergent Encryption. This allows them to securely deduplicate and share container image chunks across all customers without compromising security. Each 512KiB chunk is hashed (using SHA256) to create a unique encryption key, which then encrypts the chunk. A manifest file, listing the hash and file offset of each chunk, has its key list encrypted with a unique, per-customer key managed by AWS KMS. What does this mean for you? Identical chunks can be stored and cached just once, enabling safe sharing even if they belong to different customers. Plus, a varying “salt” in the key derivation adds an extra layer of protection, limiting the “blast radius” if highly popular chunks ever encounter issues, by creating different keys for otherwise identical chunks.

The Cost-Cutting Edge

Container-based Lambdas can now actually offer quicker cold starts than zip-based Lambdas, with performance tests suggesting that this occurs in the 50MB-100MB range.

Lambda’s per-millisecond billing model is ideal for sporadic workloads with long idle periods. One report highlights a 95% cost reduction by migrating an AI Inference solution with very low traffic from ECS to Lambda. 

Want to learn more? Dive into the details with the official On-demand Container Loading in AWS Lambda white paper and start rethinking your architecture.