AWS Lambda gains GPU support with a model shared across 48 simultaneous invocations

- Reading
- 6 min
- Code
- 2
AWS integrated G-series GPU instances into Lambda Managed Instances, letting up to 48 invocations share the same copy of a model in the card's memory. Early independent tests show real performance gains, but also friction around vCPU quotas and Terraform support.
What AWS launched
AWS added GPU support to AWS Lambda through Managed Instances, the execution model that lets the function run on EC2 capacity chosen and paid for by the customer, while Lambda handles provisioning, patching, and scaling. The feature is generally available (GA) in commercial regions where both Managed Instances and G-series EC2 instances already exist, according to InfoQ.
Managed Instances has existed since December 2025, but until now it didn't include GPU, and that was the gap that weighed most heavily against the model for anyone who wanted to run AI inference without managing a server. Now Event Functions can run on G4dn, G5, G6, G6f, Gr6, Gr6f, and G6e instances, published as container images and triggered by the same event triggers as any other Lambda function.
Pricing follows the On-Demand rate of the chosen EC2 instance, plus a 9.75% management fee and the US$0.20 per million requests that already apply to traditional Lambda. That's a lower fee than the 15% charged on CPU instances in the same Managed Instances model.
One copy of the model, 48 calls at the same time
The central change for anyone running AI in production is GPU memory sharing across simultaneous invocations. A single execution environment loads one copy of the model onto the GPU and serves up to 48 concurrent invocations sharing that same copy, without reloading weights on every call. Scaling follows GPU utilization, CPU utilization, and concurrency.
In the function's configuration, the AcceleratorConfig block sits alongside memory, timeout, and storage:
"MemorySize": 16384,
"Timeout": 900,
"EphemeralStorage": { "Size": 32768 },
"Environment": { "Variables": { "MODEL_ID": "Qwen/Qwen2.5-1.5B-Instruct" } },
"CapacityProviderConfig": {
"LambdaManagedInstancesCapacityProviderConfig": {
"CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp",
"ExecutionEnvironmentMemoryGiBPerVCpu": 4,
"PerExecutionEnvironmentMaxConcurrency": 4
},
"AcceleratorConfig": { "AcceleratorMemorySize": 16 }
}AWS distributes five GPU base images: one with just CUDA, another with CuPy, and three specific to inference, PyTorch, vLLM, and SGLang. The latter two don't require handler code: you just point the MODEL_ID variable to a repository on Hugging Face, on ModelScope, an S3 prefix, or a local path. Swapping models in production becomes a configuration change followed by a new version:
aws lambda update-function-configuration \
--function-name my-gpu-function \
--environment 'Variables={MODEL_ID=s3://amzn-s3-demo-bucket/models/Qwen2.5-1.5B-Instruct/,VLLM_LOAD_FORMAT=runai_streamer}'Publishing a version already provisions the environments, downloads the weights, and warms up the inference engine before the first event arrives. Lazy loading via Seekable OCI, according to AWS, can cut environment start time by up to 70%.
Daniel Abib, senior solutions architect at AWS, sums up what disappears from the infrastructure team's task list:
You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration. There are no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate.
Daniel Abib, senior specialist solutions architect at AWS
Launch limits: container images up to 32 GB, 32 GB of ephemeral storage, 2 to 32 GB of memory, 3 to 48 GB of accelerator memory, 15,000 vCPUs per capacity provider, and timeouts of up to 90 minutes for asynchronous invocations and per event source mapping.
The first tests found friction before they found speed
The technical community tested the feature on the same day it launched. Akaike, from Japanese consultancy Classmethod, tested every accelerated instance type available in us-east-1: of 81 types, 37 were accepted, all from the G family. P instances, G7, Inferentia, Trainium, and the Graviton-based g5g were rejected, as were zip packages: only container images work.
A PyTorch benchmark ran about 66 times faster on the L4 instance than on the same instance's CPU, according to Akaike's test. But the tool ran into two maturity obstacles: neither Terraform nor the Cloud Control provider yet recognizes the AcceleratorConfig field, and the GPU vCPU quota comes at zero by default on a new account.
if you use g6.xlarge (4 vCPU) with the default three instances, you need a quota of 12 vCPUs.
Akaike, Classmethod
An issue and a pull request in the Terraform provider, opened on November 16 by a user identified as sasidhar-aws, remain open, a sign that infrastructure-as-code support hasn't arrived yet.
One small model, 417 calls, zero failures
The second test, done by an AWS Community Builder who goes by xbill, ran the Gemma 4 E2B model on a single g6.2xlarge through the vLLM image, with Google's 4-bit (QAT) weights and a speculative decoding drafter bundled into a 9 GB image.
A single environment served all 48 simultaneous requests, and all 417 calls in the test succeeded. Throughput was about 130 tokens per second for a single request and between 3,400 and 3,600 tokens per second with the 48 running in parallel.
The friction wasn't in the GPU, but in the publishing pipeline: 484 seconds between publishing the function and it having an active version, a Docker image index that Lambda rejected until it was sent as a single-platform manifest, and a handler that only returns complete responses, which made it impossible to measure time to first token.
xbill's verdict is restricted to what he tested:
For a model behind events, queues or other AWS services, the Lambda GPU function is the simpler road: one image, one function, and Lambda keeps the server alive and routes the requests. Keep the drafter on for a chat with a few users at a time, and publish without it for heavy batch load.
xbill, AWS Community Builder
When the response needs to be streamed or the team wants full control of the server, he recommends running vLLM directly on an EC2 G6, on its own, without going through Lambda.
What it still doesn't solve
Managed Instances remains restricted to Event Functions. The model doesn't extend to Web Functions, the HTTP server format that Lambda gained this year, and AWS hasn't said whether it will extend GPU support there.
Yan Cui's analysis of the Managed Instances model, written in December 2025, when it didn't yet support GPU, remains valid for the GPU case without change:
Regular Lambda functions can scale rapidly by tapping into a large, shared pool of EC2 instances. With managed instances, it takes tens of seconds to launch new EC2 instances. As such, it's not a good fit when you have large, unpredictable spikes in traffic.
Yan Cui
For anyone running AI inference, what changes in practice
Before this change, running GPU inference in production meant choosing between managing a cluster (GPU driver, CUDA version, custom autoscaling) or using a GPU serverless service outside the AWS perimeter, without the same native event triggers, identity (IAM), and networking in the account, as Abib points out when comparing it with the competition.
With GPU on Managed Instances, the path becomes more similar to the rest of Lambda: publish an image, declare AcceleratorConfig, and let the service scale based on GPU utilization within the same account, with the same triggers (S3, SQS, EventBridge) that already trigger any function today.
This matters mainly for event-driven workloads: processing an image upload, responding to a queue message, generating a summary when a document arrives in S3. For chat services with unpredictable traffic and a need for streamed responses, both first-hand tests and Yan Cui's analysis agree on one point: the model still has cold-start and scaling friction that a dedicated EC2 doesn't have.
Before putting it into production, it's worth checking three things: the account's GPU vCPU quota, which starts at zero; whether the CI/CD pipeline accepts single-platform image manifests; and the publish-to-active time for the SLA the application already promises, since xbill's 484 seconds in testing aren't instant.
Translated from the Brazilian Portuguese original · Read the original
Anthropic debuts on Nasdaq valued at $1.5 trillion
The company behind Claude began trading on the Nasdaq under the ticker ANT on November 19, and the IPO filing reveals AI agent risks that directly affect those building on the API.






