NVIDIA releases four open Cosmos 3 models to simulate cars and robots

Four open pieces to simulate the world without relying on a closed API

On November 5, 2026, NVIDIA published on the Hugging Face Blog four new checkpoints that join the Cosmos 3 family: Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual. All of them come from post-training on top of Cosmos3-Nano, have 16 billion trainable parameters, and ship under the OpenMDW1.1 license, which allows commercial and non-commercial use within the terms defined by NVIDIA itself.

For those who build autonomous vehicle or robotics simulation, the news isn't Cosmos 3 itself, the platform already existed. It's the fact that this specific piece of the pipeline, generating video conditioned on actions and scene controls, can now be downloaded and run locally, instead of depending on a closed API from a third-party vendor.

The four models, one per piece of the problem

The first two form a pair: Transfer-Auto is the bidirectional base for offline generation, and Sim-Auto is its distilled version, trained causally to run in streaming. Those who need to adapt Cosmos 3 to their own camera and LiDAR rig start with Transfer-Auto and only then distill into Sim-Auto. Sim-Transfer and Sim-Bimanual, on the other hand, solve the same problem in opposite ways: in the first, the physics lives in the external simulator and the model only paints the RGB; in the second, the model predicts the entire scene directly from the action, with no physics simulator behind it.

In the case of Sim-Bimanual, the mixed action example released by NVIDIA uses 59 control values, padded up to 64 during training, with a schema that defines coordinate systems, rotations, gripper calibration, and validity masks. It's a strict data contract: whoever plugs in their own robotic arm needs to follow this convention exactly, or the model will misinterpret the action.

Under the hood: streaming, causal attention, and four training stages

Cosmos 3's backbone is a Mixture-of-Transformers that links a text-processing network to a visual generation network via attention. The model doesn't work on raw pixels: it operates on compressed representations called latents, where each latent step corresponds to four RGB frames. Transfer-Auto predicts one step per generated block; the optimized Bimanual configuration predicts two. Smaller blocks let the control influence generation more frequently, at the cost of more calls to the model.

The attention is temporally causal: the model reads the history and the available controls, but has blocked access to the future observations it's trying to predict. A key-value cache stores what has already been processed for the next prediction, the same kind of trick used to speed up inference in LLMs.

Training follows a four-stage recipe:

1. Stage 0, bidirectional adaptation: learns the domain and the relationship between control and video from complete, already recorded clips.
2. Stage 1, teacher forcing: predicts each segment using previously recorded observations and the available controls, with the correct answer hidden.
3. Stage 2, consistency distillation: a frozen teacher trains a student capable of generating each segment in four denoising steps.
4. Stage 3, self-forcing: the model trains on its own generated history, with signal from a frozen teacher and an auxiliary score model.

The last stage is what prevents the model from degrading over a long simulation: a slightly mispredicted object becomes context for all the following frames, so small errors tend to accumulate. Training on generated history, not just on real data, exposes the model to this kind of error before it goes into production.

The price of running this outside a data center

Before downloading any checkpoint, it's worth looking at the requirements NVIDIA itself published: Linux, an NVIDIA GPU, Docker, and the NVIDIA Container Toolkit. The reference setup, using vLLM-Omni, was tested on a single GB200, and NVIDIA is explicit: this isn't a minimum memory specification, it's just the hardware the team tested on.

In practice, this means nobody knows for sure what the real VRAM floor is for running a 16-billion-parameter model with streaming generation. For a team in Brazil without a data center GPU on hand, the most realistic path is renting a cloud instance with a data-center-class GPU, not running it locally on a consumer card. This changes the prototype's cost calculation: the model itself is free and the OpenMDW1.1 license allows commercial use, but the cloud GPU bill still exists, just without a per-API-call fee now.

Built for a coding agent, not just a human with a terminal

One detail that stands out in NVIDIA's material is how the documentation is organized: for each checkpoint, the guidance is literally to point a coding agent at an open "skill" and let it run the inference, plus ready-made notebooks for those who prefer to explore manually. This documentation choice fits how much of the technical audience already works today, delegating setup and execution to a coding assistant instead of building the environment line by line.

This reduces onboarding friction, but doesn't solve the hardware problem described above. A well-guided coding agent will still run into the requirement for an NVIDIA GPU with Docker and the Container Toolkit configured; what changes is the time spent reading documentation to get there.

When it's worth it, and when it isn't

It's worth it for those who already work with robotics using embodiments like YAM or AgiBot and need a simulation environment that responds to real actions from a bimanual arm; for autonomous driving teams that want to generate multi-sensor synthetic data (camera and LiDAR) varying weather and lighting without relying on a real fleet on the streets; and for those who already have their own physics simulator and just need a photorealistic rendering layer on top of the depth and segmentation controls that simulator already computes.

It's not worth it for those who want to locally prototype a simple game or interactive demo: the hardware footprint, with testing on a GB200 and 16 billion parameters, is far beyond what fits on a laptop or a consumer GPU. Teams that don't work with autonomous driving or physical robotics also have no direct use for the World Scenario Maps and the LiDAR controls that underpin two of the four checkpoints. It's also worth remembering that the weights exported for inference don't carry optimizer state: anyone wanting to continue post-training needs to keep the full training checkpoints, not just the inference ones.