Project compiles the NES's entire CPU into a Qwen model's weights, without training anything

An old processor, compiled into new weights

The project nesformer, by dbreunig, takes the 6502 (the processor inside the NES's 2A03 chip), the console's 2 KB of working RAM, and a cartridge's 32 KB of ROM, and turns them into the weights of a standard Qwen3_5ForCausalLM checkpoint. The README's central sentence is direct: "Nobody trained these weights; a compiler computed every one" (nobody trained these weights; a compiler computed every one).

This changes what the word "model" means here. There is no dataset, no gradient, no loss. There is a compiler that translates a digital circuit, logic gate by logic gate, into attention matrices and MLPs of an already existing Transformer architecture. The result actually plays Super Mario Bros., Donkey Kong, and Excitebike, token by token.

Where the RAM lives inside a neural network

The trick comes from an earlier project, Qwenputer, by Divyansh Agrawal, which had already compiled a CHIP-8 and later a RISC-V core inside a Qwen3.5-4B. nesformer reuses his compiler, called chipformer, and its central idea: using the Gated DeltaNet state (Qwen3.5's linear attention mechanism) as addressable memory.

Each DeltaNet head keeps a 128 × 128 state matrix. With the gate held open and a one-hot key, an update overwrites exactly one row of that matrix; a one-hot query reads that row back. In practice, each row becomes a memory cell, and it's in these rows that the cartridge, the RAM, and the 6502's registers live.

In short: DeltaNet is not "learning" to play the game. It is being used as an addressable memory table, and the compiler manually decides what each layer and each attention head stores.

What's in the weights and what's left out

The checkpoint has a clear split between what lives in the model and what lives in the host program (nesformer/host.py), written in plain Python:

In numbers, the checkpoint has 466.6 million parameters in bf16 (933 MB), but only 9.3 million of them are nonzero. That's why the compressed file fits in 3.3 MB: the structure is that of a full Qwen3.5, but almost the entire tensor is zeroed out, because the compiler only needs a tiny subset of connections to implement the circuit.

The architecture uses 7 Gated DeltaNet layers (the circuit itself) plus 1 fully zeroed attention layer, which exists only because the Hugging Face cache requires it. The hidden size is 768, and the vocabulary has 263,808 tokens, each representing a specific processor event: a decoded instruction, an I/O read, a DMA write.

Numbers that prove (and don't hide) the trick

The repository doesn't just claim it works: it publishes the verification. The reference CPU matches the standard nestest log for 3,348 instructions and the py65 emulator for 5.7 million instructions of actual gameplay. The exact circuit, run on the Qwenputer evaluator with bf16 accuracy checking, correctly predicts every reference token, including 283,610 positions of the boot sequence for two games.

Running freely (without forcing the correct next token), the results match the conventional emulator frame by frame:

1,650 tokens per second versus 700,000

Here's the point that separates curiosity from product: the standard engine, compiled with numba, runs at about 1,650 tokens per second on a single CPU core. The pure numpy engine does about 210 tokens/s, and running the checkpoint on the standard transformers library (via --engine hf) drops to roughly 10 tokens/s.

An actual NES would need about 700,000 tokens per second to run in real time. Even in its fastest configuration, nesformer runs 150 to 290 times slower than the original 1983 console. The README itself admits this plainly: a 600-frame run (10 seconds of gameplay) takes about 25 minutes to generate.

The command that kicks off one of these sessions is simple:

python -m nesformer run roms/smb.nes --frames 600 --show \
 --buttons "start@60-65,right@200-600,a@250-265+300-315+360-375+420-435+480-495+540-555"

This script turns on the game, walks Mario to the right, and jumps about once per second. The host loads the ROM into the model's state at startup, so the same weights can run any NROM game without needing to recompile anything.

What the project doesn't do

The limitations are documented and specific, not generic:

• It only runs NROM cartridges (mapper 0 of the iNES standard), because the CPU only addresses 32 KB of ROM at a time, stored in a fixed table inside the model. Larger games use bank-switching chips (mappers such as MMC1 or MMC3) that the compiler doesn't yet implement.
• Only official 6502 opcodes: an undocumented opcode triggers HALT.
• Code must run from the cartridge ROM; a jump into RAM also triggers HALT.
• There are no APU or DMC IRQs; only BRK and NMI work.
• The PPU and APU are not in the weights: rendering is done scanline by scanline by the host, mid-scanline effects are not reproduced, and there is no sound.

What this says to people who work with models

It's worth separating what nesformer proves from what it doesn't. It is not evidence that small models can reason their way through tasks that currently require expensive GPUs: there is no semantic inference happening at all, and the result is, in practice, an extremely inefficient 6502 emulator, hand-written in the form of neural network weights.

What the project concretely demonstrates is something else, and it's interesting for anyone following linear attention architectures: the state of a Gated DeltaNet (the same family of mechanism used in Qwen3.5 to reduce long-context cost) can function as general-purpose addressable memory, writable and readable via one-hot keys, with the same discipline as a conventional array. It's a proof of concept of expressiveness, in the same academic spirit as work showing Transformers simulating Turing machines, not a production-ready engineering shortcut.

For anyone working with DeltaNet, RWKV, Mamba, or any state space model in production, the practical value is in nesformer.json and the layer plan documented in the repository: they show, bit by bit, how to compile a deterministic state inside a linear recurrent architecture, something that's normally hidden behind training. The code is under the MIT license, runs with pip install numpy scipy numba pillow, and the tests (python -m pytest -q tests) reproduce the entire verification chain described in the README.