Inkling 2 Mini uses a looped transformer to run locally without a high-end GPU
Inkling 2 Mini, announced on November 20, 2026 by Thinking Machines Lab via the Hugging Face blog, uses a trick of repeated layers to simulate a larger model while weighing in at 3B active parameters per token.

- Reading
- 7 min
- Code
- 3
Inkling 2 Mini, announced on November 20, 2026 by Thinking Machines Lab via the Hugging Face blog, uses a trick of repeated layers to simulate a larger model while weighing in at 3B active parameters per token.
Hugging Face published the introduction post for Inkling 2 Mini on November 20, 2026, a multimodal model from Thinking Machines Lab built for those running AI outside of datacenters. The core idea, described in the text signed by merve, Pedro Cuenca, Aritra Roy Gosthipaty and Guarin Flück, is to simulate a much larger model by reusing the same layers multiple times during inference, instead of simply stacking more physical parameters.
For those testing agents, RAG pipelines, or just want a code copilot running on their own machine, this is the practical point: the model has 26 billion parameters in total, but activates only about 3 billion per token thanks to its Mixture-of-Experts architecture. This brings the inference compute cost closer to that of a much smaller model, even while delivering capability comparable to larger models across several benchmarks.
How the layer loop works
Inkling 2 Mini has 24 physical decoder layers, but the 12 middle layers run three times in a loop. In practice, the model behaves as if it had 48 logical layers (6 + 3×12 + 6), without needing to store weights for all of them.

Each pass through the looped core starts from the same input, which is the output of the first six layers. Communication between passes happens through "cross-pass messages": each layer reads a message from the output of the previous pass using its own learned query, and a gating controller decides how much of the current value to keep and how much of the message to incorporate. This mechanism is what lets the third pass refine what the second one computed, instead of repeating the same calculation three times.
The idea of repeating layers (looping) isn't new; architectures like Nanbeige have already explored variations of the concept. The difference here is the interplay of messages and gates between passes, which gives the model a way to accumulate reasoning without growing on disk. Each layer visit keeps its own KV-cache and short-convolution state, so the capacity gain isn't free in terms of context memory, even while reusing the weights.
Expert routing follows the standard MoE pattern: for each token, the model picks 8 out of 128 SwiGLU experts per layer, normalizes the selected scores, and scales the weights by 8. The model's hidden size is 2,304, much smaller than its larger sibling, Inkling 2 Small (4,608), which also helps reduce the memory footprint.
Where it stands in the local model ranking
According to the post, Inkling 2 Mini tops its category on the Artificial Analysis Intelligence Index v5 (AAv5) and leads in 32 of 48 benchmarks compared against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL, covering code, automated research, tool use, vision and safety.
On the front most relevant to developers, the model leads FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and falls behind it on AutomationBench-AA v1.0.6. It's a mixed result, not a complete win, and it's worth reading with some reservation: the Hugging Face team itself admits that the most eye-catching tests in the post are subjective impressions, not formal evaluation.
In our vibe tests, it's at least comparable with Qwen 3.8 27B. More validation is needed, but we are super excited about this release!
Hugging Face team, authors of the official post about Inkling 2 Mini
This caveat matters because part of the post's demo (a 3D WebGL flythrough, an animated recreation of "The Starry Night", and an app generated from a paper sketch) are cherry-picked examples, not a reproducible benchmark. They serve to show multimodal coding capability, not to replace your own evaluation before swapping models in production.
Running it in practice: transformers, llama.cpp, MLX and SGLang
Support via transformers uses the AutoModelForMultimodalLM class, both for the fixed checkpoint (thinkingmachines/Inkling-2-Mini) and for the research variant with adjustable number of passes (thinkingmachines/TML-d24-varloop-research):
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/Inkling-2-Mini"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id)
messages = [{"role": "user", "content": [{"type": "text", "text": "Why is the sky blue?"}]}]
inputs = processor.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True,
return_dict=True, return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024)In the research variant, you can reduce the number of passes through the loop via loop_num_passes in text_config, which speeds up inference by trading quality for speed, something that makes sense to test in a prototype before locking in the number of passes for production.
For those who want to run it outside a cluster, the most direct path is the Llama App (based on llama.cpp), with ready-made GGUF quants in 4-bit (IQ4_NL) and 8-bit (Q8_0):
llama serve -hf ggml-org/Inkling-2-Mini-GGUF:IQ4_NLDoing the simple quantization math, 26 billion parameters at 4 bits take up somewhere around 13 to 15 GB of memory, a range that fits on laptops with 16 GB of unified RAM or consumer GPUs with limited VRAM, even without coming close to the datacenter hardware used to train the model. It's this math, not a promise from the source, that backs up the "local setup without an expensive GPU" angle.
In MLX, support was merged into the mlx-lm project with quants shared by the mlx-community organization, including multimodal input:
mlx_lm generate --model thinkingmachines/Inkling-2-Mini --image image.jpg --prompt "Describe this image."For those who want to serve the model in production, SGLang offers day-zero support, with one important prerequisite: native mixed precision, which uses NVFP4 and MXFP8 on the expert weights, requires a GPU with SM103 architecture, that is, datacenter Blackwell like the B300 mentioned in the post. On consumer hardware, the realistic route remains GGUF via llama.cpp or the MLX quants, not the original mixed-precision format.
What changes for those building agents and RAG
The 1-million-token context and native support for tool calls (tool_calls) make Inkling 2 Mini a natural candidate for RAG with large windows and for agents that need to decide when to invoke an external function, without depending on a hosted model. The post shows a working function-calling example in which the model requests a call to get_current_temperature, receives the result, and incorporates it into the final response.
Another detail useful for those balancing cost versus quality in production is the reasoning_effort parameter, with seven levels (none, minimal, low, medium, high, xhigh, max, default high), besides accepting arbitrary numeric values between 0.0 and 0.99. This gives fine-grained control over how much the model "thinks" before responding, something that usually only works with manual prompts on other models.
To speed up generation even further, SGLang supports speculative decoding with its own drafter called DSpark, which reads intermediate states from three decoder visits (5, 23, and 41) plus the final normalized state to propose tokens that the target model verifies. The post cites speed benchmarks with and without this drafter on a B300, but without publishing concrete numbers in the body of the text, only the outline of the script used to measure them.
Where caution is warranted
The research variant with adjustable number of passes (TML-d24-varloop-research) is tempting for those who want to save on latency by cutting the loop from three down to one or two passes, but the Hugging Face team itself says it has only tested this configuration superficially, without in-depth evaluation. Before putting this variant into a real pipeline, it's worth running your own test suite, not trusting the "it works" of the post.
The same goes for the AAv5 benchmark numbers: they compare Inkling 2 Mini against a specific set of competitors and describe wins in 32 of 48 categories, not total dominance. For those choosing between local models today, this means testing on your project's specific tasks (code, RAG, agents) before swapping models in production, because the gap between "leader on FrontierCode" and "best model for your use case" can be large.
Translated from the Brazilian Portuguese original · Read the original
Google adds semantic governance policies for AI agents in Gemini
Google has launched in preview a system that judges, in natural language, every tool call an agent proposes before it runs. The idea is to catch what IAM and prompts can't: the request that makes technical sense but breaks a business rule.





