IBM introduces LLMON, a native format for structured prompts in Granite 4.1

IBM published LLMON (LLM Object Notation, pronounced "lemon") on the Hugging Face blog on November 12, 2026: a markup language designed to be read by a language model, not by a parser. The announcement came with two experimental models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, derived from the Granite 4.1 base checkpoints, plus a GitHub repository with converters and examples. For anyone building a RAG pipeline, an agent, or a prompt with multiple instructions in production, the interesting angle isn't the catchy name: it's what changes when structuring what goes into the model's context.

Why JSON isn't enough for a language model

A long context accumulates the system prompt, conversation history, instructions that act on retrieved documents, tool responses, user data, output requirements, and reasoning chains. This structure is usually encoded in JSON, a format designed for deterministic parsers, not for a causal model.

The problem is that tokens like {, ", and : also appear in ordinary text: to the model, the quotation marks that close a field and the quotation marks that appear inside a document are the same token, and escaping rules that a parser follows by grammar the model only follows by statistical pattern. Counting nesting depth by parentheses is also something transformers do poorly over long spans.

JSON was designed for parsers, not for causal language models.

IBM Granite Team, on the official Hugging Face blog

How LLMON marks structure

LLMON resembles XML, but the structural delimiters are special reserved tokens in the tokenizer, not ordinary characters, so that structure and content occupy distinct parts of the vocabulary. Each segment gets a tag at the start and a two-character ID, drawn from a fixed set that the tokenizer encodes as a single token, so addressing any block always costs two tokens.

An exec block ties instructions to the data and requirements they should act on, in a way similar to calling a function and passing arguments in a programming language. IBM shows the example of a user message with instruction, data, and two requirements:

messages = [{
 "role": "user",
 "instruction": [{"value": "Write a short paragraph about", "exec": True}],
 "data": [{"value": "IBM Company", "exec": True}],
 "requirement": [
 {"value": "Use less than 10 words", "exec": True},
 {"value": "Do not use commas", "exec": True}
 ]
}]

The model receives this as six addressable blocks: instruction, data, two requirements, an exec block that ties everything together, and an output block, with each part marked exec: True or False. A set flattened into a single string, as a conventional chat template does, forces the model to infer on its own which piece is the task, which is the subject, and which is the constraint. LLMON doesn't replace the chat template: it enriches what already exists, indicating not only which part belongs to which role (system, user, assistant), but what each component means and how it relates to the others.

Prompt injection: the numbers behind the promise

The explicit separation between instruction and data was designed to directly address indirect prompt injection, when a malicious instruction is hidden inside a retrieved document, a tool response, or user data. IBM tested the LLMON previews against open and closed models on the OPI benchmark, which injects the malicious instruction into classic NLP tasks, measuring robustness as 1 minus the attack success rate. The results reported by the team:

• Models without LLMON follow the injected instruction with high probability.
• LLMON blocks 47% more attacks than the best frontier model tested.
• LLMON blocks 95% more attacks than the best open model tested.
• A similar trend appears in the ASB and StruQ benchmarks.

It's worth noting a caveat the team itself makes: these benchmarks test static injection, the kind that appears all at once in the data. Adaptive attacks, which unfold over multiple conversation turns, are still under active development by the team. And the defense didn't come at the expense of general capability: on standard benchmarks, the LLMON previews land at the same level as their corresponding Granite 4.1 Instruct models.

How the models were trained

The previews start from the Granite 4.1 base checkpoints, not from the already-released instruct models, so that LLMON behavior is learned during post-training instead of being layered on top of an existing chat format. IBM reserialized a conventional instruction-tuning corpus into LLMON, turning each prompt component into an addressable block with an exec block naming which instruction to run, over which data, and where to send the output.

To teach the model to ignore what isn't called, the team augmented the data with instructions and data blocks deliberately left unlinked in the exec, keeping the target output unchanged. Reinforcement learning came next: five sequential GRPO stages covering multi-domain reasoning, RLHF, knowledge calibration, and math, followed by a final DPO stage over a thousand preference pairs built from the model's own responses to injection attempts.

Custom vocabulary: the experiment with ScienceWorld

LLMON isn't locked into a fixed schema: you can choose tag names and structures that make sense for your domain and fine-tune a LoRA so the model understands that vocabulary. IBM demonstrated this on ScienceWorld, a text-based agent benchmark that performs elementary science tasks (such as boiling water) by issuing one command at a time.

The team defined tags such as instr.task.goal, req.planning_policy.rule, data.state.room, data.state.inventory, and assistant.next_action, rendered the benchmark's reference trajectories in that vocabulary, and trained a LoRA on top of an LLMON model. The resulting model plays never-before-seen test episodes by writing its next action directly in the new vocabulary, and a notebook published in the repository shows the full pipeline, from converting the trajectories to comparing against the base model without the fine-tune.

What can already be tested, and what's still missing

For anyone already running Granite 4.1 3B or 8B, switching to the LLMON preview doesn't require a conversion pipeline: naming the parts of the turn (instruction, data, requirement) is already enough, because the built-in chat template turns that into LLMON blocks automatically. The LLMON-converters library goes a step further and preserves nested structure (a JSON object inside the payload, for example) as typed, addressable blocks instead of an opaque string. The Mellea framework already has native support, LLMONizing LLM calls automatically, and there's an attention visualization published as a Space on Hugging Face for anyone who wants to see the instruction/data separation happening across the model's layers.

The most important caveat is in the announcement itself: this is an experimental preview, ready for exploration and evaluation, not for production. The privilege rule (an instruction reached through an exec block is authoritative; what arrives as data is material to read, not a command to obey) is a behavior learned during training, not a guarantee enforced by engineering. Anyone deciding to test it on sensitive data should treat the robustness gain against prompt injection as benchmark evidence, not as certified protection, and should follow the GitHub repository (generative-computing/llmon) and the papers cited by the team to see how the defense evolves against multi-turn adaptive attacks.