IBM's Granite Switch 4.1 brings chat, decisions, and speech together in a single endpoint

What changes with Granite Switch 4.1

On October 15, 2026, IBM published Granite Switch 4.1 on the Hugging Face blog, authored by Kate Soule, Subhajit Chaudhury, Hendrik Strobelt, and Luis A. Lastras, from the Granite team. The package brings together two sizes, Granite-Switch-4.1-8B and Granite-Switch-4.1-30B, which do three things at once: respond in natural language, make decisions (classification, moderation, routing), and transcribe audio. All of this from the same endpoint, without switching models or infrastructure.

This solves a real problem for people building AI products: today it's common to have a chat model, a moderation/classification API (often closed, like the Jev family cited in IBM's own post), and a separate transcription service, each with its own GPU cost and its own network latency. Granite Switch tries to replace that stack with a single process.

aLoRAs: the trick behind the single endpoint

The central piece of the architecture is Activated LoRAs (aLoRAs), lightweight adapters attached to the Granite base model. Dubbed "cache-efficient" by IBM for being Activated LoRAs, these adapters allow the KV cache generated by the base model's generation to be reused in any subsequent decision made by an adapter.

In practice, this means that running the chat model, applying the decision adapter, and applying the safety adapter together consume roughly the same VRAM that running the base model alone would consume. IBM lists recent contributions from the team to vLLM to speed up inference for these low-rank adapters, together with a speech encoder co-located in the same process.

Granite Switch 4.1 models can switch between a chat request and a decision request and a spoken request, with blazing fast speeds.

Kate Soule and the IBM Granite team, on the Hugging Face blog

What comes inside the 4.1 package

Both sizes of Granite Switch 4.1 come with three pieces pre-installed:

• Decision aLoRA, a generic decision adapter, designed to replace calls of the /v1/systemone type (the API standard used by "fast" decision models, such as Jev);
• Guardian aLoRA, specialized in safety decisions, which IBM says outperforms, on average, competitors Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety tasks;
• Granite-Speech-5.0-Turbo-CTC, the speech recognition model that leads the OpenASR Leaderboard in speed, built in for transcription tasks.

These three components, plus the base model, form a single combined model - not three separate processes.

One endpoint, three tasks: the demo that proves the concept

IBM published a Hugging Face space called Speech-to-Decisions, in which granite-switch-4.1-8b receives audio, generates a summary, checks whether the content is harmful, and tests another 15 user-defined flags, all in real time, on the same endpoint. It's the most concrete use case in the announcement: moderating support calls, transcribing meetings with automatic risk triage, or customer service pipelines that today need to chain three different services.

For those building RAG pipelines or agents that need to decide "what to do next" without waiting on an external API call, this is relevant: the decision comes from the same process that already generated the response, without an extra network round-trip.

Granite Libraries: building your own Switch

Granite Switch wasn't born with 4.1. The project was introduced by IBM in the spring of 2026, along with Granite Libraries, a repository of specialized adapters ("decision model" and other tasks) that can be composed into custom Granite Switch models. The idea is that nobody needs to use IBM's "batteries included" version: you can build your own combination of adapters from that library.

The interesting point, according to the post, is that specialized adapters from granitelib tend to outperform generalist decision models (like Jev) on specific classification tasks. Since the adapters share the base model's KV cache, stacking a specialist alongside a generalist costs almost nothing in terms of resources: you get broad coverage with the generic adapter and high precision where it matters with the specialist.

IBM also announced plans to release an adapter SDK for training these modules using the Mellea framework, along with tutorials and expanded support for Granite Switch, but without a set date beyond "in the coming weeks."

Against Jev: speed benchmark

The post includes two direct comparisons with Jev, cited as the reference for a closed decision model. In the first, Granite-Switch-4.1-30B (a frozen Granite base model plus 4 specialized aLoRAs) completed a real 5-step RAG workflow faster than Jev-27B (a frozen Qwen base model plus 1 generic decision LoRA), with no latency figure disclosed in the post.

In the second, more playful one, IBM trained a 3-billion-parameter version of Granite Switch to play Doom, making decisions at 60-100 per second, versus the 10 decisions per second reported for Jev in an equivalent test, according to the typesafe.ai blog cited as a footnote by IBM itself. The difference, according to the post, is that the same model can narrate the match and answer questions while playing, because chat and decision run in the same session.

How to run this in practice

Both sizes of Granite Switch 4.1 run on vLLM via a specific plug-in, available in the project's GitHub repository, and have native inference support in llama.cpp and other downstream platforms. This lets the model replace calls made to /v1/systemone anywhere that standard is already used, while also handling conventional chat calls, all with the same shared prefix cache.

For those who already have a stack running closed decision models and want to evaluate the switch, the resources published by IBM include the Granite Switch 4.1 model cards, the Speech-to-Decisions demo space, the GitHub repository, the original Granite Switch and Granite Libraries post, and a Colab notebook for setting up your own "race" between aLoRA and traditional LoRA.

Where this still falls short

Not everything in the announcement comes with hard numbers. The RAG workflow comparison doesn't give a latency figure in milliseconds, just "faster"; the benchmarks against Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash are averages without a breakdown by task, which makes it hard to know whether Guardian aLoRA wins across all scenarios or only in the easier ones. The adapter SDK, which is what would truly unlock composing third-party modules, still has no release date.

For small teams, the VRAM advantage only translates into real savings if the workload already combines chat, decisions, and speech. Running a Granite Switch just for simple decisions, without using the chat or speech part, is probably not worth the extra complexity of learning the aLoRA adapter format and the vLLM plug-in configuration.