Ollama strengthens tool calling with new models: how to build a 100% local agent

Ollama strengthens tool calling with new models: how to build a 100% local agent

Ollama has supported structured function calling for a while, but the wave of models released between October 14 and 24, 2026 made this feature much more usable for building truly local agents.

Editor's note: this article describes how the 'tools' field and the 'tool_calls' response work in Ollama's /api/chat endpoint, based on general knowledge about the tool. The source used in reporting (the GitHub Releases page) does not document this specific API behavior, and we did not have access to the official API documentation to confirm the exact details. The rest of the article's facts, including release dates and model specifications, were checked against the official release notes.

Ollama has supported structured function calling for a while, but the wave of models released between October 14 and 24, 2026 made this feature much more usable for building truly local agents.

Tool calling in Ollama isn't exactly new

Structured function calling isn't new in Ollama: the /api/chat endpoint accepts a tools field with the JSON schema of the available functions, and the model responds with tool_calls when it decides to invoke one of them. What changed in recent weeks, according to the project's release notes on GitHub, wasn't this API itself, but the number of models released specifically to use it well.

Between October 14 and 24, 2026, Ollama published four releases relevant to anyone looking to build an agent: Nemotron 3.5 Super, Nemotron 3.5 Bolt, Nemotron 3.5 Bolt E4B, and Recursive Flint. Three of them explicitly mention tool calling or tool use in the official description: only Bolt, described as a distillation of Super for everyday agentic tasks, doesn't use these terms in its release notes, even though it inherits the same capability from the original model. It's this package that gives real support to the 'local agent' angle, not a change in the API.

The models that arrived to power up local agents

NVIDIA released Nemotron 3.5 Super on October 14 (v0.40.5): a vision-and-language model with a hybrid Mamba-2/Transformer mixture-of-experts backbone, 120B total parameters and 12B active per token, context of up to 1M tokens, and tool calling support with an on/off reasoning mode. The same release brought Nemotron 3.5 Bolt, a distilled version with 74B parameters (9B active) sized to fit on a 24 GB NVIDIA GPU, with speculative decoding via an included drafter.

Nine days later, on October 23 (v0.40.10), Nemotron 3.5 Bolt E4B arrived: a smaller model based on Gemma 4, with a 128K-token window, aimed at quick conversations, image reading, and tool use on consumer hardware. And on October 21 (v0.40.8), Ollama brought Recursive Flint: a 40B-parameter mixture-of-experts model (about 2.5B active per token), licensed under Apache 2.0, designed for the work code agents do across an entire repository (understanding unfamiliar code, tracking down bugs, implementing features, writing tests), with multi-turn tool use and 262K-token context.

It's worth noting that, according to Recursive itself, Flint scored 54.9% on Terminal-Bench 2.1 and 74.2% on SWE-bench Verified in its release evaluations. These are self-reported numbers from the model's maker, published in Ollama's release notes, not independent measurements.

Building a simple agent with tool calling

The usage pattern doesn't change across these models: you declare the tools as a JSON schema and let the model decide when to call each one. A minimal agent using the Python ollama library looks like this:

import ollama

def get_weather(city: str) -> str:
 return f"27 graus em {city}, com poucas nuvens"

tools = [{
 "type": "function",
 "function": {
 "name": "get_weather",
 "description": "Retorna a temperatura atual de uma cidade",
 "parameters": {
 "type": "object",
 "properties": {
 "city": {"type": "string", "description": "Nome da cidade"}
 },
 "required": ["city"]
 }
 }
}]

response = ollama.chat(
 model="nemotron-3.5-bolt:e4b",
 messages=[{"role": "user", "content": "Qual a temperatura em São Paulo agora?"}],
 tools=tools,
)

if response.message.tool_calls:
 for call in response.message.tool_calls:
 if call.function.name == "get_weather":
 print(get_weather(**call.function.arguments))

This is the script I'd follow for a quick test: a model small enough to run on a consumer GPU, a single function, a short response. For a real agent, the loop needs to execute the function, return the result as a tool-role message, and send it back to the model until it stops requesting tools, the same pattern used in any function-calling-compatible API.

Decision models: a shortcut to route without spending a whole LLM

Not every step of an agent needs a full generative model. Ollama has also been packaging so-called decision models: small models (from 2B to 27B parameters) trained only to classify, answer yes or no, or score options with a probability. The d3 family, brought over from vLLM Semantic Router and published on October 24 (v0.40.11), accepts text, structured state, or image as input.

Usage is straightforward from the terminal, with the --ask flag:

ollama run d3 "A customer says their card was charged twice." --ask "Is this a billing issue?"

The same pattern has existed since October 12 (v0.40.4) with other decision models, as in the example ollama run nimble --ask "Is a refund being requested?" < ticket.txt. In a real agent, this type of model acts as a router: it decides which tool or sub-agent to trigger before spending an expensive call on a 40B or 120B parameter generative model.

Where latency and accuracy weigh against going local

The hardware math is the first limit. Running Nemotron 3.5 Super (120B parameters, 12B active) requires a datacenter GPU; even the Bolt version, designed to fit in 24 GB, leaves little room to run another heavy process at the same time. Large contexts, such as Super's 1M tokens or Flint's 262K, cost prefill time: the more tool history the agent accumulates, the slower each response gets.

In my reading, the second limitation is reliability in call formatting. Smaller local models (Bolt E4B, for example) tend to make more mistakes extracting arguments from a complex function than a frontier model specifically trained for tool use at scale, like the ones running behind cloud APIs. This isn't measured in Ollama's release notes; it's an inference based on the known pattern of smaller models versus frontier models. Before trusting a production agent to this, it's worth measuring the rate of malformed calls in your own use case.

The comparison with the cloud also comes down to network: a cloud API adds round-trip latency on every tool call, but it doesn't depend on the local GPU or on a model-loading queue. Locally, network latency disappears, but inference time becomes the bottleneck on modest hardware, especially when switching between a generative model and a decision model multiple times in the same flow.

When it's worth staying 100% local

In short: going local pays off when the data can't leave the machine (proprietary code, customer data) and when the agent makes few tool hops per turn, a case where a decision model plus a 9B-to-40B active-parameter model gets the job done.

It doesn't pay off when the agent needs to orchestrate several tools in sequence with a high success rate, or when there's no GPU available to sustain a 24 GB or larger model running all the time. Recursive Flint, which started running on Linux and Windows in addition to macOS as of October 22 (v0.40.9), widens who can test the full code-agent scenario today.

A recent interoperability improvement helps with migrating between the two worlds without rewriting everything: since October 22 (v0.40.9), Ollama correctly reports failures during streaming via endpoints compatible with OpenAI Responses and Anthropic Messages, instead of simply cutting the output off halfway through. This makes it easier to test the same agent code by swapping only the baseURL, from local Ollama to a cloud API and back.