AIARTICLE

Datasette gains support for local models via datasette-llm 0.1a11 plugin

The datasette-llm plugin, which serves as a common LLM configuration layer for other Datasette plugins, now accepts any endpoint compatible with the OpenAI API. In practice, this opens the door to LM Studio, Ollama, and llama.cpp running on your own machine.

Datasette gains support for local models via datasette-llm 0.1a11 plugin
Image: Alan Andrade
Article · AI
Reading
5 min
Tables
1
Code
2

The datasette-llm plugin, which serves as a common LLM configuration layer for other Datasette plugins, now accepts any endpoint compatible with the OpenAI API. In practice, this opens the door to LM Studio, Ollama, and llama.cpp running on your own machine.

Simon Willison, the creator of Datasette, published version 0.1a11 of datasette-llm on November 14, 2026, the plugin that works as an intermediate configuration layer for language models used by other plugins in the Datasette ecosystem. The core change is small to write and large in effect: now you can point Datasette at any endpoint that speaks the OpenAI protocol, which includes servers running on your own machine.

Before this release, configuring an LLM-dependent plugin in Datasette generally meant using a cloud provider with an API key. With support for OpenAI-compatible endpoints, you swap the OpenAI key for an api_base pointing to localhost, and the rest of the pipeline stays the same. It's the difference between depending on a specific provider and depending only on an open protocol.

What datasette-llm actually is

datasette-llm isn't a plugin you use on its own. It exists so that other Datasette plugins, like datasette-agent, don't each need to reinvent their own logic for "which model to use and how to talk to it." It's a piece of shared infrastructure, similar to the role a package like httpx plays for several HTTP libraries in Python: everyone delegates to the same layer instead of duplicating code.

The version number, 0.1a11, follows Python's PEP 440 convention: it's the eleventh alpha build of the 0.1 series. It's worth reading this as a stability warning, not a defect. The configuration API may change in future releases, which is why it makes sense to pin the version with uvx --from datasette-llm==0.1a11 instead of letting the resolver grab whatever is newest without warning.

Prerequisites before running it

To reproduce the official example, you need:

  • uv installed (it provides the uvx command, which runs Python packages without permanent installation)
  • a local server that speaks the OpenAI protocol: LM Studio, Ollama, or the llama-server from llama.cpp, which just reached version 1.0.0
  • a model loaded on that server; the source example uses Qwen 3.8 27B, the same model Simon Willison tested running on his own laptop in a post from November 3, 2026
  • a SQLite database for Datasette to open; the example uses the project's own demo content.db

The path I'd follow to get this running

Editor's note: the specific behavior of some flags in the command below is not detailed in the source consulted; consult the official Datasette documentation before using them outside a test environment.

The first step is downloading the example database:

bash
wget https://datasette.io/content.db

With LM Studio (or Ollama, or llama-server) running and the model loaded on port 1234, the official command starts Datasette with the datasette-agent plugin and configures datasette-llm to point to that local server:

bash
uvx --with datasette-agent datasette \
 --root --secret 1 \
 -s plugins.datasette-llm.default_model '{
 "model": "lmstudio-qwen",
 "model_name": "qwen/qwen3.8-27b",
 "api_base": "http://localhost:1234/v1"
 }' content.db

Each piece of this command has a role in the official example. The --root and --secret 1 flags are part of the command documented by Simon Willison to make local server execution easier; the source doesn't detail the exact behavior of each one, so it's worth checking the official Datasette documentation before using them in any environment other than an isolated test one. The -s flag writes a plugin configuration key, and the inline JSON defines three fields: model (an internal nickname), model_name (the identifier the local server itself expects), and api_base (the address of the OpenAI-compatible server).

If it works, Datasette comes up normally on the default port (localhost:8001) with an instance of Datasette Agent talking to the local model instead of a paid API. To confirm the connection is working, just open the agent's interface and ask a question about the data in content.db; if an answer comes back, the api_base is correct and the model is responding.

Comparing the three possible local engines

The choice of local server changes the setup experience, not the final result, since they all speak the same protocol:

ToolHow it exposes the OpenAI-compatible endpointUser profile
LM StudioBuilt-in server in the GUI itself, configurable portThose who want to test models without the command line
OllamaThe ollama serve command exposes the endpoint automaticallyThose who already download and manage models via CLI
llama.cppThe llama-server binary, now at v1.0.0, exposes the same protocolThose who want fine-grained control over build and performance

Why this matters for those building data pipelines

The practical angle here isn't "yet another plugin," it's the dependency swap. Before, if your data pipeline in Datasette used an LLM-dependent plugin, the inference bill (and the data traveling to it) were tied to an external provider. With a configurable api_base, the same pipeline runs entirely inside your network perimeter, with no byte going outside.

This has two concrete effects for those working with sensitive data in Brazil: marginal inference cost near zero once the hardware is paid off, and no dependence on a contract with a foreign vendor to keep the pipeline running. The trade-off is that response quality now depends on the model you can actually run locally, and a 27-billion-parameter model like Qwen 3.8 27B requires generous GPU or unified memory to run with acceptable latency.

What's still alpha

Three points deserve attention before putting this into anything beyond an experiment. First, the 0.1a11 notation is a real warning: the plugins.datasette-llm.default_model configuration key may change format in future releases, so it's worth pinning the package version in any automation script.

Second, --secret 1 appears in the official example command meant to run quickly in a test environment; the documentation consulted doesn't detail its exact behavior, but in any environment exposed beyond localhost it's worth checking the official Datasette documentation before using this fixed value. Third, model_name needs to match exactly the identifier the local server expects; a mismatch between what's loaded in LM Studio or Ollama and what's in the configuration JSON is the most common cause of "model not found" errors in this kind of OpenAI-compatible proxy.

datasette-agent, the plugin that actually uses this configuration to talk to the data, remains young and evolving alongside it. For those already running Datasette in production, the safest path now is to test this combination on a non-critical database before deciding whether the local model delivers enough quality to actually replace a paid API.

Translated from the Brazilian Portuguese original · Read the original

View profile →