Apple shows 816 billion-parameter LLM running locally on Mac Studio, no cloud

What Apple Published

On October 15, 2026, the Apple Machine Learning Research team published a technical post showing how to run "frontier"-scale language models entirely on in-house hardware, without going through any cloud API. The protagonist is Muse Spark Open, a mixture-of-experts (MoE) model with 816 billion total parameters, of which only 42 billion are active per inference.

The experiment runs on MLX, Apple's open-source array framework optimized for the unified memory architecture of Apple silicon. The demo uses a cluster of four Mac Studio units with the M5 Ultra chip, connected via Thunderbolt cables, and manages to generate more than 36 tokens per second with this giant model, a rate Apple itself classifies as above the average human reading speed.

Why a Model This Size Doesn't Fit on a Single Machine

At native precision (BF16), Muse Spark Open would require more than 1.6TB of RAM just for the parameters. Even quantized to 4 bits (NVFP4), the model still takes up about 493GB, which, combined with the memory needed for intermediate inference calculations, exceeds the capacity of a single Mac Studio (which tops out at 512GB of unified memory). Apple's solution is to split the model across machines.

There are two ways to do this split, according to the post:

• Pipeline parallelism: each machine holds a piece of the model's depth (a set of layers) and processes its part in sequence, passing the result to the next.
• Tensor parallelism: each machine holds a slice of the width of each layer, processing part of the calculation of each layer in parallel with the others.

Tensor parallelism delivers more speed, but depends on very fast communication between machines, because it exchanges data at every layer and every token.

What Made This Possible: RDMA over Thunderbolt

The piece that unlocks tensor parallelism is a low-latency driver called RDMA over Thunderbolt 5, introduced in macOS 26.2. Starting with version 0.30.1, MLX began using this driver through a set of communication routines named jaccl (pronounced "jackal"). In practice, the researcher connects the Mac Studios in a mesh with Thunderbolt cables and starts the job with the mlx.launch command, as in this example from the post itself, used to host Muse Spark Open as a local server on port 8080:

mlx.launch --hostfile hostfile.json --backend jaccl -- env MLX_METAL_FAST_SYNCH=1 -- \
mlx_lm.server --model meta-models/Muse-Spark-Open-816B-A42B-NVFP4 --port 8080

Apple also compared the M5 Ultra to the previous generation, M3 Ultra, on smaller models that fit on a single machine (Qwen 3.8, at 27B, and Mistral Medium 3.5, at 128B). Time to first token is up to 4.6 times faster on the M5 Ultra, driven by the dedicated matrix accelerators on the GPU; token generation speed is up to 1.6 times higher, thanks to the M5 Ultra's 1.2TB/s unified memory bandwidth versus 819GB/s on the M3 Ultra.

The Power Consumption That Stands Out

One figure Apple highlights separately is power consumption: about 200W peak on the M3 Ultra and 300W on the M5 Ultra during inference, which the company compares to "a fraction of what a common toaster uses." The argument, explicit in the post, is that this consumption level makes it feasible to keep this kind of AI workload running at home or in a regular office, without data center infrastructure.

What Changes for Product Decision-Makers

This post is research material, not a finished product announcement, but it carries a relevant reading for those who design and decide on AI features. Running a frontier model entirely within one's own network, with no call ever leaving it, changes the build vs. buy calculus for products that today avoid generative AI because of sensitive data, such as medical records, legal contracts, or customer financial data.

In short: the technical barrier of "you can't run cutting-edge AI without sending data to a third-party cloud" is getting thinner, at least for those with hardware budget.

It's worth noting, though, that this independence trades one kind of dependency for another. Instead of a cloud API contract, the requirement becomes owning two or four Mac Studios with M5 Ultra, each already at the 512GB memory ceiling, connected by specific Thunderbolt cables. For most product teams, this is still a considerable capital investment, far from "any laptop on the team."

There's also a positioning reading behind the publication. Apple open-sources the MLX code, publishes reproducible benchmarks, and writes a step-by-step guide on how to build the cluster: a clear product move to establish itself as the platform of choice for AI researchers and developers, competing for the same audience that today rents cloud GPU to train and serve models.

Where This Shows Up Day to Day: Local Agentic AI

The post ends by connecting the experiment to a concrete use case: agentic coding tools running locally. With the MLX server up, you can point an OpenAI API-compatible client, like OpenCode, directly at the local endpoint:

{
 "$schema": "https://opencode.ai/config.json",
 "provider": {
 "mlx": {
 "npm": "@ai-sdk/openai-compatible",
 "name": "Local MLX",
 "options": { "baseURL": "http://127.0.0.1:8080/v1" }
 }
 }
}

For a PM or designer evaluating the UX of an internal copilot, the number that matters here is generation speed: the 36.73 tokens per second recorded with four Mac Studios in tensor parallelism (versus 15.74 tokens/s with pipeline-only across two machines) is the kind of data that determines whether a chat or agent experience feels smooth or sluggish.

What Remains Open

Apple has not disclosed the total cost of setting up a cluster of four Mac Studios with M5 Ultra, nor a direct cost-per-token comparison against equivalent cloud APIs for a model of Muse Spark Open's size. It's also unclear, from the published material, whether this path is meant to move beyond the research environment and become something packaged for product teams without a dedicated infrastructure engineer: for now, it's a developer and researcher tool, not a ready-to-use product.