Independent benchmark puts open model GLM-5.3 at the top, at one-fifth the cost of GPT-5.5

An independent leaderboard called Ed-o-meter, maintained by Ed Yau (Applied AI Architect at Kerv), put the open-weight model glm-5.3 at the top of a battery of 28 real-world tasks, outperforming models from OpenAI and Anthropic at a fraction of the cost. The ranking was updated on August 23, 2026, when four new models entered the track, among them glm-5.3 itself, grok-4.6, deepseek-v4-pro, and gemini-3.7-flash.

According to the author, glm-5.3 is the first model on the board to pass all five evaluated categories (coding, data, realworld, security, and tool-use) with a 100% pass rate. It completed the "lap" (the set of 28 tasks) for $0.28, versus $1.43 for gpt-5.5 in the same run, that is, about one-fifth the cost. The model also scored 9.3 on the quality rubric, the third highest in the ranking.

How the benchmark works

Ed-o-meter's differentiator is its methodology, described in detail in the source. The proposal is to measure real tasks that real people perform, not academic metrics. Each task is treated as a unit test for agents: "an agentic flow is, in the end, a series of these tasks," the author writes. The full suite is cheap to run, costing about $30.

The measurement rules are strict and transparent:

• Same prompts, same API calls for all models, through a single streaming path via OpenRouter, run in series ("time-trial, no other cars on the track").
• Latency measured as time-to-first-token (TTFT), with the clock isolated by serial execution.
• Binary, automated checkers; the only component judged by an LLM is the quality rubric, and its bias is disclosed in the footnotes.
• Routing pinned with allow_fallbacks:false, to prevent the provider from silently serving a quantized variant.
• Refusals are logged, not hidden: a provider-side block is logged as a refusal with its category, never resent to another model.

The harness, tasks, and checkers are open source under the MIT license in the Featherbench project, and anyone can clone it and run the lap on their own, or request the inclusion of a new model via a GitHub issue.

The five evaluated categories

The 28 tasks are divided into: Coding (7 Python tasks, such as fixing the classic mutable default argument bug, correcting an off-by-one error in pagination, and implementing a rate limiter with an injected fake clock), Data (4, including deriving a star-schema model from a stakeholder transcript), Realworld (9, such as refusing to make up flight prices without live data or correcting the wrong premise of giving honey to a 5-month-old baby), Security (6 prompt injection and jailbreak tasks), and Tool-use (2, such as calling only search when explicitly forbidden to book).

According to the author, coding, data, and tool-use act as the "floor" of the test; the race is decided in realworld and security. The only cost of glm-5.3, the text notes, is patience: a median TTFT of 16.3 seconds. gpt-5.5 shows up as the faster alternative, at 13.2s and the same 100% security score, but 89% in the realworld category.

Each model has its role

The ranking doesn't crown a single winner for everything. For high-volume, low-risk work, gpt-5.6-luna is named the cheapest, $0.064 for the entire lap ($0.0023 per task) and a TTFT of 5.3s, with the heavy caveat of a 79% overall pass rate and only 33% on security. haiku-4-5 is the alternative with the most correct answers (96% overall, TTFT of 0.9s). deepseek-v4-pro, meanwhile, despite being nominally even cheaper, has a median TTFT of 40 seconds, the slowest on the board, which rules it out for interactive use and limits it to batch processing.

At the top of the quality rubric is kimi-k3, with 9.5, but with a TTFT of 26.4s. The text also documents security issues: the gpt-5.6 line (luna, terra, and sol) triggered the jailbreak "canary" in 11 of 12 test cells (33 to 50% security pass rate). The Claude trio and gpt-5.5 passed 6/6 clean.

The asterisks matter

The author is explicit about the limitations, and they matter for anyone interpreting the numbers. These are single trials (one attempt per task), which produces wide confidence intervals, visible as "whiskers" in the charts. There are also four recorded caveats, among them the case of fable-5, whose 9.3 score is self-judged (the model evaluating its own answers) and comes from an earlier run, shown "for completeness, not as a comparable number."

Another point of attention is what the author calls "measurement risk": both opus-5 and fable-5, both from the Anthropic family, had benign debugging tasks blocked by a provider-side classifier before generating a single token. opus-5 scores 43% on coding not due to an inability to debug, but because four tasks were blocked by the filter. "Treat this as a measurement risk, not a model quirk," the text recommends.

What changes for the Brazilian developer

For those building software in Brazil under a tight token budget, the central data point is the savings: an open-weight model delivering a 100% pass rate at one-fifth the cost of a top proprietary model changes the math for any product that relies on LLMs at scale. Open-weight models also open up the possibility of self-hosting and running on local or regional infrastructure, which helps in scenarios involving latency, dollar costs, and compliance with LGPD (Brazil's data protection law), when sensitive data cannot cross the border.

The author himself, however, makes a practical caveat right in the summary: "if you're running a model, run glm-5.3, but check with compliance first." The recommendation applies doubly in the Brazilian corporate context, given the model's Chinese origin and each company's internal data governance policies.

Two things remain open. First, since these are single trials, the results call for replication, and the benchmark itself makes that easier by being open source. Second, the ranking measures isolated tasks as a proxy for agentic flows, but it's no substitute for testing your own real use case. The most honest path, one that Featherbench allows, is to clone the harness and run your own lap before switching models in production.