AEO: how frontier models choose what to recommend (and what that changes for devs)

If SEO was about optimizing content for Google to rank it, AEO (Answer Engine Optimization) is the agent-era version: optimizing so that an LLM, when answering "what's the best X," cites your product. Latent Space just published a Frontier AEO Tracker that treats the topic with the rigor it deserves, and the material delivers far more than marketing curiosity. For those building with AI in production, it's an X-ray of how different models diverge in search architecture, confidence, and bias, and that has direct consequences for your code.

What the tracker actually measured

The experiment extended AmplifyingAI's methodology (inspired by the work "What Claude Code Actually Chooses"): it ran 6 prompt variations across 7 frontier models with search enabled, over 161 categories, from coding agents to managed databases, ASR models, AI sandboxes, and even exotic categories like angel investors and payroll software.

Response extraction was done by Astra itself (the OpenAI model in the scenario described by the source), and each response received a proprietary score that weighs:

• first choice (highest weight);
• alternative choices;
• simple mentions;
• negative weight for mild and strong anti-recommendations (rare, but they happen).

The detail that matters for credibility: every prompt/response pair is inspectable. That answers the obvious question (training-data contamination/leakage), and the source states it checked. It's worth noting that Gemini/Antigravity, GLM/Zcode, and DeepSeek/DeepCode were left out of this first round due to errors and rate limits, meaning the picture is still partial.

Manufacturer bias is real and measurable

The most uncomfortable finding (and the most useful one) is that models favor their own lab when asked about coding agents:

| Model | Coding agent it tends to recommend | |---|---| | Fable/Opus (Anthropic) | Claude Code | | Sol/Astra (OpenAI) | Codex | | Grok | Cursor | | Muse | Muse Code | | SWE-1.7 | Devin |

The source comments with irony: "I wonder why." There are also "soft biases" in other categories. But the tracker records honest counterexamples, such as GPT models recommending Claude, which the source classifies as "praiseworthy nonbias."

For the Brazilian developer building an agent that queries an LLM to recommend tools, rank options, or curate content, the lesson is direct: the model choice embeds a commercial preference. If your product asks GPT "what's the best managed database" and shows the answer to the user, you're inheriting that lab's bias without knowing it. The practical mitigation is to cross-check responses from different manufacturers' models before exposing a recommendation as neutral.

Another actionable data point: out of 161 categories, only 28 have a universally dominant primary choice across all models. The rest are "tight contests," which the source nicknames categories that are "always the vibesmaid, never the vibe." In other words: in most cases, the recommendation depends on which model you called.

Efficiency vs. confidence: the tradeoff that weighs on cost

Here the tracker touches on what matters most to those paying the token bill. When comparing generations from the same lab (Sol→Astra, Opus→Fable), the source found a strong divergence in how many sources each model searches before answering:

| Model | Median sources consulted | |---|---| | Astra | 5 | | Sol | 9 | | Opus | 11 | | Fable | 15 |

The source's reading is that Anthropic is biasing its models toward searching for more sources, while Astra is described as much more "confident" or "efficient," depending on the angle. And the crucial point: Astra is far less likely to change its mind when you lightly paraphrase the question.

This tradeoff has concrete impact in production:

• More sources (Fable, Opus) tends to mean broader, more defensible answers, at the cost of more tool calls, more latency, and more tokens per response;
• Fewer sources (Astra) means faster, cheaper responses, with less variance across runs, but a higher risk of anchoring on a narrow set of evidence.

The observation about stability to paraphrasing is the most valuable one for anyone doing prompt engineering. If a model changes its recommendation every time the user rephrases the question, you have a reproducibility problem. As the source puts it, the lower the randomness in the choices, the greater the value of AEO itself, because the answer becomes a stable target.

In practice, the sensible path is to measure this for your own case: run the same intent with 5 to 10 phrasing variations and count how many times the main answer changes. Here's some pseudocode for what you could put together:

variacoes = [
    "qual o melhor banco gerenciado para startups?",
    "me recomende um managed database",
    "o que devo usar de banco gerenciado em produção?",
]
respostas = [chamar_modelo(v, search=True) for v in variacoes]
# count the mode of the first choice and the dispersion across variations

If the dispersion is high, either you switch models, or you lock the phrasing of the question into your system prompt.

What holds up (and what doesn't) when it comes to being recommended

The sourcing part comes with an honest caveat from the source itself: the sample size is small and only reflects what can be scraped from attempted tool calls, not the pretraining dataset. In other words, you can't infer training priority from sources cited at search time.

What is validatable, according to the source, are AEO practices measurable by Ora and Vercel, such as markdown content negotiation: serving a markdown version of your content for when the agent comes to read it. And the finding is operational: failures here discourage the model from reading your content. If the agent's crawler hits your page and can't extract clean text, you simply fall off the list of candidates to be recommended.

For developers, this connects AEO to infrastructure decisions already familiar from the web world:

• serving markdown or clean semantic HTML on routes that agents can consume;
• making sure content negotiation doesn't break on Accept: text/markdown;
• treating your technical content as something that needs to be machine-readable, not just renderable in the browser.

When this isn't worth your time

AEO isn't for every product. If your software doesn't depend on discovery through AI assistants (an internal tool, a closed B2B system, something sold through relationships), investing in being recommended by an LLM is premature optimization. The ROI the source cites ("naive autoresearch investment in our AEO have yielded impressive ROI") is its own, in a niche where founders and DX leaders actively ask about the topic, not a transferable promise.

The most defensible use of the tracker for builders isn't chasing ranking position, but treating it as a dataset on model behavior: manufacturer bias, search cost per generation, and stability under paraphrasing are variables you should be measuring in your own pipeline before trusting an LLM's answer as if it were neutral truth. The source provides separate analysis pages on the shifts between generations (Opus→Fable and Sol→Astra) and even a Family Feud-style game to test whether your guesses match the data. It's worth the visit as a methodology reference.