Claude Code, Codex, and Cursor Choose Different Tools: What 17,000 Runs Reveal

When you ask a coding agent to "pick the best database and implement it," the one who actually decides is the agent, not you. And the decision changes depending on which agent you use, which language is in the repository, and how the vendor presents pricing on the landing page. That's what a study by Armature, published in September 2026, tried to measure at scale: 16,893 sessions with Claude Code, Codex, and Cursor installing third-party services in real repositories.

A caveat that comes from the source itself and matters for reading the numbers with caution: Armature (YC P26) sells growth services for dev tools. In the words of co-founder screm on the Hacker News thread, "This study is part of our broader work on how to influence coding agents choices and get products picked". In other words, the company's business is precisely helping products get picked by agents. The raw data and traces are public, but the incentive exists.

What's at Stake, and for Whom

The practical question for anyone building software is direct: can you trust the agent's judgment on which third-party service to install? The study doesn't assess which agent "writes better code" in general. It measures a specific and increasingly common task: selecting and implementing a service (database, email provider, storage, payment, deploy) within an existing codebase.

This focus is relevant because it's the step everyone outsources to the agent, from the vibe coder with no software background to the senior engineer. Armature set up 75 repositories in 10 languages, with fake company names, fake git histories, fake API keys, and real lockfiles checked against registries like npm. On top of that it ran 1,163 prompt variations, embodying four profiles: vibe coder (only describes the symptom), junior engineer (names the category), senior engineer (is precise about requirements), and large-company engineer (details compliance and procurement).

Of the 16,893 runs, Armature considered 5,292 sessions valid to publish in this first batch, across 51 codebases and 18 sectors. The question for you, the reader, comes down to this: is your agent a reliable decision-maker, or a bias vector that installs whatever was best positioned on Google that day?

Comparison Criteria in Table Form

| Criterion | Claude Code | Codex | Cursor | |---|---|---|---| | Web search usage | ~30% of sessions | 94% of sessions | 2/3 of sessions | | How it searches | Browses 3x more pages than Codex when it searches | Uses operators (site:) in 9 of 10 searches | Bases decision on the web in 2/3 of cases | | Tendency to build in-house | 19% | 10% | 10% | | Agreement among the three (same cell) | 42% of cases | 42% of cases | 42% of cases | | Example of divergence (voice agents) | Twilio | OpenAI Realtime API | Vapi |

All cells come from Source 1. Where the study did not separate Cursor from Claude/Codex on a criterion (the in-house case, reported as "10%" for both), the table repeats the number given.

Analysis by Criterion

Decision sources diverge. Codex is the most search-dependent: 94% of sessions, and in 9 of 10 queries it uses operators like site: to focus on trusted domains or dig into a specific solution. The example given in the source is literal: site:auth0.com password reset MFA social connections. Claude Code does the opposite, trusting its own priors and only searching in ~30% of cases, but when it does search it browses 3x more pages than Codex. In newer sectors, like sandboxes, where its priors are weak, Claude rose to ~80% search usage. Cursor sits in the middle, basing its decision on the web in two-thirds of sessions.

The three rarely agree. In only 42% of cells do all three agents pick the same tool. The voice agents case is illustrative: Claude Code goes with Twilio, Codex picks the OpenAI Realtime API, and Cursor prefers Vapi. Three agents, three answers to the same need.

Build versus install. Claude Code builds in-house almost twice as often as the others: 19% versus 10% for Codex and Cursor. A methodological detail explains part of this: when the orchestrator asked for immediate implementation without allowing questions, the agent tended to build everything internally because it couldn't ask for authorization to use a third-party service. Armature added a "simulated human" to the loop (a role played by Gemini 3.7 Flash) to make the scenario realistic. In the object storage experiment, Cloudflare R2 started winning sessions where the agent had previously always used Amazon S3.

The Repository's Context Matters More Than the Agent

This is the finding that most changes your routine. With the same request across four repositories in four different languages, four distinct email provider winners emerged:

| Language | Email winner | Score | |---|---|---| | TypeScript | Resend | 55/89 runs | | Python | SendGrid | 22/24 | | Go | Postmark | 20/24 | | Java | Azure ACS | 22/23 |

The same holds for deploy: Vercel won in TypeScript repositories (and in 100% of cases when Next.js was present), but was never recommended in Python repositories, where Render dominated. The practical takeaway is that the agent reads your stack as a strong decision signal. If you work with Go or Java, don't expect the same recommendations that circulate in TypeScript examples.

Being Mentioned Isn't Being Chosen

Another useful pattern: famous names come up in almost every conversation and are almost never installed. In the payments sector, PayPal was mentioned 139 times and chosen zero (Stripe won 124 of those 139 sessions). Adyen was mentioned 175 times and chosen 3. LangChain is the most-mentioned framework (194 mentions) and was chosen 4 times. Netlify: 152 mentions, 6 choices. Supabase is the most-mentioned database (242 mentions) and was still widely dominated by Neon.

Details on the vendor's page turn into tiebreakers. According to Armature, Mailgun frequently lost to Postmark when agents read "1-day retention" on the free plan. Supabase almost always lost as a database because it presents BaaS features the agent didn't ask for (auth, storage, realtime) bundled into the price, while the agent was only looking for a database.

Some markets are dominated, others contested. Stripe won in 9 out of 10 cases, losing only in EU-regulated scenarios to specialists like Paddle and Mollie. Neon took 66% of the database category. In file storage, Amazon S3 dominates with 45%, followed by Azure and GCP with 20% each. In email, Resend and Postmark lead with 35.6% and 27.4% installation rates.

Verdict by Use Case

There's no single winner, and the study makes that explicit by showing that the three agents disagree in most cells. Here's guidance by profile:

• You're a vibe coder or junior and accept the first recommendation: know that the agent may be installing whatever was best positioned on the web that day, not what best serves your case. Codex and Cursor, more search-dependent, inherit more bias from what's published. Explicitly ask for alternatives.
• You work on a stack outside the TypeScript/Next.js axis: expect recommendations different from those in tutorials. In Python, Render won deploy; in Go, Postmark won email. Don't copy a TS example's choice without checking.
• You want the agent to consider a specific service: Codex responds well to precise instructions because of its heavy use of site:. A senior who already knows what they want tends to have more control by guiding the prompt.
• You care about predictable cost or compliance: the data shows that how the price is presented weighs more than the price itself. Review the diff and the agent's justification before approving, because a feature bundle or a retention line item may have disqualified a better option.
• You need a deterministic, auditable outcome: running the same task on two different agents can yield two different stacks. Standardize on one agent per project if you want consistency.

What Can't Be Claimed Yet

The study has limits that matter for any decision based on it. First, the conflict of interest: Armature sells exactly the service of getting products picked by agents, so the numbers conveniently support the thesis that the choice "can be influenced." Second, the methodology uses Gemini 3.7 Flash as both the orchestrator (the "simulated human") and the judge that validates sessions and identifies the winner, meaning one AI model evaluating others. Third, 5,292 sessions were deemed valid out of nearly 17,000; Armature says it may publish the rest in a second batch, but for now only a fraction of the conclusions has been drawn.

The study also doesn't measure the quality of the generated code, nor whether the "winning" choice was actually the best technically. It measures what the agent installs, not whether you should have accepted it. And the market scores (Stripe 9 in 10, Neon 66%) reflect agents' behavior today, with the priors and web index of that moment, something that changes with every model release.

On the Hacker News thread, unease about the direction showed up. User ai_critic summed up the fear of repeating the SEO era:

Can we not encourage the same strip-mining and ad and SEO bullshit that previously ruined the last decade+ of the Internet?

>

-- ai_critic

Meanwhile, jdw64 points to a possible concentration effect: "the tools that AI gives the highest priority to are the ones people already choose. There might be a concentration effect toward the tools that AI selects". If the agent always installs Stripe and Neon, those become even more the default, and alternatives disappear from the radar before they get a chance.

The complete traces, with prompts, reasoning chains, and applied diffs, are public on Armature's website for anyone who wants to audit them independently. No official image or video was released in the source material to illustrate this piece.