Ai2 study shows LLMs fail to judge whether a research idea is original

The pattern known as LLM-as-judge has become a common piece in any pipeline that uses AI to evaluate other AI: quality scores in RAG, pull request triage, duplicate content detection, ranking of agent responses. A study published by the Allen Institute for AI (Ai2) on the Hugging Face blog on October 27, 2026, written by journalist Kyle Wiggers on behalf of the institute's team, tests this pattern in a specific domain, but with implications that go far beyond it: can an LLM reliably judge whether a research idea is original?

The paper, titled "Old Ideas, Novel Problems," stems from a practical problem at Ai2 itself. The institute develops systems like DataVoyager, AutoDiscovery, and Theorizer, which help researchers explore data, generate hypotheses, and synthesize scientific literature. For these systems to be of any use, the ideas they produce need to be genuinely new, not just repackaging of existing work. This judgment is typically delegated to another LLM, the so-called "novelty judge," and it is exactly this piece that the study decided to audit.

How the benchmark was built

The team built an automated pipeline on top of ICLR 2026 peer reviews, but avoided using paper acceptance as a proxy for novelty (a paper can be accepted without being particularly original, if it is strong in other respects). Instead, it mined the reviews themselves for passages where reviewers explicitly discussed the originality of the contribution.

From this came two sets: a high-novelty pool with 154 ideas from papers accepted in the top 10% of their area, with reviewers who praised the novelty without contestation; and a low-novelty pool with 145 ideas from papers rejected at the bottom of the ranking, with reviewers who questioned the originality without any defense. These two groups form the Human-Only scenario, which tests whether an LLM judge can distinguish human ideas with strong evidence of higher or lower novelty.

But novelty judges are increasingly used to evaluate ideas generated by AI, not just by humans. That's why the team created a second scenario, Human+Generated, pairing the same 154 validated human ideas with 154 ideas produced by a simple LLM generator, without literature search, tools, or iteration. An expert manually reviewed 30 of these pairs and preferred the human idea in all 30 cases, not considering any generated idea to be novel.

A word swap in the prompt cuts accuracy in half

The study's most striking finding isn't about which model is best, it's about how much the evaluation depends on how the question is asked. The team tested six judges (GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5, and Claude Opus 4.6), varying one factor at a time: prompt, reasoning budget, access to related papers, idea format, and how many evaluations were combined into the final verdict.

In the most extreme experiment, GPT-5.4 evaluated the same Human+Generated pairs with two different prompts. One told the model that reviewers had already identified which of the two ideas was novel and asked it to find which one; the other simply asked which idea was newer. The swap dropped accuracy from 93.5% to 40.9%, below chance, and the model changed its verdict on more than half of the pairs it had previously gotten right.

In the Human-Only scenario, the same prompt swap moved accuracy by only about 7 percentage points, although Claude Opus 4.6 swung by 21 points on this same set. In other words, a judge that appears stable when evaluating human-written research can behave completely differently when faced with AI-generated ideas, which is precisely the use case that matters most for those automating scientific discovery.

If AI is going to play a larger role in scientific discovery, we need more dependable ways to tell whether it is actually proposing something new.

Ai2 Team, authors of the study "Old Ideas, Novel Problems"

There is no universally best prompt. Removing an instruction that warned the judge not to treat trivial contributions as novel improved accuracy for some models and worsened it for others. A prompt tuned for one model simply doesn't transfer to the next, which is a serious problem for anyone building automated evaluation with the intention of switching models in the future.

More search and more reasoning didn't help

The obvious hypothesis would be that giving the judge access to the literature and more reasoning budget would solve the problem. The study tested exactly that: an agentic judge based on GPT-5.6 Sol, with extra-high reasoning budget and live iterative search restricted to arXiv, was compared with standard retrieval judges that received five pre-selected abstracts.

The result reversed expectations. The agentic judge reached a macro-F1 of 0.78, 11 points behind Claude Opus 4.6 (0.89), and in the same range as simpler configurations with Claude Opus 4.5 and GPT-5.4, except costing about 10 times more per decision at current API prices.

The same pattern repeated when comparing specialized systems against a simple baseline:

The cheaper baseline beat both systems built specifically to judge novelty, by a considerable margin and at roughly one-thirtieth the cost of the most expensive competitor. For those deciding on evaluation infrastructure in production, this is a hard fact: pipeline complexity is not a proxy for judgment quality.

What this means for everyday users of LLM-as-judge

This study is about scientific novelty, but the pattern it takes apart is the same one used in RAG evaluation, automatic triage of duplicate PRs, scoring of agent responses, and any CI pipeline that uses an LLM to decide whether an output is "good" or "new" enough. Three lessons carry over directly to that context.

First, the judge's prompt needs to be validated against a known ground-truth set before going into production, and revalidated every time the underlying model changes. A prompt that works with Claude Opus 4.6 may not work with GPT-5.4, and the difference doesn't show up until someone tests both side by side with labeled data.

Second, giving the judge more tools (search, extended reasoning, agents) is no guarantee of greater precision, and definitely no guarantee of better cost-effectiveness. Before paying 10 times more for an agentic setup, it's worth measuring whether it actually outperforms a simple prompt with multiple calls and majority voting, which in the Ai2 study was the most efficient option.

Third, the judge needs to be tested on the type of content it will actually evaluate. Validating an evaluator only with examples written by humans, when it will mainly judge AI-generated content (such as PRs made with copilot assistance, or responses generated by another model), is the same methodological error the study points out: stability observed in one scenario doesn't transfer to the other.

Where this judgment still shouldn't be automatic

Ai2 made the benchmark, data, and pipeline code available alongside the paper, open for anyone who wants to reproduce the test in other scientific communities or train more robust judges. But the paper itself is clear in stating that LLM-based novelty evaluation "remains an important open problem," and that the measured instability doesn't disappear with more model scale or more search.

For any team that currently uses an LLM as the final judge of quality, originality, or duplication without its own validation set behind it, the message is direct: confidence in that verdict shouldn't come from the model's name, but from explicit prompt-sensitivity tests, done with the data the system will actually encounter in production.