METR says Anthropic's AI progress has roughly doubled in speed, and most of the evidence is hidden

On 5 November METR, an independent non-profit that tests AI systems for dangerous capabilities, published an assessment of how much AI is speeding up AI development inside Anthropic. Its headline is that Anthropic is "likely" improving its models at around twice the rate it managed between March 2024 and October 2025, and that most of the speed-up probably comes from Anthropic's heavy use of AI in its own research. It is the first outside view of a question that sits behind many of this year's debates about slowing down. It also comes with an unusual warning: the public cannot check much of it.

What METR did

METR staff, about two and a half full-time equivalents over eleven weeks from 12 August to 29 October, were given access to internal Anthropic data. That included benchmark scores for internal models, headcount, lines of code, spending on AI agents, internal surveys and around 15 staff interviews. They wrote a roughly 100-page full report for Anthropic and a shorter public version. METR says it did not have query access to unreleased models, worked on Anthropic's devices, and took no payment, though Anthropic provided free tokens and laptops. It describes the access as beyond what it expected, but narrower than having the run of the company.

The three conclusions

1. Progress is faster. As of October 2026, METR believes Anthropic is improving model capabilities on "well-specified" tasks, the kind that are easy to score automatically, at about twice its earlier rate. It says it would not be surprised by anything between 1.3 and 3 times.
2. AI is the main reason. METR guesses that a majority, perhaps around 80%, of today's acceleration is down to AI doing AI research. It stresses the uncertainty: Anthropic's own staff did not agree on this, and it thinks other causes mattered in late 2025.
3. No obvious brake. METR saw no clear bottleneck, or steep diminishing returns, that would stop the speed-up compounding over the next year, so it sees a significant risk of further acceleration. It adds that it may have missed bottlenecks.

The numbers behind it

The central measurement uses Anthropic's own version of the Epoch Capabilities Index, a way of combining many benchmarks into one score, which METR calls AECI. Since October 2025, METR estimates, scores have grown 1.9 times faster than before, with a 95% confidence range of 1.5 to 2.8 times. In METR's words, that means roughly two years of the old pace of progress in the past year.

METR says it is confident that the acceleration is not because Anthropic scaled up training compute faster, though it cannot share the details. The report also includes some striking self-reports. In internal Slack polls, Anthropic staff estimated the speed-up from the most recent model at about 12 times in October. METR calls these "very loose evidence", since the votes come from a self-selected group giving low-effort answers, and says personal speed-up is not the same as faster overall progress. Interviews with researchers were mixed: answers centred on roughly a doubling, some people credited AI with most of the change, and at least two pointed mainly to other factors such as hiring.

The assessment says AI helps mainly by assisting people, for example by writing code and running experiments, rather than by running research on its own. But researchers described "signs of life" for fully autonomous contributions that could come soon.

Why the caveats matter

METR gives several reasons the true figure could be lower or higher:

• Benchmarks are an imperfect stand-in. If Anthropic's models have improved faster on neatly scored tasks than on messy real-world ones, the real-world speed-up would be smaller. The estimate also ranged from 1.1 to 2.9 times depending on how the benchmark scores were modelled.
• Anthropic's models grew more slowly than the industry's best before late 2025, which could make its earlier pace look unusually low. When METR includes other labs' models, the estimated acceleration falls to about 1.4 times, and METR says either figure could be the better guide.
• On the other side, the benchmarks may be saturating. Anthropic's newest internal model scored above 50% on every benchmark in the index, which makes measurement harder, and METR thinks real-world impact could jump suddenly once models cross certain thresholds.
• METR also includes a conflict-of-interest note: one team member had discussed a role at Anthropic, received an offer and declined it, so that person's work was double-checked.

The redactions

The most unusual part is METR's description of what it could not say. Anthropic had the right to remove any non-public information, and METR says Anthropic redacted "a majority of the evidence we found most informative". The redacted material includes the more detailed estimates of how much AI contributes at each stage of training, breakdowns of progress by cause, the analysis of bottlenecks, and forecasts for future models. Anthropic described the redactions as "aimed at removing confidential information that could accelerate other AI developers", and METR says they were limited to that kind of information, though it does not judge whether they were justified.

The consequence, METR says, is that the public "has to rely on METR's judgment much more than would be ideal". The report does not achieve what it calls transparency about evidence. METR also notes that Anthropic cancelled one part of the original scope, a check on how much computing power goes to capabilities versus safety work, and said it would revisit it later.

How it fits the wider debate

The timing matters. In September Anthropic's chief executive called to pace the frontier and said independent evaluators should sit inside labs. METR says this assessment is separate from the evaluator access Anthropic announced on 15 October and is not the unmediated, "employee-like" access that plan describes. The Verge reports that an Anthropic co-founder, Sam McCandlish, says the company plans to publish its own analysis of the speed of AI progress inside the company. Until then, the best public evidence is a summary written by outsiders who are legally limited in what they can show.

For readers, the sensible takeaway is neither alarm nor dismissal. A respected evaluator, with access to data nobody else has seen, now says the feedback loop between AI and AI research is probably real and strengthening. Its own error bars are wide, and the strongest evidence is hidden.

What to watch

• Anthropic's promised report on its own progress, and how much of the redacted evidence it releases.
• Whether other labs allow similar assessments, and on what redaction terms.
• Whether the next Anthropic models show a faster pace on messy, real-world tasks, not only on benchmarks.
• Evidence that AI is moving from assisting researchers to running whole research projects on its own.