SAN FRANCISCO, 30 AUG 2026 — Anthropic has published research in which automated agents proposed and tested methods for training a strong model from a weaker one, and beat what experienced human researchers proposed, on average within six hours. The paper also states the limit: it works only where progress can be scored automatically.
What the agents did
The problem set was weak-to-strong generalisation: how to train a capable model using supervision only from a less capable one. It is an open question in alignment research and a good test case, because any proposed method has a clear success metric: it either closes more of the performance gap or it does not.
The agents proposed ideas, ran experiments and iterated. On the open-weights models used for the test — Qwen 3-4B-Base as the strong model and Qwen 1.5-0.5B-Chat as the weak teacher — human researchers recovered 23 per cent of the total performance gap. The best automated method beat that.
The cost comparison is the figure that will travel: roughly US$4 per hour in API inference against about US$150 per hour for a human researcher.
The limit is in the paper, not in the criticism
The constraint that makes this result interpretable is stated by the researchers: the approach works on problems where progress can be automatically scored, and most real alignment problems do not fit that mould.
That is worth spelling out because it defines the boundary precisely. An automated researcher needs a signal telling it whether attempt 400 was better than attempt 399. Where such a signal exists, the agent can search a space far faster and far more cheaply than a person, and the six-hour result follows almost mechanically from that.
Where no such signal exists — which covers most of what alignment researchers actually argue about, including what a model should refuse, how to specify a value, and what counts as deception — the method has nothing to optimise against. The agents are not being modest about this. It is the first caveat in their own account.
Why the cost figure matters more than the win
Beating a human proposal on a benchmark is a headline. Doing it at a fortieth of the hourly cost is the operationally significant part.
At US$150 an hour, a researcher's time is rationed and search is narrow: an experienced person picks a small number of promising directions and tests them carefully. At US$4 an hour the economics invert, and exhaustive search over a wide space of mediocre ideas becomes cheaper than careful selection among a few good ones.
That changes what research on scoreable problems looks like. It does not make the human contribution redundant; it moves it. Choosing which problem to pose, constructing the scoring function, and deciding whether the resulting method means anything are all still human work, and the third is where the failure modes live.
What weak-to-strong generalisation is for
The problem the agents were set is worth explaining, because it is not an arbitrary benchmark and it is the reason this line of work exists at all.
Oversight assumes the supervisor can tell whether the supervised output is right. That assumption holds while models are worse than the people checking them, and it stops holding when they are better. Weak-to-strong generalisation asks whether a weaker supervisor can still elicit good behaviour from a stronger system, which is the technical form of the question of how anyone oversees a model they cannot themselves evaluate.
It is also, conveniently, scoreable. You can measure how much of the gap between weak-teacher performance and strong-model potential a method recovers, which is what the 23 per cent figure is. That is why it was chosen as the test case, and it illustrates the constraint rather than escaping it.
So the result is narrower than a general claim about automating research, and pointed at something that matters: the one alignment problem that happens to have a number attached is also one of the more consequential ones.
Scoring functions are where this goes wrong
If a method's value depends on a metric, the method will be optimised toward the metric rather than toward the thing the metric stands for. That is not a hypothetical risk in machine learning; it is the default outcome.
An automated researcher searching hard against an alignment benchmark will find whatever the benchmark actually rewards, including artefacts of how it was constructed. A human researcher searching the same space is slower and, being slower, is more likely to notice that a result looks wrong.
The paper's own framing accommodates this: automated research is presented as scaling scalable oversight, which is a claim about throughput on a well-posed problem, not a claim about judgment. The distinction between those two is the whole of the caveat.
How to read AI safety results
This desk has its own record on the subject and it is worth naming. We once reported a validation of Constitutional AI that did not exist, and published the retraction rather than quietly amending it.
The lesson from that error applies directly here. A safety result is easiest to over-read when it is favourable to the lab that produced it, and the check is always the same: what was measured, on what population, against what baseline, and what did the authors themselves say it does not cover.
On those four questions this result holds up unusually well. The task is specified, the models are open-weights and named, the human baseline is quantified at 23 per cent, and the limitation is stated in the paper rather than extracted from it by a critic.
RECATOOLS discloses that Claude, an Anthropic product, is used in producing content for this publication. That is a reason to read this article's account of Anthropic's own research carefully, and a reason we have leaned on the paper's stated limits rather than on its headline.
View author profile → · Editorial policy
