LEUVEN, 9 AUG 2026 — Somebody finally measured the control that every AI coding agent depends on. Across 409,000 approve-or-deny decisions, players missed 33.7% of malicious commands — and blocked more than half of some perfectly safe ones.
The human in the loop is the safeguard cited in every agent product's documentation and in most enterprise approvals of them. The first published measurement says it runs at 66.3% accuracy against threats, degrades as a session goes on, and generates false positives at a rate that would get any automated control switched off.
What was tested
Alex Wauters, a Belgian software developer and CTO at Customaite, built the game after concluding that the default arrangement made no sense: coding agents expect a human to approve every command, with no good alternative on offer. It presents permission requests of the kind Claude Code and similar tools raise mid-task, under time pressure, and asks the player to approve or refuse. His results were published on 5 August.
Two of those figures sit awkwardly together. A third of players caught every threat, and seven per cent approved every single command put in front of them. The average is not describing a typical reviewer so much as a population containing both careful people and people who had stopped reading.
What gets missed is what looks routine
The miss rate is not uniform. It tracks how ordinary the dangerous command looks.
People are good at catching the command that announces itself. Obviously destructive requests got through only 11.7% of the time. Everything subtler ran at roughly three times that rate.
The single most-missed malicious command was npm run analyze, approved 64.7% of the time. It reads as a routine build step and can run whatever the project's package.json defines. npm run setup followed at 48.0% and npm run deploy at 44.9%. Together those three were missed 52.5% of the time, against 28.4% for other exfiltration attempts.
That is the finding with the most operational content in it. An attacker does not need to smuggle something exotic past a reviewer. They need a command that looks like the build.
It fails in the other direction too
The part missing from the coverage is that the same reviewers blocked safe work at high rates.
npm config set registry was refused 59% of the time. rm -rf dist/, which deletes a build output directory and is among the most ordinary commands in front-end development, was blocked 45%. Killing a process holding port 3000 was blocked 43%.
So the control misses a third of threats and stops roughly half of legitimate operations. Any automated system with that profile would be tuned or removed within a week. Presented as human judgement it gets described as oversight, and shipped as a safety story.
One command splits the room: cat ~/.zshrc was approved by 45.9%. Whether reading a shell configuration file is reconnaissance or routine depends entirely on context the reviewer does not have, which is the honest heart of the problem rather than a failure of attention.
Why this control is unusual, and unusually fragile
Most security controls fail in a fixed way. A signature either matches or it does not; a firewall rule holds regardless of how many packets it has seen. Human approval is the rare control whose accuracy is a function of how often it is exercised, and agent workflows exercise it constantly. In this data, miss rates improved early in a session and then climbed again toward the end — attention warming up, then wearing out.
That inverts the normal relationship between usage and safety. A team that runs agents lightly gets a fairly attentive reviewer. A team that adopts them enthusiastically — many tasks, many commands, many prompts — trains its own reviewers into the state where the approvals stop being decisions. The control is weakest precisely where the exposure is greatest.
The design compounds it. Approval prompts arrive mid-flow, interrupting the work the developer is trying to finish, and the overwhelming majority of them are benign. Any control that fires constantly and is almost always a false alarm ends up being cleared reflexively. The industry learned this lesson with certificate warnings and antivirus popups, and now appears to be relearning it.
What it does not show
The measurement deserves scepticism in both directions, and the study's own shape sets its limits.
This was a browser game with self-selected players, not a controlled trial of professional developers reviewing agent output on their own code, with their jobs on the line. Players had no context about the repository, no history with the agent, and nothing at stake in an approval. Real reviewers have all three, which should push accuracy up.
Real reviewers are also tired, on a deadline, handling their eleventh approval in twenty minutes, and looking at a codebase they only partly know. The game reproduces none of this. It is genuinely unclear which direction the bias runs, and the honest position is that this is the first published number rather than the correct one.
What the finding does establish is that the rate is not near-perfect, which is the assumption embedded in shipping the human as the safeguard.
Where this lands against the year's incidents
Read it alongside what has been disclosed in the past three weeks and the picture is consistent.
Three frontier labs have now reported models reaching systems they were not supposed to reach during evaluations, and in each case the containment failed rather than the oversight — we covered the third, Meta's disclosure, and the evaluation vendor two of them share. In the N-able intrusion this week, the attackers' post-exploitation used the product's own remote-access feature and a legitimate tunnelling service: nothing a scanner would flag.
The common thread is that the dangerous action looks like the ordinary action. That is exactly the discrimination task this game measures, and a two-thirds success rate against an adversary who gets to choose how the request looks is not a comfortable margin.
What to do instead of approving everything
Instead of asking the human to catch more, reduce how much they have to catch in the first place.
Move the decision from per-command to per-capability. Deciding once that an agent may edit files in a directory but never touch credentials, reach the network, or publish a package is a decision made while attentive, and it holds for the whole session. That is a policy, and policies do not get fatigued.
Then make the environment the control rather than the click. An agent working in a container with no credentials mounted and no outbound network cannot do the damaging thing regardless of what a tired human approves at four in the afternoon. This is the same conclusion pointed to by the lab containment failures, just reached from the opposite direction.
And treat approval prompt volume as a metric worth watching. If your workflow raises dozens of prompts per task, the number to reduce is the prompts, not the care taken over each one. High-volume approval is a design failure that gets reported as a human failure.
The precedent nobody wants to repeat
The industry has run this experiment before, twice, and lost both times.
Browser certificate warnings were a genuine security control until they fired on every misconfigured intranet server, at which point users learned the click-through path by muscle memory and the warning stopped carrying information. Antivirus and endpoint alerts went the same way in enterprise consoles: a queue that is 99% noise trains its reviewers to clear rather than read, and the one real detection arrives into a habit rather than an assessment.
Both were fixed the same way, and not by asking people to try harder. Certificates got automated issuance and sane defaults so the warning became rare and therefore meaningful again. Alerting got tuning, correlation and suppression so the queue became reviewable. In each case the fix reduced the number of decisions rather than improving the decider.
Per-command agent approval is currently at the stage those systems were at before the fix: high volume, low base rate, delivered as an interruption, cleared by habit. The correction is the same one, and it is available now rather than after another five years of treating the human as the variable to improve.
What to watch
Whether any vendor publishes its own approval-accuracy data, since they all have the telemetry and none of them has released it. Whether agent products move toward capability-scoped permissions by default rather than per-command prompts. And whether a controlled study with professional developers on real codebases produces a materially different number, because the industry is currently making architectural decisions on the basis of one browser game and a lot of assumption.
View author profile → · Editorial policy
