BURLINGTON, 17 AUG 2026 — Veracode tested more than a hundred language models on code-generation tasks and found an average security pass rate of 56 per cent, unchanged from a year earlier. The models produce syntactically valid code almost every time. On the tasks where security matters, they introduce an OWASP Top 10 vulnerability roughly 44 per cent of the time.
The failure rate itself is less interesting than the fact that it is static. A year of industry effort has not moved the needle.
The measurements
The report covers more than 100 models across four testing snapshots. The tests were run without security-specific prompting. This is an important detail: the results measure a model's default behaviour, not what it might produce if specifically asked to be careful.
Four vulnerability categories were tested — SQL injection (CWE-89), cross-site scripting (CWE-80), log injection (CWE-117) and use of insecure cryptographic algorithms (CWE-327). The best performer was GPT-5.5 at 68 per cent and the weakest was Alibaba's Qwen3.7-max at 50 per cent, with six of eleven models bunched between 50 and 53.
Every lever the industry pulled failed to move it
The report's internal comparisons are more useful than the headline figure, because they test several industry assumptions about what improves security.
Coding-specialised models scored 51 per cent against 52 for general-purpose models. A model trained specifically to write code is not measurably safer at writing it, which is an uncomfortable result for a product category sold on exactly that specialisation.
Scale did almost nothing. Models above 100 billion parameters reached 53 per cent, against 51 for medium and 51 for small. Whatever capability grows with size, secure-by-default coding is not obviously part of it.
The single exception is reasoning. Reasoning models scored 56 per cent against 51 for non-reasoning ones — five points, the only lever in the report that visibly moved. The finding is modest, but it is there: the five-point gain suggests some of the gap is attention rather than knowledge, and that a model can sometimes catch itself if given room to think.
Chris Wysopal, Veracode's co-founder and chief security evangelist, put the shape of it plainly: models are close to syntactically perfect and still failing on nearly half the tasks where security is needed.
The Java result cuts both ways
The widest performance gap in the report is between Java (30 per cent pass rate) and Python (63 per cent). The easy conclusion is that Java is the dangerous choice, but the data is more nuanced.
Java is also described as the only language with a clear, consistent upward trend over the past year. So the language with the worst absolute result is the one improving, which suggests the gap is about training data and idiom rather than anything intrinsic to the language.
For anyone running an enterprise Java estate, though, the absolute number is what governs today. Two out of three generated snippets carrying an OWASP Top 10 issue is not a code-review problem, it is a throughput problem. The required review capacity scales with generation volume, which is exactly what AI assistants are built to increase.
This is the same problem we found at the other end
We reported this month that only about a quarter of AI-generated security patches were fully clean, with the rest either incomplete or introducing something new.
Put the two together and the picture is coherent rather than contradictory. Models write code that compiles and is insecure about half the time, and when pointed at a vulnerability they fix it correctly about a quarter of the time. The generation and remediation ends of the same pipeline fail in the same way, and the common factor is that neither task is checkable by the model that performed it.
The practical advice, then, remains unglamorous: treat all AI-generated output as unreviewed code from an unknown contributor. Put a scanner and a human between it and production.
Why this lands harder in this region
The exposure in Southeast Asia is structural rather than technical, and it follows from who is adopting these tools fastest.
Small development shops, agencies and in-house teams across the region have taken up AI coding assistants quickly, for the obvious reason that they compress the cost of delivery. Many of those organisations have no application security function at all, no static analysis in the pipeline and no security review step that a 44 per cent defect rate would run into.
A large enterprise with a mature scanning pipeline absorbs this finding as a workload increase. A twelve-person shop shipping a government portal absorbs it as vulnerabilities in production. The tool is the same and the outcome is not, and the difference is entirely the review capacity around it.
There is a procurement consequence too. Anyone commissioning software in this region — a bank, a ministry, a hospital group — is now buying output that was partly machine-written, whether the contract says so or not. Asking a vendor what proportion of delivered code is AI-generated and what scanning it passes through is a reasonable question, and almost nobody is asking it.
What we could not establish
The report does not establish whether its no-security-prompting methodology reflects real-world use. Measuring default behaviour is correct for a baseline, but careful teams prompt for security. This makes the 56 per cent pass rate a floor, not a field measurement; the report does not seem to publish a figure for prompted output.
Also unestablished: the size of the task set behind each model's score; whether the eleven models quoted individually are a representative subset of the hundred-plus tracked; whether the four vulnerability categories were equally weighted, which matters because insecure cryptography and SQL injection carry very different real-world consequence; and how the pass rate varies with task complexity rather than language.
We did not use one widely quoted figure: the claim that AI now writes roughly half of all committed code. While it appears in some coverage of this report, the report itself does not make this measurement, and we could not source it.
What to watch
The next snapshot is the thing to look for, because a fifth data point on a flat line is a much stronger statement than a fourth. Two years of no improvement would suggest the problem is not going to be solved by better models.
Then watch whether any vendor publishes a security pass rate as a product claim. Every model card leads on capability benchmarks; none leads on this. The first vendor to compete on it would change what the category optimises for, and the absence of that competition is itself informative.
Finally, watch whether the reasoning advantage holds as reasoning becomes standard. If the five-point gain is attention rather than knowledge, it should shrink as every model reasons by default — and if it does not shrink, that points at something more durable than prompt discipline.
View author profile → · Editorial policy
