9 SEP 2026 — Cycode has released agentic code scanning, reporting that it caught all six published CVEs in a benchmark of ten repositories, including two authorisation flaws it says no rule engine could express. The benchmark was designed by the vendor, and every target was a repository pinned to a known-vulnerable commit. The number that is missing is the false positive rate.
What was announced and what was measured
The product runs deterministic and agentic analysis as one system, deciding which engine to use where rather than shipping a standalone AI scanner. A companion feature, attack chaining, links related findings into multi-step exploit paths that per-finding severity scoring misses.
The benchmark covered ten repositories across six languages. Six were real applications pinned to a commit carrying a published CVE. Four were deliberately vulnerable applications with answer keys the engine was not tuned against. It caught all six of the published CVEs.
The two highlighted results are a broken access control issue in Gitea and a middleware authentication bypass in Next.js — both authorisation flaws, which is the category static rules genuinely struggle with.
Recall is the easy half of this problem
Finding a known bug in a repository checked out at the exact commit where that bug exists is a recall test. It answers whether the tool can see the thing when the thing is there.
The reason static analysis has a reputation problem in engineering teams is the other half. A scanner that reports fifty findings on a clean pull request gets muted within a fortnight, and the muting is rational, because the cost of triage falls on the same engineers every time.
So the figure that would predict whether this tool survives contact with a real codebase is how often it reports a vulnerability where none exists. The announcement does not give it, and six for six on known-vulnerable snapshots does not imply anything about it.
Why the authorisation claim is the interesting one
The specific claim worth taking seriously is that these two findings could not be expressed as a rule. That is plausible in a way most vendor claims are not.
Authorisation bugs are context-dependent by nature: the code is syntactically fine, and the defect is that a particular route does not check a particular thing for a particular class of user. A pattern matcher has no representation of who is supposed to be allowed to do what, because that lives in the application's intent rather than its syntax.
A model that reads the surrounding code can in principle infer that intent. Whether it does so reliably is a different question, and two examples do not establish it — but the category is the right one to be testing, because it is where rules provably cannot go.
Ten repositories is a small n
Six real applications is not a sample from which to generalise, and the four deliberately vulnerable applications are a weaker test again. Purpose-built vulnerable apps contain flaws placed to be found, in code written to host them, which is a different distribution from production software.
The claim that the engine was never tuned against those answer keys is the right thing to say and is not independently checkable. It is also worth noting that deliberately vulnerable applications are widely published and heavily discussed, so a model trained on public code has plausibly seen both the applications and their write-ups.
That contamination risk applies to the six published CVEs too. A CVE from a well-known project comes with an advisory, a patch commit and usually a blog post, all of which are training data.
What an independent evaluation would look like
Three properties, none of which requires the vendor to give anything away. Precision reported alongside recall, on repositories with no known vulnerability at the tested commit. A holdout of flaws disclosed after the model's training cutoff, which removes the contamination question entirely.
And a run by somebody else. The pattern we keep meeting is not vendors lying but vendors choosing the test — the same structure as a model launch whose headline benchmark depended on a harness the lab built, where the independent index told a different story.
Where it still points somewhere useful
Attack chaining is the part with the clearest theoretical case. Individual findings scored in isolation systematically miss the path that matters, because the exploitable route is often three medium-severity issues in sequence and no single one of them justifies attention.
That is a real gap in how vulnerability management works, and connecting findings is a sensible thing to build. It also creates its own measurement problem: a chained finding is harder to verify than a single one, and a false chain is more expensive to disprove than a false positive.
Nothing here says the product does not work. It says the evidence published is a recall result on a vendor-chosen set, and the question a buyer needs answered is what it does on their own code on a normal Tuesday.
View author profile → · Editorial policy
