The pitch for AI-powered code security review arrived in 2026 with the numbers you would expect from a category that suddenly has an urgent buyer: fewer false positives, more true positives, faster fixes, benchmarks in the fifties. The most useful thing published about it was not a press release. It was Semgrep benchmarking its own system against frontier models on identical tasks, and publishing the table.

The volume problem is real

This is not security tooling chasing a trend. The inputs changed faster than the controls did.

Cursor’s 2026 Developer Habits Report found that the top 1% of AI-active developers produce 46 times more AI-assisted lines of code per day than the median active user, and merge 15 times as many pull requests per week. Average weekly code output per developer rose from roughly 3,600 lines in January 2025 to 8,600 by May 2026. Cursor’s own chief technology officer put it more bluntly in a May interview: the traditional product development line had “totally combusted.”

Application security teams report seeing ten times the vulnerabilities they were handling two years ago. And the pressure is not only on the scanning side. Cursor’s telemetry has the share of agent-authored changes reaching a commit without a human reading the diff climbing from 7% on January 1 to roughly a third by mid-2026. The median developer on Cursor generates about 700 lines of code a week with it, and the top percentile generates 30,000 to 40,000.

Cursor also reports the distribution of that output is unusually unequal, with a Gini coefficient of 0.77 for AI-generated code, comparable to the most unequal national income distributions. Whether that represents 46 genuinely better engineers or 46 people generating more unreviewed code is a fair question nobody has settled, and it is the question your security posture actually depends on.

What the actual benchmark table looks like

Semgrep published a benchmark methodology in June 2026 that compared four ways of finding the same vulnerability classes, deliberately using the same underlying model so that the harness, not the model, was the variable. The comparison used Claude Opus 4.8 and GPT 5.5, including a steel-manned guided prompt and Anthropic’s own Claude Security harness.

The headline is not that one vendor beat another. It is that a frontier model given a well-written prompt and told to find vulnerabilities scores an F1 in the high teens to low twenties.

Precision is essentially flat across all four. Recall is where the gap is, and recall is the metric that finds the bug. The stated reason is that Multimodal looks at more of the right places, because static analysis narrows the candidate set before the model reasons. Across three generations of Opus, Semgrep reports three to eight times the recall of the same model on its own, at comparable or better precision.

The cost column is the more interesting part of that table. Claude Security, Anthropic’s own harness running on the same Opus 4.8, cost $72.55 per true positive against Multimodal’s $0.62, over a hundred times more, for slightly worse precision and recall. Wrapping a model in scaffolding does not automatically make it efficient, which is a result worth knowing before anyone buys an “AI security review” line item on a budget.

Semgrep’s public metrics page gives the production figures on the same system: more than 3,500 customers, 6.5 million findings analyzed, an average 60% reduction in findings filtered out as noise, a 96% human-agree rate, and a median time to resolution 22% faster than baseline. Launch marketing in March claimed up to eight times more true positives with 50% fewer false positives than foundation models alone and dozens of zero-days discovered at customers.

Where deterministic analysis still wins

SAST has a specific, unglamorous advantage: it does not get tired, does not get impressed by a plausible function name, and produces the same result on the same input every time you run it. Detecting SQL injection, SSRF, and exposed secrets through dataflow analysis has been reliable for years for a reason unrelated to model quality, and both vendors agree with this framing rather than fighting it.

The useful way to read the 2026 architecture is that nobody credible is claiming patterns died. Semgrep’s Multimodal is the Pro engine’s precise program analysis with LLM reasoning layered on top, because traditional rule-based scanning has always struggled with business logic flaws: IDORs, broken authorization, and authentication bypasses that require understanding intent. Snyk’s DeepCode AI is built the same way, combining frontier models fine-tuned with security context on top of a symbolic analyzer that the company says has processed more than 25 million dataflow cases across 19-plus languages and produces autofixes it rates at 85% accuracy.

Snyk’s Evo Continuous Offensive Security goes one step further and uses a dedicated validation model as an independent judge that has to confirm exploitability before any finding surfaces. That design choice is an admission of what the industry is generally reluctant to say out loud: a security finding produced by a model needs a second, independent check before it is worth an engineer’s afternoon.

The asymmetry is what should drive the purchase decision. A missed vulnerability is something you discover later. A false positive is an afternoon. Tools that measurably reduce the second while claiming large multiples on the first are worth adopting, and on this benchmark the precision numbers are honest across the board while the recall numbers are where the differentiation lives.

The part nobody’s benchmark covers

Every accuracy figure above is about code in your repository. The more interesting shift is that the attack surface moved.

Semgrep’s own account of why it built an IDE-time scanner is that if an AI chat can walk a product manager, a marketer, a sales engineer, or an intern through installing an IDE and opening a terminal, then that person is now writing code. The people writing code are no longer the ones who went through a security review process, and no scanner you run in CI was built with that in mind.

The second shift is supply chain. A malicious package does not need your code path to be executed. Once npm or pip finishes installing, the machine is already compromised, which means scanning after install is too late to be a control. And when an agent resolves dependencies, that resolution is no longer a developer’s deliberate choice with a human reading the package page.

This is what Semgrep Guardian, launched on June 23, 2026, is aimed at: scanning every file an agent writes, in the IDE, covering OWASP Top 10 issues, malicious open source packages, and hardcoded secrets. It ships as an MCP server with IDE hooks and agent-callable skills, with official integrations for Cursor and Claude Code and support for Copilot, VS Code, Windsurf, and Amazon Kiro. Semgrep reports over 3 million scans a week across its customer base with 95% completing in under 5 seconds, which is the actual requirement, because a check that takes 40 seconds does not run inline.

That last point is worth checking against your own tooling, because the integration quality varies in ways that matter. The documentation notes Claude Code connects to a hosted remote Guardian server by default, needs no local install at all, and always runs Guardian’s default ruleset rather than your organization’s custom policies. GitHub Copilot does not expose a post-write hook, so on that IDE the scan only happens when you ask for it.

Where does each check belong, and when? In the editor before the commit In the pull request before review After install already compromised Deterministic rules never miss these Model reads the logic you care about Too late, but still worth having

VERDICT

Do not replace your SAST gate with an AI reviewer. The one number that settles it is in Semgrep's own benchmark: a frontier model with a well-tuned prompt reaches 12.6% recall on real vulnerability classes while the hybrid system reaches 43.5%, with precision essentially identical for both, so the model is not the part that matters and the harness is. Run deterministic analysis to narrow, then a model over the logic, and treat any vendor accuracy percentage you cannot reproduce as marketing. Spend your attention on the two shifts the benchmarks do not cover, because they are where the new risk actually is: an IDE-time gate that runs in under five seconds, since the share of agent-written changes reaching a commit unread is climbing toward a third, and dependency scanning at resolution time, because a malicious package does not wait for your code path to execute. And budget for the verification step vendors still cannot automate, because Snyk shipping a second model to independently confirm exploitability is an admission that a model-generated finding is a hypothesis, not a result.

Sources