In early 2025, a mid-size fintech quietly rolled back three weeks of AI-assisted commits after a routine SOC 2 audit flagged authorization logic that bypassed row-level security in Postgres. The engineers who wrote it didn't know. The AI that suggested it didn't know either. That incident is a preview of what 2026 looks like at scale, and most CTOs still don't have a framework for it.
AI code risk is not one number on a dashboard. It is at least four distinct categories of exposure, each with different detection surfaces, different ownership questions, and different mitigation strategies. Collapsing them into a single "AI risk score" is the kind of abstraction that gets you to the audit, not past it.
The Four Categories That Actually Matter
Correctness risk is the one everyone talks about. The model hallucinated a library method, the unit test was too narrow to catch it, the thing shipped. Correctness failures are the most visible and, paradoxically, the easiest to address with existing tooling: broad test coverage, type enforcement, static analysis. If your AI-assisted PRs are failing at a higher rate than hand-written ones, that signal is already in your CI data. Look at it.
Security risk is the category that will define careers in 2026. AI models are trained on public code, which includes every insecure pattern ever published to GitHub. The authorization bypass in the fintech incident wasn't a hallucination; it was a plausible pattern the model had seen hundreds of times in tutorials that predated modern security standards. The model was confident. The suggestion was wrong.
This is where static analysis tools like Semgrep earn their keep, but they only catch known patterns. What they miss is context-specific logic: a route that looks fine in isolation but bypasses a middleware guard in your specific framework configuration. That requires knowing your architecture, not just the syntax.
Intent drift is underappreciated. This is the gap between what a developer asked for and what the AI produced, compounded over many small acceptances. Each individual acceptance looks reasonable. The cumulative effect is a feature that technically does what the code says, but not what the product spec said. Intent drift is almost impossible to catch at the diff level. You catch it at the integration test level, or you catch it in production.
A practical heuristic: if a developer cannot explain, in plain language, why each non-trivial block in an AI-assisted PR exists, the PR should not merge. This sounds obvious. It is not enforced anywhere near enough.
Dependency and provenance risk is the newest category most teams haven't formalized yet. AI assistants routinely suggest package additions. Some of those packages are abandoned, some have known CVEs, and a small but growing number are outright typosquats that a model pulled from low-quality training data. Your existing software composition analysis pipeline should already catch CVEs, but it won't catch a package that was clean when the model was trained and compromised last month.
Here is a minimal check worth running on any AI-assisted PR that adds a dependency:
import subprocess
import sys
def check_package_freshness(package_name: str, max_days_since_release: int = 180):
result = subprocess.run(
["pip", "index", "versions", package_name],
capture_output=True, text=True
)
# parse latest release date, compare to today
# flag if last release > max_days_since_release
# also flag if download count < threshold (use PyPI JSON API)
pass
This is not a complete solution. It is the starting point for a policy, which is what you actually need.
The Metrics That Predict Incidents
Most engineering teams tracking AI code risk are measuring the wrong things. PR acceptance rate for AI suggestions tells you about developer behavior, not risk. Lines of code generated per day tells you about throughput. Neither predicts incidents.
The metrics that correlate with production incidents, based on patterns visible across the industry, are:
- Post-merge churn rate on AI-assisted files: How often do files with significant AI contribution get modified within 14 days of merge? High churn is a proxy for latent defects.
- Test coverage delta on AI-assisted PRs: Are developers adding tests when they accept AI suggestions, or just accepting the suggestion? A coverage delta below zero on merge is a red flag.
- Security finding rate by contribution type: Semgrep and similar tools should tag findings by whether the flagged code was AI-assisted. If AI-assisted code has a 3x higher finding rate than hand-written code in your codebase, that is a calibration problem, not a tooling problem.
- Intent mismatch rate: This requires your team to log, even informally, how often a completed feature required significant rework because the AI-generated implementation didn't match the original intent. Hard to collect, worth collecting.
The post-merge churn rate is the single easiest to instrument and the most actionable. If you are running GitHub or GitLab, you can compute this with a weekend's work against your existing API access. OpenThunder surfaces this automatically as part of its change verification layer, which is worth knowing if you'd rather not build the instrumentation yourself.
None of these metrics replace code review. They inform where to focus review attention. At scale, you cannot give every AI-assisted PR the same depth of scrutiny. These metrics let you triage.
One thing I would push back on hard: the instinct to solve AI code risk with more AI. Using a second model to review the first model's output is useful at the diff level and nearly useless at the architecture level. The model reviewing the PR doesn't know that your UserContext object is supposed to be immutable after initialization, or that your team made a deliberate decision to never do direct DB calls outside the repository layer. That knowledge lives in your team, your ADRs, and your architecture docs, not in any general-purpose model.
The human review layer stays. What changes is what humans are reviewing for. In 2026, your senior engineers should be reviewing AI-assisted code for intent, architecture fit, and security context, not syntax. If they are still reviewing for syntax, your static analysis pipeline is not doing its job.
OpenThunder's approach here is to verify AI-assisted changes against your specific architectural constraints before they reach human review, which moves the syntax and known-pattern work out of the human loop entirely. That is the right division of labor.
See what OpenThunder verifies
OpenThunder independently verifies AI-assisted changes against your architecture, security, and intent before they ship, so your senior engineers spend review time on the decisions that actually require judgment. Try it here.
The CTO who treats AI code risk as a single metric will learn the hard way that it is at least four problems, and they tend to surface at the worst possible times.