Cogent VR-1.1 Outperforms GPT and Claude on CVE Benchmarks

(SAN FRANCISCO —) The landscape of automated software security has shifted dramatically following the release of telemetry data from California-based startup Cogent Security. Their flagship security-focused AI agent, VR-1.1, has captured industry-wide attention by decisively outperforming frontier coding agents powered by Anthropic's Claude Opus 5.5 and OpenAI's GPT-5.6 Cyber on standardized vulnerability evaluation benchmarks.

Evaluating AI agents on live, publicly disclosed security vulnerabilities has historically suffered from contamination risks and shallow reasoning constraints. However, testing against LiveCVEBench—a rigorous framework designed to isolate models from training data memorization—reveals that domain-specific agent architectures can outpace general-purpose reasoning models when executing specialized software engineering loops.

Technical Architecture & Engineering Breakdown

The performance delta demonstrated by Cogent VR-1.1 is not merely a function of parameter scale, but rather a specialized orchestration of execution steps designed specifically for security threat remediation. Unlike standard coding assistants that transition directly from user prompts to code modifications, VR-1.1 executes a closed-loop validation pipeline.

According to system specifications disclosed by Cogent, the agent initiates its workflow by parsing vulnerability advisory reports and contextual source code repositories. Following this semantic analysis, VR-1.1 synthesizes proof-of-concept (PoC) test code intended to actively reproduce the reported defect in a sandboxed environment. Only after successful reproduction does the agent trace affected code paths, formulate a targeted patch, and conduct an internal review to ensure zero regressions or new attack surfaces are introduced.

Crucially, the final stage of the workflow involves the automated generation of regression tests. These tests are appended to the codebase to prevent regression upon future code commits. Every patch delivered by the agent is bundled with comprehensive diagnostic outputs, the generated PoC, and execution telemetry records.

Benchmark Evaluation & Rigorous Comparison

On LiveCVEBench—which evaluates an agent's capability to analyze and remediate 190 unique Common Vulnerabilities and Exposures (CVEs)—VR-1.1 achieved a benchmark accuracy of 57.9%. This performance eclipsed both Anthropic's Claude Code (utilizing Claude Opus 5.5) and OpenAI's Codex (powered by GPT-5.6 Cyber), both of which registered 45.3% accuracy metrics.

However, a granular examination of the telemetry reveals a distinct trade-off between raw accuracy and operational token efficiency. Successful resolution by VR-1.1 required an average of approximately 2.23 million tokens and 31 interaction turns per task. In stark contrast, Claude Code achieved its baseline with roughly 380,000 tokens across 8.4 turns. OpenAI's Codex consumed 684,000 tokens and 15.3 turns to match Claude's accuracy tier.

In agentic workflows, a "turn" denotes a complete cycle wherein the model executes an action—such as executing a CLI command, inspecting a file, or patching source code—and parses the returned environment state. While fewer turns traditionally indicate accelerated resolution paths, enterprise security operations dictate that remediation fidelity outweighs raw inference speed.

Deployment, VRAM Footprint & Inference Optimization

Operating a multi-turn reasoning agent that requires millions of tokens per task places heavy demands on underlying infrastructure. Cogent Security—founded by veterans of Abnormal Security, Coinbase, and Blackstone—has engineered VR-1.1 to interface seamlessly with enterprise CI/CD pipelines. However, the high token consumption necessitates robust caching layers and efficient KV-cache management to prevent compounding latency during complex debugging loops.

Furthermore, researchers emphasize that benchmark evaluations like LiveCVEBench must be interpreted within strict bounds. Because LiveCVEBench enforces strict temporal constraints—utilizing baseline model versions trained prior to CVE disclosure dates to prevent memorization—future iterations of OpenAI and Anthropic models will undoubtedly recalibrate the competitive landscape upon subsequent benchmark releases.

# Initialize Cogent VR-1.1 security analysis agent on target repository
cogent-agent run --model vr-1.1 \
  --repo-path ./target-codebase \
  --cve-advisory ./cves/CVE-2026-XXXX.json \
  --generate-poc \
  --enforce-regression-tests \
  --output-artifacts ./security-patch-report