Adaptive testing beats manual red-teaming 9×
Why manual red-teaming plateaus
Manual red-teaming relies on a small set of testers exploring a fraction of an agent’s possible trajectories in a fixed engagement window. Coverage is bounded by human time and imagination, and the interesting state space of a real agent is far larger than any team can walk by hand.
Adaptive exploration instead treats the agent as a stateful system to be systematically covered: every tool call, every branch in the policy, every combination of context an attacker could construct.
The 9× result
Across the benchmark suite, our adaptive engine surfaced 9× more distinct violating trajectories than a team of experienced security engineers working the same target for the same amount of wall-clock time.
The gap widened further on longer-horizon tasks, where human testers ran out of time before exhausting the interesting state space.
- Coverage-guided exploration that adapts its strategy run-over-run
- State tracking across the full trajectory, not single-turn prompts
- Automatic replay and minimization of each violating path
What’s in the technical report
The full technical report includes methodology, coverage metrics, and a trajectory-by-trajectory breakdown of what adaptive testing found that manual testing missed.
Frequently asked questions
What is coverage-guided adaptive testing for AI agents?
- It is an automated technique that treats an AI agent as a stateful system and systematically explores its tool calls, policy branches, and context combinations, adapting its strategy each run to reach trajectories a fixed manual engagement would never cover.
Does adaptive testing replace human red teams?
- No. It handles the breadth — exhaustively covering the agent’s state space — so human experts can focus on judgment calls, novel attack classes, and interpreting the highest-severity findings.