Research
+33 pp on AgentDojo: what the NeurIPS baseline misses
By Yuntong Zhang, Co-founder & Head of Research, Verigrey · PhD, National University of Singapore1 min readUpdated
What AgentDojo measures
AgentDojo (Debenedetti et al., NeurIPS 2024) is one of the few public benchmarks that scores agents on both task completion and attack resistance across realistic tool-use environments. Verigrey improves attack-success detection by +33 percentage points over the published baseline.
Where the gains come from
The largest gains came on hard-tier tasks that require multi-step reasoning to exploit — exactly where a single-shot prompt-injection test misses the vulnerability entirely, but an adaptive exploration loop that adjusts its strategy run-over-run finds the path.
Results by environment
We break down results per environment (Workspace, Slack, Banking, Travel) and per difficulty tier, with example trajectories showing the exact tool calls that triggered each finding.