APort Vault: Benchmarking AI Agent Payment Authorization with the Open Agent Passport
APort Vault benchmark evaluates AI agent payment security across 14 models, showing Open Agent Passport blocks all unauthorized transfers.
APort Vault is a benchmark for evaluating payment authorization in tool-using AI agents, testing 4,371 human-written attacks across 14 models from 8 labs. The Open Agent Passport (OAP) specification successfully blocked all unauthorized transfers, reducing unauthorized payments from 140 to 0. The benchmark includes 225,964 evaluations with public release of data, scoring code, and analysis scripts.
55