PatchHolmes: Agentic Patch Retrieval via Listwise Selection
PatchHolmes uses listwise agent inspection to find vulnerability-fixing commits missing from CVE advisories.
PatchHolmes retrieves the commit that fixes a known vulnerability using a hybrid retriever and a listwise agent that inspects the top 100 candidates and reads 3 to 10 commits. The authors report that 60% to 63% of CVEs in major advisory databases lack a patch link. On GitHubAD it beat Favia by 25.34% Recall@1 and IRCoT by 31.40% with one agent conversation per CVE, and it runs locally on a frozen open-weight model without fine-tuning or external search APIs.
- Major advisory databases lack patch links for 60-63% of CVEs.
- On GitHubAD, Recall@1 beat Favia by 25.34% and IRCoT by 31.40%.
- Identical candidates: agent added 27.32% Recall@1 over retriever top-1.
- Runs on a frozen open-weight model without fine-tuning or search APIs.
Full article214 words · extracted from huggingface.co · click to collapse
Patch retrieval, the task of finding the commit that fixes a known vulnerability, is the foundation of vulnerability management workflows, yet 60% to 63% of CVEs in the major advisory databases lack a patch link. We present PatchHolmes, a two-phase patch retrieval system that pairs a hybrid first-stage retriever with an agentic second-stage inspection loop. Unlike pointwise prior work that scores each candidate independently, the Phase 2 agent reads the top-100 listwise: it sees the full candidate list at once and selectively reads 3 to 10 commits through four budgeted tools before submitting a single best commit. On GitHubAD, PatchHolmes beats the pointwise binary classifier Favia by 25.34% Recall@1 and the retrieve-and-CoT baseline IRCoT by 31.40%, at one agent conversation per CVE versus Favia's ten; with the candidate set held identical, the agent adds 27.32% Recall@1 over taking the retriever's top candidate, and the same agent, transferred unchanged to PatchFinder_top10, lifts Recall@1 from PatchFinder's own top-1 pick (24.28%) to 39.86%. Swapping the LLM backbone within the Qwen family changes Recall@1 by under 1%, and a second model family (gpt-oss) stays far above the no-agent floor, so the gain comes from the listwise agent loop; the entire system runs on a frozen open-weight model over a local Git repository, without fine-tuning or external search APIs.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.38807