SurgKnowBench and SurgAgentGym test generalist models on surgery scenes
Original title: How to "train" a surgeon? Benchmarking generalist agents in surgical scene understanding
Agentic tool use mainly helps perception-heavy tasks; localization and expert-style assessment stay weak.
Researchers present a two-stage evaluation of multimodal large language models for surgical scene understanding. SurgKnowBench harmonizes public surgical datasets and finds partial visual priors but weak relational, procedural, and workflow understanding. SurgAgentGym lets generalist agents use bounded search over SurgWiki and visual grounding without extra surgical training. Agentic inference narrows the gap mainly on perception-heavy tasks.
- SurgKnowBench maps pretrained competence across procedures and tasks.
- Models remain weak on relational understanding and workflow assessment.
- SurgAgentGym adds SurgWiki search and general-purpose visual grounding at inference time.
- Fine-grained localization and expert-style assessment remain persistent bottlenecks.
From the original article
1 min read at arxiv.orgEvaluating general-purpose multimodal large language models (MLLMs) for surgical scene understanding remains limited due to scarce surgical data and fragmented evaluation protocols. We address these challenges through a two-stage evaluation framework for surgical scene understanding.
By Gengyuan Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.