Frozen Models, Evolving Expertise: Model-Agnostic Learning from Deployment Experience for Multimodal Medical AI
A model-agnostic memory framework lets frozen medical LLMs and VLMs improve online by up to 34.2%.
The paper proposes external expertise so frozen LLMs and VLMs can learn after deployment: a Skill for reasoning and tool use, a Knowledge Memory of trusted facts, and a Multimodal Knowledge Base of visual examples. An update is retained only when it helps on new cases without degrading earlier performance. Across six benchmarks and four open-weight and closed-source models, online gains reach 34.2% over the base model on medical tasks, generalize to unseen cases, transfer across models, and also work outside medicine.
- Frozen models gain Skill, Knowledge Memory, and a visual case base.
- An update is kept only if new cases improve without regressions.
- Medical-task gains reach 34.2% over the base model.
- Stored expertise transfers to other models without further optimization.
From the original article
1 min read at huggingface.co
Large language models (LLMs) and vision-language models (VLMs) are usually frozen after deployment, so they do not learn from the cases they solve. This is especially concerning in medicine, where new clinical evidence, updated guidelines, and new therapies can change established practice. Fine-tuning can update the model, but it requires access to model weights and additional training.
By Yexiao He, Yucheng Tang, Pengfei Guo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.