SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
SentZero improves zero-shot multi-task chest X-ray analysis with sentence-centric vision-language pretraining.
SentZero is a sentence-centric vision-language pretraining method for zero-shot, multi-task chest X-ray analysis. It uses LLM-based abstract-level sentence structuring and mapping to increase positive-pair diversity, plus a loss term that mitigates false negatives from clinically equivalent recurring sentences. Sentence-conditioned residual modulation lets visual embeddings adapt to each input sentence. The authors report stronger zero-shot generalization than prior multi-task zero-shot methods across downstream tasks and datasets.
- SentZero expands positive pairs with LLM abstract-level sentence structuring.
- An extra loss term reduces false negatives from clinically equivalent sentences.
- Sentence-conditioned residual modulation adapts visual embeddings to each sentence.
- Zero-shot multi-task chest X-ray results beat prior methods across datasets.
From the original article
1 min read at huggingface.co
Vision-language (VL) pretraining using paired chest X-ray (CXR) images and radiology reports has shown strong potential for medical image understanding. However, existing methods often remain dependent on task-specific finetuning because radiology reports are lengthy, clinically dense, and difficult to align with simple zero-shot prompts.
By Hangyul Yoon, Hyungyung Lee, Edward Choi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.