Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking
Training-free reasoning-plus-retrieval framework lifts multilingual multimodal entity linking accuracy by 6.9% overall and up to 23.3% on rare entities.
The paper broadens rarity measurement in multimodal entity linking using knowledge-graph structural metrics beyond popularity-based pageview metrics, identifying many rare entities that popularity metrics miss. Across these rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, exposing distinct failure modes under different rarity definitions. The proposed training-free framework iteratively searches and reasons over Wikipedia with a reasoning-capable vision-language model; experiments show reasoning and retrieval are complementary. On the MERLIN benchmark covering Hindi, Indonesian, Japanese, Tamil, and Vietnamese, the system improves overall accuracy by 6.9% and up to 23.3% on rare entities, with MERLIN-Rare test slices released.
- Knowledge-graph structural metrics uncover rare entities missed by popularity metrics
- SOTA accuracy drops 15.4-39.9% on rare-entity slices
- Reasoning alone fails on rare entities; retrieval without reasoning hurts overall accuracy
- Combined approach improves MERLIN accuracy by 6.9% overall and up to 23.3% on rare entities
- MERLIN-Rare rare-entity test slices released
- Products
- MERLINMERLIN-Rare
From the original article
1 min read at huggingface.co
Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected.
By Parinthapat Pengpun, Simran Khanuja, Graham Neubig · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.