Skip to content
NewA calmer, faster ZeroHour

279M Urdu UI ranker beats 2B and 7B VLMs on referential grounding

Original title: Rank, Don't Regress: A 279M Text Ranker Beats 2B and 7B VLMs on Referential Urdu GUI Grounding

arXiv cs.AI / cs.LG / cs.CLby Hamza Raziq Khan ()1 min read1 · 1 read
AI summary · x-ai/grok-4.7Summaries by Vani AI

On 304 held-out instructions from six Urdu news sites, the DOM ranker scores 73.4% point-in-box accuracy versus 29.6% and 23.7% zero-shot.

Instead of regressing click coordinates from screenshots, the authors rank candidate DOM elements with a 279M-parameter mDeBERTa-v3-base cross-encoder and click the centre of the chosen box. A frozen set from six Urdu news sites keeps test domains out of training; 149 of 304 instructions are referential. The ranker leads both zero-shot VLMs and a string-matching baseline, with errors concentrated on identical text and ordinal instructions.

Key facts
  • On the referential slice the ranker scores 65.8%, against 30.2% for untrained string matching.
  • A similar-size coordinate regressor trained on the same pages scores 1.6%.
  • Adding element geometry as input tokens lifts the best variant to 75.7%.
  • Zero-shot comparators are UI-TARS-2B at 29.6% and Qwen2.5-VL-7B at 23.7%.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

UI grounding asks a model to find the screen element that a user instruction refers to. Most current systems use a vision-language model (VLM) that reads a screenshot and outputs pixel coordinates. We study this task on Urdu mobile web pages, a setting with no published UI grounding results.

arXiv cs.AI / cs.LG / cs.CL

By Hamza Raziq Khan · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.