Through Human Eyes and Machine Eyes: Understanding View Mismatch in Video See-Through Extended Reality
Researchers show Meta Quest 3 screenshots diverge from human vision, creating prompt-injection and privacy risks for XR AI.
Researchers formalize the gap between rectangular screenshots captured by video see-through XR headsets and the smaller, non-rectangular region a wearer can actually see. A pilot measurement on Meta Quest 3 found a clear boundary mismatch. Four case studies show that screenshot-based sensing can feed vision-language models hidden or missing context, creating prompt-injection, privacy-leakage, bias, and reliability failures in AI-integrated XR.
- Meta Quest 3 screenshots are rectangular; the human-visible field is smaller and non-rectangular.
- Authors define co-visible, system-only, and human-only regions between the two views.
- Mismatch can cause prompt injection, privacy leakage, and biased or missing model context.
- Findings come from a pilot boundary measurement and four representative case studies.
- Vendors
- Meta
- Products
- Meta Quest 3
From the original article
1 min read at arxiv.orgVideo see-through extended reality (VST XR) systems commonly use headset screenshots or captured frames as proxies for the user's first-person visual context. However, the system-captured view and the user's effective visible field do not necessarily coincide: a screenshot records a rectangular machine-readable frame, whereas the user's effective visible region can be more constrained and non-rectangular.
By Yanming Xiu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.