WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data
WearableQA benchmark introduces 4,084 questions over real longitudinal wearable data, showing 14 LLMs score 19.6-72.9% on health reasoning, far from solved.
WearableQA is a benchmark of 4,084 10-option multiple-choice questions built from wearable time series, blood biomarkers, and demographics of 200 real users with up to 500 days of daily measurements. It defines 16 question types along two axes: data versus health reasoning, and single- versus cross-signal reasoning, using a dual-grounding framework combining literature and population-validated patterns. Evaluations of 14 proprietary and open-source LLMs show accuracy ranging from 19.6% to 72.9% against a 10% chance baseline, with most models below 60%.
- 4,084 questions built from 200 real users with up to 500 days of wearable data.
- 16 question types split data versus health and single versus cross-signal reasoning.
- 14 LLMs tested; scores range 19.6%-72.9% versus a 10% chance baseline.
- Products
- WearableQA
From the original article
1 min read at huggingface.co
Recent advances in wearable sensing enable continuous monitoring of physiological and behavioral signals, yet existing benchmarks rarely evaluate whether AI systems can reason over a real user's longitudinal wearable record.
By Ji Soo Lee, Xilun Chen, Pierce Chuang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.