LLM discovery capacity linked to monofact rates
Original title: Tracing the Scientific Discovery Capacity of LLMs to Their Training Data
Higher monofact rates in training data increase scientific discoveries in LLMs.
LLM scientific discovery capacity relates to missing probability mass in training data. Monofacts set discovery capacity. In materials science and regulatory genomics, increasing monofact rates yields more discoveries across models and strategies. Relationship holds beyond training-set diversity and transfers across domains with in-domain data.
- More monofacts consistently yield more discoveries
- Works across two model families
- Cannot be explained by diversity alone
- Transfers across scientific domains
From the original article
1 min read at arxiv.orgLarge language models (LLMs) show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? We study this question as one of missing probability mass and develop a theoretical account in which monofacts-the facts observed once during training-set the discovery capacity.
By Arda Uzunoglu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.