LongChang tests whether LLMs discover altered physical laws unaided
Original title: Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Of 12 models, only GPT-6-Astra found a law, at a 16.7% discovery rate; the other 11 scored 0%.
LongChang builds simulated worlds where one or more physical laws differ from ordinary physics and lets models explore without being told what to investigate. Separate agents generate scenes, review tools, and score reports using the altered laws and exploration records. The paper says records show uninvestigated anomalies and only partial identification of the changed laws.
- Explorers manipulate objects and build tools without assigned research tasks or hints.
- Scene generation, tool review, and report scoring run without human intervention.
- GPT-6-Astra’s reported discovery rate is 16.7% among 12 LLMs.
- The other 11 evaluated models scored a 0% discovery rate.
From the original article
1 min read at arxiv.orgScientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate?
By Zhen Yang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.