When Do Biological Reasoning Models Use Their Biological Inputs?
BioReason models mostly follow text rather than Evo2 and ESM3 biological inputs.
Researchers tested six biological reasoning models by perturbing DNA, protein, and single-cell inputs, creating evidence conflicts, training linear probes, and inspecting reasoning traces. Evo2 and ESM3 contributed little to BioReason and BioReason-Pro: shuffling DNA barely changed BioReason disease prediction, and the models followed conflicting text in 97.9% and 99.7% of cases. Foundation-model inputs contributed more to ChatNT, Prot2Text-V2, and CellWhisperer, while differentially expressed genes mattered for Cell2Sentence-Scale. Across supervised and reinforcement-learning checkpoints, higher accuracy did not mean greater reliance on biological inputs.
- Six biological reasoning models were tested on DNA, protein, and cell tasks.
- BioReason followed conflicting text in 97.9% of cases; BioReason-Pro in 99.7%.
- Shuffling DNA barely changed BioReason disease-prediction accuracy.
- ChatNT, Prot2Text-V2, and CellWhisperer relied more on foundation-model inputs.
- Accuracy gains did not increase use of biological representations.
From the original article
1 min read at huggingface.co
Biological reasoning models use post-training to connect LLMs to biological foundation model representations and biological text. Their benchmark accuracy is taken as evidence that LLMs reason over these inputs. We test this assumption in six biological reasoning models across DNA, protein, and single-cell tasks.
By Ada Fang, Nikitha Thoduguli, Lukas Fesser · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.