LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators
LLM-to-Flow Distillation uses a frozen LLM and DPO to steer scientific generators toward language goals.
LLM-to-Flow Distillation aligns domain-specific flow and diffusion generators with high-level natural-language scientific goals without an explicit proxy reward. A frozen LLM builds a curriculum of intermediate sub-goals, judges sampled candidates, and those preferences are iteratively distilled into the generator with DPO. The method is intended to provide learning signal even when the generator initially assigns almost no probability to designs that meet the final goal. Experiments on therapeutic peptides and small-molecule structure design show the approach can steer generators toward out-of-distribution discovery objectives.
- A frozen LLM teaches a domain-specific flow or diffusion generator without a proxy reward.
- The teacher builds a curriculum and its judgments are distilled with DPO.
- Experiments cover therapeutic-peptide and small-molecule structure design tasks.
From the original article
1 min read at huggingface.co
Generative discovery in science requires coupling high-level scientific reasoning with the ability to generate samples respecting the structure of a design space. Yet these capabilities remain largely separated: foundation LLMs encode broad chemical and biological knowledge, while domain-specific flow- and diffusion-based generators capture the priors needed to produce valid sequences and geometries.
By Riccardo De Santi, Federico Di Gennaro, Angelo Gnazzo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.