Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models
HyperQ trains quantum residual branches in a frozen 1.1B diffusion model, beating the backbone by 4.71 points.
HyperQ adds token-conditioned quantum residual branches to a frozen 1.1-billion-parameter masked-diffusion language model and trains only those branches. A circuit hypernetwork emits rotation angles, coupling strengths, and measurement axes; expectation values have an exact classical form whose cost grows linearly with qubit count, covering 16 to 64 qubits. Wider circuits raise the average downstream score from 47.65 to 54.30. At 64 qubits, HyperQ exceeds the backbone by 4.71 points and a low-rank-adapted counterpart by 3.67, after fine-tuning on 20,000 pairs versus 200,000 for the classical baselines.
- Frozen 1.1B diffusion backbone; only quantum residual branches are trained.
- Exact classical expectations scale linearly, enabling 16-to-64-qubit circuits.
- Average score rises from 47.65 to 54.30 as circuit width grows.
- At 64 qubits, gains are 4.71 over the backbone and 3.67 over LoRA.
- Fine-tuned on 20,000 pairs versus 200,000 for classical baselines.
From the original article
1 min read at huggingface.co
Language models can be adapted by changing the computations applied to individual tokens. Quantum circuits offer one such approach, but evaluating wider circuits inside a large model can be computationally demanding. Here we introduce HyperQ, which adds token-conditioned quantum residual branches to a frozen masked-diffusion language model.
By Xiaoqiang Wang, Mengyang Xiong, Jun Dai · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.