Born-Measured Attention provides exact word evidence in transformers
Original title: Reading Why: Born-Measured Attention with Exact Word Evidence on Light and Qubits
Born measurements enable exact integrated-gradient evidence for word contributions with 76.98% e-SNLI accuracy.
Every attention step uses a Born measurement of |<q|k>|^4 normalized over rows. Feed-forward nonlinearity is replaced by a measured product. Class probabilities are polynomials in word amplitudes enabling quadrature-based evidence with no error. The model reaches 76.98% accuracy on e-SNLI matching softmax transformers of similar size. The architecture compiles exactly to optoelectronic optics and 23-qubit circuits.
- Exact evidence computation via finite quadrature to 10^-11
- 76.98% test accuracy on e-SNLI
- Model compiles to 852 CNOTs on 23 qubits
- Hardware reproduces answers under deletion on ibm_kingston
- Attention ranks words more like annotator highlights
From the original article
1 min read at arxiv.orgWhen a language model answers, we can usually only estimate why. We build a transformer in which every step is a Born measurement - attention $|\langle q|k\rangle|^4$ normalised over each row, a measured product in place of the feed-forward nonlinearity, and a Born readout.
By Teerawat Tongloy · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.