Persona directions rank LLM lies better than probes on three models
Original title: The Geometry of Lies: Knowledge and Persona Directions for Detecting Deception in Large Language Models
On knowledge lies the directions mostly track wording, and their sum beats a same-depth probe on all four models.
The paper tests label-free persona directions for detecting lies in large language models, including at 70B scale and on lies about knowledge. A widened persona direction ranks lies on Liars' Bench better than single-dataset probes on three of four models and fails on Qwen. On knowledge lies, persona directions largely follow wording. A knowledge direction separates identical honest denials on two models, and the sum of both directions ranks lies better than a same-depth supervised probe on all four models.
Unconfirmed: Several reported gains sit inside or at the edge of a random-direction band.
- The Llama-3.3-70B gain over the published direction stays within random widenings.
- Shared wording keeps three contrasts inside the random band on Llama and Qwen.
- The knowledge direction fails on the fourth model and partly depends on wording.
- More varied role-play questions do not help the persona direction on Liars' Bench.
From the original article
1 min read at arxiv.orgLinear probes are the cheapest LLM lie detectors but often fail to transfer across kinds of lie. Label-free directions from honest and deceptive role-play are untested at 70B scale and on lies about knowledge, where a lie and an honest denial can be identical. We widen a published persona bank with roles that deny knowledge they have, plus honest counterparts.
By Md. Abdullah Al Zami · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.