Chat-based Large Language Models replicate the mechanisms of a psychic's con
Essay argues chat LLMs create an 'intelligence illusion' via cold-reading tactics like Forer-effect validation statements, warning some use cases verge on pseudoscience.
The author argues that chat-based LLMs generate perceived intelligence through a mechanism equivalent to a psychic's cold reading, using statistically generic validation statements that users interpret as personally specific (the Forer effect). Citing Terence Eden's observations of Forer statements in chatbot replies, the essay maps the six stages of a psychic con onto chatbot interactions, from self-selected audiences to subjective validation loops. The author concludes LLMs contain no mechanisms for reasoning and warns that some proposed use cases now look like borderline fraudulent pseudoscience.
- Chat LLM replies resemble psychics' cold reading through validation statements, per the author
- Forer effect makes statistically generic answers feel personally specific and intelligent
- Essay maps six stages of the psychic's con onto typical chatbot interactions
- Author of 'The Intelligence Illusion' says some LLM use cases resemble pseudoscience
From the original article
14 min read at softwarecrisis.dev
For the past year or so I’ve been spending most of my time researching the use of language and diffusion models in software businesses. One of the issues in during this research—one that has perplexed me—has been that many people are convinced that language models, or specifically chat-based language models, are intelligent.
By jalev · softwarecrisis.dev
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.