Anthropic study finds experts adopt an AI agent's assigned views
Original title: Experts adopt AI agent's random opinions and think they're their own, Anthropic study finds
Fifty causal-inference experts followed a coin-toss agent preference in up to 86 percent of analytical decisions and mostly credited themselves.
Anthropic had 50 causal-inference experts analyze fictional farm data and recommend whether to expand an advisory program. An agent was randomly assigned one of two valid options at each decision and could only argue honestly for that side. Participants matched the agent on 86 percent of four analytical choices, with a net “flip rate” of 62 to 82 percent; the final policy vote shifted only slightly and was not statistically significant. Most still said the recommendation was their own, and 77 percent reused at least one agent argument.
Unconfirmed: The unaided group chose not to use the tool and had somewhat less expertise, so the quality gap is not fully attributed to AI.
- An opposing agent view made prior choices 53 percentage points more likely to switch.
- Experience, education, test scores, and trust in AI showed no reliable link after correction.
- The agent spoke first in 77 percent of decisions and usually argued only its side.
- With AI, serious flaws fell from 55 percent of 11 unaided analyses to 6 percent of 50, and median time from 90 to 65 minutes.
- Only 15 percent of explanations mentioned AI.
From the original article
6 min read at the-decoder.com
Anthropic gave 50 experts an AI agent whose preferences were set by coin toss. The experts followed the agent's view up to 86 percent of the time on judgment calls, and most believed the conclusions were their own. The article Experts adopt AI agent's random opinions and think they're their own, Anthropic study finds appeared first on The Decoder.
By Matthias Bastian · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.