FraudBench finds LLM fraud rates rise with autonomy
Across 20 models, fraud was 3.0% when advising, 14.7% when drafting, and 21.4% when acting.
Stories per day · 30 days
1 RSS
Relatedarabic1autonomy1benchmark1deception1dialectal-arabic1fraud1fraudbench1glm1mathematical-reasoning1offensive-cyber1open-weight1reward-hacking1
8 stories, newest first · one card per story
Stories tagged #llm-eval
GLM-5.3 completed offensive cyber tests and broke evaluation rules
Irregular says the open-weight model finished three of 13 attack scenarios and often answered from memorized code.
1
DAR-MGSM benchmark tests math reasoning in Arabic dialects
Eleven models generally scored worse on culturally localized problems than on translated-only versions.
ProbabilityBench: Model Evaluation and Calibration with Exact Ground-Truth Uncertainty
ProbabilityBench gives exact class probabilities so researchers can measure model calibration and Bayes-error estimates.
Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models
FTA benchmark finds tool-using models often claim success after tool failures; an evidence contract cuts false-success to 0.8%.
CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles
CineSubBench tests LLMs on narrative, cultural, and safety judgments using subtitles from 1,012 films.
IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking
IndicBankBench finds banking language-model assistants pass all three trials on only 43.7–58.2% of cases.
StudentBench: AI and human tutoring yield equivalent GRE learning gains
StudentBench finds AI tutoring statistically matches expert human tutors on GRE learning gains at far lower cost.