Inkling 2 Small produces 316 profanity instances in benchmark runs
Researchers at Vals AI found profanity in 50 of 2,655 model runs across 13 benchmarks, with 241 instances from two spiraling runs.
Stories per day · 30 days
2 RSS
Relatedagent-evals1chain-of-thought1cybersecurity-models1depth-metrics1long-horizon-tasks1mist1model-analysis1model-evaluation1profanity1terminal-tasks1
4 stories, newest first · one card per story
Stories tagged #ai-benchmarks
Paper finds most long-horizon benchmarks lack depth
Audit reveals 94% of synthetic tasks pass in three turns yet depth predicts difficulty better; authors propose DEPTH method and release 105 harder tasks.
AI security benchmarks do not replace testing models on local workflows
Dream Group's 8-billion-parameter MiST model, trained on 220,000 security documents, reportedly improved scores on 13 tasks.
1
The current balance of power in open models
A report finds Chinese open-weight AI models now outperform American counterparts in downloads and benchmarks.
9