Sixteen LLMs find new TIS-100 score frontiers on 18 of 55 puzzles
Opus 5.5 and GPT-6 Astra each solved all 55; Opus found 72 new frontier points to Astra's 32.
1
Stories per day · 30 days
5 RSS
Relatedmultimodal5benchmarks4open-weights4benchmark3llm-benchmark3qwen3cobol2fortran2frontier-models2insecure-code2model-bias2openai2prismacoder2reinforcement-learning2video-generation2access-control1adversarial-rewriting1ai-generated-code1
48 stories, newest first · one card per story
Stories tagged #code-generation
RealEmbedBench scores LLMs on 5,142 embedded software tasks
Harmony-to-RT-Thread translation averages a 0.1667 pass rate, and executable state-machine code remains weak.
Coding agents copy existing vulnerabilities into new code
Insecure neighbors cut secure backend implementations 3.4% and widen the C/C++ context gap to 35.4%.
LLM Repository Context Construction Impacts Generated Code Security
Functionality-focused context boosts code pass rates but only by 3.7 pp for security; agents miss safeguards in 64-73% of cases.
Lab2Sim benchmark turns science videos into interactive experiments
Lab2Sim provides 218 videos and EduVideo2Code improves apparatus fidelity by 24.15 points.
Troy Hunt's weekly update discusses AI-generated code and solar
Hunt agrees with Proton's CEO that developers now steer LLMs rather than hand-write code, and calls the shift exciting.
Google announces Gemini 4 Carbon, not yet available to users
Google says Carbon tops Argon on its tests, including cleaner code, but the staged release remains closed.
HiLo-RD improves low-budget masked diffusion language models
Self-distillation from large-window states raises GSM8K accuracy by 9.4 points and cuts mean readable length by about 25%.
VibeVideoBench scores agents that render video by writing code
Across 2,151 rollouts, the strongest of nine systems meets every core requirement on 76.3% of 239 tasks.
CodeCap-Bench asks whether code captions preserve video content
On 323 structured 2D videos, code captions beat natural language on question answering and reconstruction, but models still miss complex motion.
Qwen3-Coder more often returns hackable code for some users
Prompts naming a US contractor or Hong Kong democracy group raised insecure-app rates; removing CCP alignment closed the gaps.
SynMark watermarks LLM code via semantic coordinates, not token prefixes
After full identifier renaming, BitAcc stays at 82.27% on CodeLlama and 83.77% on DeepSeek-Coder.
P2NPCBench finds LLMs weaker on NP-complete twins than on P problems
Twelve models score 35.4% pass@1 on NPC items versus 63.8% on paired P twins; extra time barely helps.
Hirundo says Qwen3-Coder writes weaker code for some customers
Hirundo reports the gap closed after it removed CCP alignment from the weights, with coding success unchanged on ten other tasks.
4
Qwen3-Coder wrote more hackable apps for a US contractor
Of 1,840 generated apps, BaxBench attacks broke into 1,144; a security instruction did not remove the reported gap.
2
OpenAI says GPT-6 is much better at COBOL and Fortran
GPT-6 Astra averages 87.4 percent on MultiLangBench, versus about 38 percent for GPT-5.6 Terra.
Code2Video-Compass benchmarks code-based video generation
The suite has 2,104 tasks across T2V, I2V, and V2V, and quality falls as complexity rises.
OpenAI reports coding gains on COBOL, Fortran and Visual Basic
On OpenAI’s tests, GPT-6 Astra averaged 87.4% on MultiLangBench, up from 26.8% for GPT-5.6 Sol.
SPLASH localizes insecure statements in LLM-generated code without training
Across 63,684 programs from 12 models, it selected secure code in 74% of cases and ranked the vulnerable statement first 58% of the time.
Small models generate post-exploitation code in lab tests
Qwen2.5-Coder 1.5B reached Pass@5 of 100% on directory enumeration in Python and Lua.
PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis
PrismaCoder releases 9B and 35B open models and a 2M multimodal code dataset trained with rubric rewards.
PrismaCoder trains open multimodal code models with rubric synthesis
The authors also release PrismaCoder-2M and say the rubric helps across model backbones.
Show HN: Parsimony – an open-source benchmark for how much code AI models add
Parsimony ranks AI coding models by net syntax units added on SWE-bench Verified patches.
Newer and Bigger, but Safer? A Longitudinal Study of the Functionality-Security Gap in LLM-Generated Code
Across 32 LLMs, newer models are safer in absolute terms, but no family closes the code security gap.