30
30
30
30
Large Language Models Develop Belief State Geometry In-Context
Probing six open-source LLMs on HMM-generated data shows belief states are linearly decodable (R² 0.83–0.99), suggesting in-context learning approximates Bayesian prediction.
Researchers prompted six open-source LLMs with data from 40 hidden Markov models selected for non-trivial belief structure and probed residual-stream activations for belief states (posteriors over hidden states). Belief states were linearly decodable with peak R² values of 0.83–0.99 across HMM/LLM combinations, spanning early to late layers. Patching and steering the probe-identified subspace preserved downstream prediction quality while control interventions degraded performance substantially, establishing functional relevance. The results provide representation-level evidence that in-context learning approximates optimal Bayesian prediction over a context-inferred generative model.
45
PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
PhysBrain 1.5, an 8B physical foundation model, sets open-source state of the art across 28 embodied understanding benchmarks.
The paper presents PhysBrain 1.5, a unified 8B model for understanding physical environments, generating actions, and predicting future states, built from a vision-language model with joint autoregressive next-token prediction over language, end-effector motion, and dense visual targets. Pre-training uses embodied supervision from human interaction videos, followed by supervised fine-tuning on human demonstrations, robot trajectories, and simulated experience. The model averages 72.5 across 28 embodied benchmarks, setting a new open-source state of the art and performing on par with proprietary GPT-6-Astra and Gemini 3.6 Flash, with best open-source results on 14 benchmarks.
45
Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models
Researchers release Explainability Assistant, an open-source conversational XAI tool using LLM function calling, lifting intent-parsing accuracy from 76.8% to 94%.
The paper introduces the Explainability Assistant, an open-source conversational XAI system for interpreting energy consumption forecasting models such as genetic-programming symbolic regressors. It uses LLM function calling instead of rigid custom grammars, achieving 94% intent-parsing accuracy versus 76.8% for prior work TalkToModel, and adapts to different ML problem types without task-specific fine-tuning. Comparative evaluation with energy domain specialists against a traditional XAI dashboard showed improved usability, with all experts preferring the conversational interface.
25
SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs
SAFIRE, an 83K-image fire and smoke benchmark, shows open-source multimodal LLMs average only 61.9% accuracy on safety-critical fire reasoning.
SAFIRE is a large-scale benchmark for fire-smoke understanding in multimodal LLMs with 83K captioned images across 20 scenarios and 193K multiple-choice VQA questions spanning 10 evaluation dimensions from perception to higher-order reasoning. Annotations were built via a GPT-5.4-assisted multi-stage pipeline with MLLM majority voting. Ten open-source MLLMs (8B-38B) average 61.9% accuracy, exposing major gaps in safety-critical reasoning. Adapting vision encoders on 7% of the domain data raises fire-scene classification from 20.1% to 64.5%.
33
LLM Agents as Computational Typologists
AUTOTYPOLOGIST is an LLM agent that performs evidence-grounded linguistic typology analysis over 25 open-source reference grammars.
The agent retrieves relevant grammar sections, analyzes interlinear glossed text (IGT), and iteratively reasons over typological hypotheses in a ReAct-style workflow. It was evaluated on typological feature coding against expert annotations and hypothesis testing against universals using 25 open-source reference grammars. Results suggest LLM agents can support scalable, inspectable crosslinguistic analysis but still require expert validation.
16
Dr. Claw: An AI Scientist Workspace for Vibe Research
Researchers release Dr. Claw, an open-source auditable workspace that wraps coding agents like Claude Code for end-to-end AI-assisted research workflows.
Paper 2609.00365 presents Dr. Claw, an open-source workspace that wraps existing coding-agent executors such as Claude Code and Gemini CLI in a controllable, human-in-the-loop research workflow. It uses persistent state objects, a reusable skill library, and multi-executor coordination to make research decisions auditable and recoverable, rather than adding another autonomous agent. Holding the executor fixed, Dr. Claw scores higher on research completeness than a bare command-line agent while preserving an auditable process trail. The code is released under AGPL-3.0 on GitHub (OpenLAIR/dr-claw).
45
PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
PlannerForge unifies scenario-based testing of autonomous driving motion planners in one LLM-agent framework, outperforming prior baselines.
PlannerForge is an LLM-agent framework that covers the full scenario-based testing pipeline for autonomous driving systems, spanning scenario generation, selection, modification, routing, planner testing, plus new enhancement and benchmarking stages. In evaluations with 10 off-the-shelf LLMs, best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends such as Qwen3.6:35B match commercial APIs on most tasks. End-to-end chaining retains 83% (commercial) and 78% (open) of seed queries, beats Scenario Factory 2.0 on executable generation, and cost-tuning lifts planner success from 50.4% to 70.2% while cutting collisions from 19.0% to 8.4%.
28
VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification
VDiff-Bench, a 1,756-question benchmark, shows multimodal LLMs struggle with fine-grained image-difference identification, scoring as low as 8.7% on low-level changes.
VDiff-Bench is a multiple-choice benchmark of 1,756 four-way questions over image pairs covering 10 change categories including position, motion, color, texture, OCR/text and illumination, with curated hard negatives. Evaluation of 11 state-of-the-art open- and closed-source MLLMs shows fine-grained visual comparison remains brittle: 7-8B-scale open-source models score 52.5-70.6% on semantic changes but only 8.7-33.3% on low-level changes like noise and texture. Notably, Grok 4.3 shows a sharp performance drop on noise and texture differences, falling behind large open-source models like Kimi K2.5 and K3.
32
Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe
Researchers introduce KOPA-Bench, a 145-task Korean public API tool-calling benchmark, and EDGE, an execution-grounded data synthesis method.
An arXiv paper presents KOPA-Bench, a benchmark of 145 real-world tasks chaining multiple tool-calls across live Korean government APIs, motivated by data-sovereignty requirements for on-premise open-source LLM agents. It also introduces EDGE, an execution-grounded dynamic graph that keeps only tool-output-to-input links verified by live API calls before synthesizing executable multi-step trajectories. A 9B model fine-tuned with GRPO on the resulting dataset nearly matches its untuned 27B family sibling on KOPA-Bench and improves on the BFCL benchmark.
32
30
Verifiable Social Reasoning for LLM Assistants
Fuse, a multi-agent simulation with hidden motives, evaluates LLM social reasoning, revealing compounding difficulty from user mediation and bias sensitivity.
Fuse is a multi-agent simulation framework in which a target agent with a hidden motive interacts with other agents including one representing the user, who consults the evaluated assistant to infer the motive, providing verifiable ground truth by construction. Simulation faithfulness is validated through a human study with 24k annotations. Applied to 12 LLMs, it shows user mediation compounds social reasoning difficulty, models are systematically sensitive to biased user framing, models may need more details than humans, and longer conversations do not always improve performance. The framework and a 21k-example dataset are open-sourced.
18
OPEN-1B: A Fully Auditable Training Run
Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.
The paper introduces a 'fully auditable' tier of model transparency: every training operation is reproducible with bitwise certainty on heterogeneous commodity hardware by imposing definite ordering on GPU kernel reductions, data batch ordering, and collective communication. Because replaying a full run on one machine is infeasible, a collective verification scheme lets many independent auditors certify individual steps covering the whole run. The authors release Open-1B with its full pretraining dataset, every intermediate checkpoint, the training codebase, and an audit harness. This rules out undisclosed data, injected biases, or backdoors that proof-of-learning or proof-of-training-data techniques cannot exclude.
58
How much of F-Droid is LLM generated?
A FOSS maintainer manually graded 102 F-Droid apps from the September 12, 2026 update batch, finding many show signs of LLM-generated code.
A student and FOSS app maintainer reviewed 102 apps pushed to F-Droid on September 12, 2026, assigning each a three-tier rating for likelihood of LLM-authored code (mostly AI >50%, hard to say/mostly human, no signs of AI). The heuristic relies on commit aesthetics, README and branding style, and the presence of agentic infrastructure like Claude Code or Codex, which automatically places an app in the 'mostly AI' tier. Example ratings include Amber (Nostr event signer) as mostly AI, and Aria for Misskey as showing no AI signs. The author stresses reliable detection of LLM-generated code from text alone is impossible, so ratings are approximate.
15
Agent Harness vs Agent Framework vs MCP: Which Layer Owns the Loop, State, Tools, Permissions, and Recovery
Architecture explainer separates agent harnesses, frameworks, and MCP by which layer owns the loop, state, permissions, and recovery.
The article distinguishes agent harnesses (OpenAI Codex, Claude Agent SDK), which own the execution loop, sandbox, permission model, and recovery; frameworks (LangGraph, OpenAI Agents SDK, Microsoft Agent Framework), which supply composable primitives; and MCP, a stateless JSON-RPC wire protocol governed by the Linux Foundation's Agentic AI Foundation since December 2025. An ownership matrix maps the execution loop, state, tool transport, permissions, recovery, sandboxing, and multi-agent orchestration to each layer. The 2026-07-28 MCP specification made the protocol fully stateless, retiring the initialize handshake and session headers.
35
RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments
RSIAgent, a training-free multi-agent framework, builds reusable environment memory enabling Kimi-K3 and GLM-5.3 to beat GPT-6.
RSIAgent is a training-free framework for recursive self-improvement through autonomous memory construction, coordinating curriculum, actor, and verifier agents. It uses broad-then-deep exploration to capture environment structures, hidden constraints, and causal dependencies, and freezes the resulting memory for direct reuse without parameter updates. On OSWorld-v2 and Agent's Last Exam it substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.
52
Autonomous Research for Open-Ended Problems: A Case Study on Telecom Ticket Retrieval
Case study shows autonomous LLM research reaches 90% of SOTA on telecom ticket retrieval in 10 weeks versus 10 months human work.
The paper explores adapting autonomous research to open-ended, industry-grade ML problems through a telecom ticket retrieval case study with commercial and open-source agents. Autonomous research reached 90% of state-of-the-art performance (0.34 vs. 0.38 Recall@1) in 10 weeks versus 10 months of human work, at up to $200 per Cursor campaign. The authors find agents excel at narrow hyperparameter optimization but lack human-like intuition, recommending human-agent collaboration.
18
GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay
GPU-CFR compiles counterfactual regret minimization into static dataflow with CUDA Graph Replay, achieving 29.8-80.4x speedups over prior GPU solvers.
The paper presents a compiler and runtime that turns any fixed game's CFR iteration into a static dataflow graph of flat arrays and precomputed indices, cutting framework operations by up to 18.1x. Because shapes and buffer addresses never change, CUDA Graph Replay records the iteration once and replays it with a single launch. On one A100 across an eight-game suite, GPU-CFR runs 29.8-80.4x faster than the fastest prior GPU CFR and 14-258x faster than the CPU implementation LiteEFG on the four largest games, while reproducing reference iterates bitwise on CPU.
38
Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness lets VLM agents control robots via discrete semantic action units, outperforming VLA baselines zero-shot and after light fine-tuning.
Show-Harness is an embodied agent harness that exposes discrete semantic action units a VLM reasons over, with embodiment-specific interpreters grounding them into local robot actions. It enables zero-shot robot control with closed-source frontier VLMs and low-cost adaptation of small open-source VLMs using only a few GPU-hours of fine-tuning. The companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection without specialized teleoperation hardware. Experiments show robust generalization across tasks, embodiments, and environments, beating representative agentic and VLA paradigms.
32
JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
JarvisGUI benchmark tests GUI agents on cross-device workflows across Android, Windows, and Ubuntu, revealing major gaps in state transfer and long-horizon reasoning.
JarvisGUI is a dynamic benchmark that formulates GUI tasks as input-output transformations under a lightweight type system, automatically composing multi-step cross-device workflows across Android, Windows, and Ubuntu virtual environments. Evaluation shows state-of-the-art open-source GUI agents struggle with state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management, exposing a capability gap invisible to existing single-device benchmarks.
40
Show-Harness: Just a VLM Agent Can Play Robots
Show-Harness enables VLM agents to control robots via a semantic action interface, achieving zero-shot frontier control and few-GPU-hour adaptation of small VLMs.
Show-Harness exposes discrete semantic action units that VLMs reason over, with embodiment-specific interpreters deterministically grounding them into local robot actions. It enables zero-shot closed-source frontier VLM control and adapts small open-source VLMs for low-cost deployment with a few GPU-hours of fine-tuning. The companion GUMI interface extends the same semantic action space to GUI-based demonstration collection without teleoperation hardware, and Show-Harness-equipped agents outperform representative agentic and VLA paradigms.
28
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Grouped Value Attention stores grouped values and reconstructs content keys via a learned linear map, cutting KV-cache size about 45-47% versus GQA.
GVA stores only grouped values and reconstructs content keys with a learned linear map absorbed into the query at decode time, while a small shared decoupled RoPE channel preserves positional information via a separately cached positional key. At 350M parameters trained on 30B FineWeb-Edu tokens, the 16-dimensional positional variant scores 44.18 average accuracy across five tasks versus 44.36 for GQA and 43.88 for MLA. Custom decoding kernels are in development with an open-source release planned.
42
WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data
WearableQA benchmark tests LLM health reasoning over longitudinal wearable data; the best of 14 evaluated LLMs reaches 72.9% accuracy.
WearableQA comprises 4,084 ten-option multiple-choice questions built from wearable time series, blood biomarkers, and demographics of 200 real users with up to 500 days of daily measurements. It defines 16 question types along two axes: data versus health reasoning, and single- versus cross-signal reasoning. Evaluation of 14 proprietary and open-source LLMs shows performance from 19.6% to 72.9% against a 10% chance baseline, with most models below 60%.
14
Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models
New ROBORMBENCH benchmark shows vision-language reward models can flip robot success/failure judgments when goal instructions are paraphrased.
The authors show that paraphrasing the instruction alone can substantially change progress scores from VLM reward models, even flipping identical robot trajectories between failure and success. ROBORMBENCH comprises 2,390 real-robot trajectories with ground-truth progress labels and 21,673 verified paraphrases covering lexical, syntactic, and action-goal rewrites. Instability is widespread across proprietary and open-source VLMs, grows with more divergent rewrites, and is not reliably reduced by scale or explicit reasoning, while trajectory-grounded dedicated reward models are markedly more stable.
35
Most of the bugs Claude Mythos found have never been checked by a human
Echo's analysis found only 1,900 of 23,019 Claude Mythos-found vulnerabilities were externally reviewed, 90.8% held up, but the model overstated most severities.
Echo analyzed results from Anthropic's Claude Mythos Preview vulnerability sweep across 281 open-source projects, which produced 23,019 candidate vulnerabilities, of which only 1,900 were externally reviewed. Of those, 90.8% held up as real, 1,451 of 1,596 maintainer reports were acknowledged, 97 fixes landed upstream, and 88 became advisories, but 14 of the 27 CVE-assigned severity ratings mismatched independent scoring, mostly overstated. On Anthropic's SpiderMonkey benchmark, Claude Mythos turned known crashes into working code execution exploits in 72.4% of 250 trials, versus below 1% for Claude Opus 4.6. Echo cautions the reviewed sample likely was not randomly drawn, so the accuracy figure may not generalize to the other 21,119 unreviewed candidates.
62
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
BeaconKV introduces training-free KV cache compression using beacon queries, cutting long-reasoning inference memory up to 5.8x while preserving accuracy.
The paper shows recency-based KV cache compression assumptions fail in long-horizon reasoning because Thought Revisiting Tokens (TRT) re-attend to distant context such as early task-solving plans. TRT queries cluster into a small number of similarity groups, which BeaconKV exploits by maintaining compact beacon query representatives to anticipate revisited KV pairs without storing full query history. The training-free method achieves up to 5.8x memory reduction and over 4.3x throughput improvement across four open-source large reasoning models while nearly preserving full cache accuracy.
33
WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data
WearableQA benchmark introduces 4,084 questions over real longitudinal wearable data, showing 14 LLMs score 19.6-72.9% on health reasoning, far from solved.
WearableQA is a benchmark of 4,084 10-option multiple-choice questions built from wearable time series, blood biomarkers, and demographics of 200 real users with up to 500 days of daily measurements. It defines 16 question types along two axes: data versus health reasoning, and single- versus cross-signal reasoning, using a dual-grounding framework combining literature and population-validated patterns. Evaluations of 14 proprietary and open-source LLMs show accuracy ranging from 19.6% to 72.9% against a 10% chance baseline, with most models below 60%.
35
Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech
Researchers distill a compact 82M-parameter Thai TTS from synthetic OmniVoice data, enabling on-device fixed-voice synthesis without reference audio.
The paper uses a large voice-cloning model (OmniVoice) as a synthetic data source to train Wayu-Paxa-TTS-Edge, an 82M-parameter fixed-voice Thai TTS student. The model achieves 68.2% Challenge-Set Keyword Accuracy (85.5% of Gemini 3.1), 91.4% pause precision, and CERs of 3.7% on Thai and 1.1% on English. It outperforms its teacher on pause placement and is open-sourced with its evaluation framework.
25
TempCloze: Can Video-LLMs Identify the Missing Middle?
TempCloze benchmark tests Video-LLMs' temporal reasoning with 1,521 videos, finding temporal alignment is the primary failure mode across 31 models.
TempCloze is a video cloze benchmark in which models must identify the true missing middle clip given the beginning and ending clips, using 1,521 carefully filtered videos from seven sources, mostly long-take and egocentric footage. Distractors are constructed along three dimensions: Semantic, Alignment and Progression, with shared scenes and objects to reduce appearance cues. Evaluation of 10 proprietary and 21 open-source Video-LLMs found Alignment is the primary bottleneck, with models often recognizing plausible semantics and local event progression but struggling with temporal alignment.
25
🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing
Caltech professor Anima Anandkumar discusses Neural Operators and FourCastNet for physics modeling, arguing inductive biases beat pure token scaling.
Anima Anandkumar, Bren Professor at Caltech and co-founder of Accelerated Understanding, describes Fourier Neural Operators that learn in frequency and spherical-harmonic domains to model weather, fusion, and fluid or heat flow. Her team built FourCastNet 3, a global weather model competitive with physics-based simulations that runs on consumer-grade GPUs. She also introduced TorchLean, a framework for writing PyTorch-style networks inside the Lean proof assistant for formal verification, and was appointed to the United Nations Scientific Advisory Board. She argues physical domains resist scaling due to tiny datasets and context lengths in the hundreds of billions, so progress comes from built-in structure and physical priors.
42
Import AI 470: No rights for machines; automating environment generation with SPADE; and building better GPU kernels with Hawkeye
METR analysis finds AI accelerating cyber vulnerability discovery, while SPADE self-play environment generation improves Qwen3 reasoning benchmark scores at 30B scale.
Import AI 470 discusses a METR research note reporting differential acceleration from AI: major acceleration in reported cyber vulnerabilities (cURL, OpenSSL, Firefox, Microsoft, NVD, OSV), minor acceleration in mathematics, and no measurable acceleration in AI-research optimization benchmarks. It also covers SPADE, a self-play framework from a multi-university team (University of Washington, Stanford, MIT, CMU, and others) that co-evolves executable training environments and agent capability using Environment Designer and Reasoning Agent roles with hint-based regret rewards. Trained on Qwen3-4B-Instruct-2507, Qwen3-8B, and Qwen3-30B-A3B-Instruct-2507 via GRPO (400 rollouts of 25 environments), SPADE lifted the 30B-A3B game-environment suite average to 58.3, +8.1 over base, and improved tool-use results across backbones. The issue also references Hawkeye for building better GPU kernels.
42
30
30
30
30
30
30