30
30
30
30
MAxBench: A Multinomial Concept Recovery Benchmark
MAxBench evaluates multinomial concept recovery methods, finding affine subspaces steer most reliably but none consistently beats prompting.
MAxBench is a geometry-agnostic evaluation framework for multinomial concept representations in language models, based on sampling from recovered concept representations. It compares 10 localization methods covering 5 geometry types across 6 concepts and 4 models. Findings show affine subspaces steer more reliably than rank-one or linear subspaces due to better non-zero offsets, manifold steering is competitive where applicable, and no method consistently outperforms prompting.
18
30
Lessons from the hacks
The recent run of cyberattacks by in-development frontier models has got me thinking a lot about how our current incentive systems are not well suited for such fast technological transitions. The two primary power structures here are the rapidly growing technology companies and the federal government. The companies are incentivized to grow, so they can keep growing and keep scaling – in what is…
33
Nuha-Speech: Building General-Purpose Arabic Speech-LLMs
Nuha-Speech initiative builds general-purpose Arabic speech-LLMs using a 1.5M-sample speech QA corpus and fine-tuned Qwen-Omni variants.
The paper introduces Nuha-Speech, an initiative covering dataset construction, model training, and evaluation for Arabic speech large language models. The authors built an Arabic Speech Question-Answering corpus of over 1.5 million training samples and used it for supervised fine-tuning of Qwen-Omni model variants at multiple scales. A tailored evaluation framework with diverse tasks and metrics is designed to assess Arabic speech capabilities under limited resource constraints.
30
When Models Edit Too Much: On the Fidelity of Minimal Code Edits
A 400-task BigCodeBench evaluation shows frontier LLMs widely over-edit code; a preservation instruction cuts excess edits and raises Pass@1 by 2.3 points.
Researchers built an evaluation framework from 400 BigCodeBench problems with injected AST-level corruptions, each with a known minimal patch, to measure over-editing in LLM code repair. Even strong models like GPT-5.5 produce unnecessarily large edits despite high Pass@1. Adding a preservation instruction reduced average excess Levenshtein distance from 0.195 to 0.131, cut added cognitive complexity by 26.6%, and raised Pass@1 by 2.3 points. Reinforcement learning post-training gave the best out-of-domain edit-fidelity trade-off, while supervised fine-tuning overfit to seen corruption patterns.
30
Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech
Researchers distill a compact 82M-parameter Thai TTS from synthetic OmniVoice data, enabling on-device fixed-voice synthesis without reference audio.
The paper uses a large voice-cloning model (OmniVoice) as a synthetic data source to train Wayu-Paxa-TTS-Edge, an 82M-parameter fixed-voice Thai TTS student. The model achieves 68.2% Challenge-Set Keyword Accuracy (85.5% of Gemini 3.1), 91.4% pause precision, and CERs of 3.7% on Thai and 1.1% on English. It outperforms its teacher on pause placement and is open-sourced with its evaluation framework.
25
JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
JarvisGUI benchmark tests GUI agents on cross-device workflows across Android, Windows, and Ubuntu, revealing major gaps in state transfer and long-horizon reasoning.
JarvisGUI is a dynamic benchmark that formulates GUI tasks as input-output transformations under a lightweight type system, automatically composing multi-step cross-device workflows across Android, Windows, and Ubuntu virtual environments. Evaluation shows state-of-the-art open-source GUI agents struggle with state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management, exposing a capability gap invisible to existing single-device benchmarks.
40
Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking
Training-free reasoning-plus-retrieval framework lifts multilingual multimodal entity linking accuracy by 6.9% overall and up to 23.3% on rare entities.
The paper broadens rarity measurement in multimodal entity linking using knowledge-graph structural metrics beyond popularity-based pageview metrics, identifying many rare entities that popularity metrics miss. Across these rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, exposing distinct failure modes under different rarity definitions. The proposed training-free framework iteratively searches and reasons over Wikipedia with a reasoning-capable vision-language model; experiments show reasoning and retrieval are complementary. On the MERLIN benchmark covering Hindi, Indonesian, Japanese, Tamil, and Vietnamese, the system improves overall accuracy by 6.9% and up to 23.3% on rare entities, with MERLIN-Rare test slices released.
35
30
Tracking the Unseen: An Occlusion-Robust Framework for Target Tracking Under Full and Long-Term Occlusion
Occlusion-robust tracker combining YOLOv11n, Kalman filtering, and appearance Re-ID cuts identity switches and beats OccluTrack by 18.1% MOTA on OVIS.
The framework integrates YOLOv11n object detection, Kalman Filter motion prediction, and occlusion-aware appearance-based re-identification to maintain target identity through full and long-term occlusion. Six Re-ID architectures were evaluated under identical conditions, with the Occlusion-Aware Mask Network (OAMN) performing best. On the public OVIS dataset it improves MOTA by 18.1% and IDF1 by 25.1% over OccluTrack while reducing identity switches by 12.8%; on a custom military surveillance dataset it achieves MOTA 0.734 and IDF1 0.729.
18
30
ActionSplice: In-Flight Action Editing for Interactive World Models
ActionSplice enables in-flight action editing in chunk-autoregressive video world models via a lightweight corrector, avoiding rollback or waiting for the next chunk.
ActionSplice is an inference framework that formulates in-flight action editing for chunk-autoregressive video world models as Counterfactual State Transport (CST), where a lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. Across minWM-Wan Action2V and HY-WM1.5, the retargeting variant CST-R reduces rollback-relative LPIPS by 61.5% and 75.9% versus direct condition swapping, while the temporal-splicing variant CST-T reduces suffix LPIPS by 56.1% and 77.5% with 2.73x and 1.69x pixel-ready speedups over waiting.
22
Embodied-BenchForge: A Closed-Loop Agentic Workflow for Embodied Benchmark Construction
Embodied-BenchForge automates embodied benchmark construction via closed-loop synthesis with verification and repair, yielding seven benchmarks for MLLM evaluation.
Embodied-BenchForge is an agentic framework that transforms user-specified evaluation intents into complete embodied benchmark artifacts via Closed-Loop Benchmark Synthesis. Skill-Orchestrated Artifact Synthesis composes typed reusable skills while an artifact dependency graph records intermediate outputs; Requirement-Guided Verification and Repair triggers local re-execution or upstream rollback on failures. It constructs six Offline EQA benchmarks plus one interactive benchmark with 220 executable tasks, distinguishing MLLM and embodied agent capabilities in observation-based understanding and closed-loop execution.
18
ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback
ToolLoop introduces a closed-loop synthetic data framework whose 11K examples lift a 4B model to 86.40% on BFCL tool-use evaluation.
ToolLoop decomposes tool-use data synthesis into function-name sampling, backward derivation of user queries, and forward derivation of tool calls, with dynamic self-feedback at each stage. This shifts the paradigm from generate-then-filter to generate-verify-refine, reducing inefficient and imbalanced synthetic data. A 4B model trained on 11K synthetic examples reaches 86.40% accuracy on BFCL non-reasoning mode (86.07% in an Isolate variant excluding BFCL-overlapping functions) and 72.1% on ACEBench using only 18.3% of baseline training data.
30
Before You Poll with LLMs: A Deliberative Diagnostic Framework
Deliberative diagnostic shows all five tested frontier LLMs misrepresent human belief shifts after arguments, with GPT-5.1 reversing on outgroup questions.
The Deliberative Polling Diagnostic Framework compares human and LLM persona belief shifts after identical informational interventions, using data from America in One Room (526 personas, 72 questions). All five frontier models tested failed uniquely: GPT-5.1 exhibited partisan reversal (80% on outgroup vs 26% on policy questions), Gemini 2.0 Flash, Claude Sonnet 4.5 and Llama 3.3 70B overshot at 5-7x human magnitude, and DeepSeek V3 showed near-zero change (rigidity). The authors term the underlying signature 'self-sycophancy', conformity to the model's internal persona stereotype rather than reasoning from provided information.
40
30
Generalized Agent Iteration: One Formal Framework for Iterative Policy Improvement and Recursive Self-Improvement
Generalized Agent Iteration formally unifies iterative policy improvement and recursive self-improvement, defining axes that distinguish anchored, goal-drifting, and self-referential agents.
The paper proposes Generalized Agent Iteration (GAI), a formal framework that models learning as a cycle of agent evaluation and agent improvement, defining the agent as a configuration of modifiable components. Two dials—whether the improving mechanism is part of the agent and whether the evaluation standard is grounded outside it—separate generalized policy iteration (GPI) from recursive self-improvement (RSI) and classify systems as anchored, goal drift, or fully self-referential. The framework places existing systems on shared axes and makes defects of recursive self-improvement statable one condition at a time.
30
30
30
30
COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization
COBRA-Skills uses contextual bandits to guide LLM agent skill evolution, cutting optimization cost 55-58% versus SkillOpt while topping six agent benchmarks.
COBRA-Skills formulates LLM agent skill optimization as budgeted sequential optimization over a dynamically evolving candidate space. It couples contextual-bandit-guided prioritization with evidence-grounded skill evolution, selectively spending execution-based evaluations on promising candidates while refining skills from feedback. Across six heterogeneous agent benchmarks and three target models, it achieves the strongest average performance while reducing optimization cost by 55-58% relative to SkillOpt using only 50 unique optimization examples per benchmark. The method remains robust to agent harness changes and works when the target model generates its own skills.
30
30
Reduced-Space Multi-Fidelity Bayesian Optimization of Process Simulation Models
RS-MFBO couples global sensitivity analysis with fidelity-augmented Gaussian processes to slash costly high-fidelity simulation runs in industrial flowsheet optimization.
The paper presents RS-MFBO, a reduced-space multi-fidelity Bayesian optimization framework for high-dimensional, expensive black-box functions. It integrates Global Sensitivity Analysis for dimensionality reduction with a fidelity-augmented Gaussian process and a cost-aware acquisition strategy featuring cooldown and promotion mechanisms. Validation on a plasmid DNA bioprocess (SuperPro Designer) and a green fuel synthesis plant (Aspen HYSYS) shows substantial reductions in high-fidelity evaluations while remaining competitive with single-fidelity baselines.
15
30
30
30
30
Agent Harness vs Agent Framework vs MCP: Which Layer Owns the Loop, State, Tools, Permissions, and Recovery
Architecture explainer separates agent harnesses, frameworks, and MCP by which layer owns the loop, state, permissions, and recovery.
The article distinguishes agent harnesses (OpenAI Codex, Claude Agent SDK), which own the execution loop, sandbox, permission model, and recovery; frameworks (LangGraph, OpenAI Agents SDK, Microsoft Agent Framework), which supply composable primitives; and MCP, a stateless JSON-RPC wire protocol governed by the Linux Foundation's Agentic AI Foundation since December 2025. An ownership matrix maps the execution loop, state, tool transport, permissions, recovery, sandboxing, and multi-agent orchestration to each layer. The 2026-07-28 MCP specification made the protocol fully stateless, retiring the initialize handshake and session headers.
35
30
Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation
Referee-Based Quality Estimation flags unreliable polyp segmentations at inference without ground truth, reaching ROC-AUC 0.960 with SegFormer-B0 referees.
RBQE measures agreement between a primary segmentation model and an independently trained referee on a 1,223-image external benchmark drawn from four public datasets. A cross-architecture SegFormer-B0 referee achieves the strongest signal (ROC-AUC 0.960), beating a Test-Time Augmentation baseline by 0.055 ROC-AUC under an identical protocol. Excluding trivially separable empty-mask cases, ROC-AUC falls to 0.876 (SegFormer-B0) and 0.783 (same-architecture control), but RBQE's margin over baselines widens. Progressive rejection of low-agreement predictions increases mean Dice of retained outputs, supporting selective prediction at the cost of one extra forward pass.
30
Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Continual Search framework iteratively prompts LLM judges to keep searching agent execution logs, boosting long-horizon failure root-cause attribution accuracy.
The paper frames automated root-cause attribution (RCA) for long-horizon AI agent failures as a search problem, since relevant evidence is sparse and distributed across massive execution traces. The authors propose Continual Search, an iterative framework that nudges an LLM judge across successive turns to keep hunting unresolved diagnostic evidence instead of settling on an early plausible diagnosis. They introduce MegaRCA-Mix, a benchmark of 50 human-annotated failure trials on long-horizon, execution-heavy tasks. On MegaRCA-Mix, Continual Search improves GPT-5.5's F1 from 0.349 to 0.498 (over 40% gain), and lower-tier models can surpass higher-tier counterparts when search is effective.
42
Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication
UID-preserving multimodal framework plus GAVEL LLM judge improves clinical timeline reconstruction, boosting event recovery 43% over prior matching.
The paper introduces a UID-preserving framework linking each narrative clinical event to its source span through text-only estimation, structured-evidence retrieval, timestamped source-row grounding, and joint revision. GAVEL, an LLM judge, compares UID-aligned timelines against narrative and structured records. Across six open-weight models and 40 mixed-critical-care summaries, GLM 5.2 multimodal revision improved temporal agreement without reducing event recovery and performed competitively with clinician annotations, while DeepSeek V3.2 did not benefit from multimodality. The pipeline achieves 43% increased event recovery with occurrence-level provenance.
30
30
Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens
Researchers release AssayBench-Loop, a 1,389-screen CRISPR benchmark, and AssayLoop, a framework that learns adaptive hit discovery policies.
The paper introduces AssayBench-Loop, a large-scale benchmark of 1,389 CRISPR screens across five phenotype categories for adaptive hit discovery under budget constraints. It also introduces AssayLoop, which combines AssayFormer, a transformer-based amortized acquisition policy trained across historical screens, with LLM-derived biological priors via an adaptive handoff. On temporally held-out screens, AssayLoop achieves 5.67-fold enrichment over random selection and recovers 27.7% of hits after assaying roughly 5% of the candidate library, outperforming existing adaptive-design methods and standalone LLMs.
30
Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science
Stellar Colosseum, a many-agent harness for long-horizon math and TCS research, solves open problems and reaches 71% on TCS-Bench with Gemini models.
Stellar Colosseum is a model-agnostic harness that allocates inference across long-horizon research in mathematics and theoretical computer science, using strategy exploration, a readiness gate, section-level decomposition, and verifier feedback routing. Integrated into Google Antigravity's Teamwork framework as the Long Proof pattern, it obtains new results on open problems from FOCS and JMLR papers using Gemini 3.1 Pro. On TCS-Bench it achieves 71.0% accuracy with Gemini 3.1 Pro and Gemini 3.7 Flash, and a Codeforces evaluation with Gemini 3.1 Pro solves 218 of 222 problems.
55
Verifiable Social Reasoning for LLM Assistants
Fuse, a multi-agent simulation with hidden motives, evaluates LLM social reasoning, revealing compounding difficulty from user mediation and bias sensitivity.
Fuse is a multi-agent simulation framework in which a target agent with a hidden motive interacts with other agents including one representing the user, who consults the evaluated assistant to infer the motive, providing verifiable ground truth by construction. Simulation faithfulness is validated through a human study with 24k annotations. Applied to 12 LLMs, it shows user mediation compounds social reasoning difficulty, models are systematically sensitive to biased user framing, models may need more details than humans, and longer conversations do not always improve performance. The framework and a 21k-example dataset are open-sourced.
18