AutoScaffold builds training-time hints for language-model reinforcement learning
Scaffold-free students beat RL baselines on four tasks, including +7.2 ALFWorld success and +4.0 QA exact match.
Stories per day · 30 days
11 RSS
Relatedbenchmarks11reinforcement-learning6evaluation5pretraining5attention4distillation4rlvr4self-distillation4alignment3diffusion-models3embeddings3inference3mixture-of-experts3olmo3post-training3privacy3reasoning3retrieval3
161 stories, newest first · one card per story
Stories tagged #language-models
Text encoding changes byte-level language model accuracy by language
Aether uses script-specific encodings with UTF-8 fallback; Chinese and Korean gain, while Georgian and Tamil do not.
Second-order screen predicts quantization mismatch on five language models
On unseen Mistral-7B-v0.3, all six predictions fell in band and vLLM throughput was predicted within 0.4%.
Tree-Ising repairs Qwen reasoning loops with sparse BF16 edits
On two layers it scores 54.6% on MATH-500 versus 51.7% for a GPT-6 Astra edit and 7.9% for the base model.
Study rechecks where Platonic representation alignment appears in newer models
The capability-alignment trend holds when captions are clean, but same-modality alignment still far exceeds cross-modal scores.
Audit finds 41 models change mental health advice by who asks
Across 19 vignettes and 35,055 responses, models often gave clinicians consultation-style answers and lay users simpler, hedged advice.
Spectral unlearning applies matrix-structured updates to transformers
Applied atop existing objectives, it narrows most of the gap to a retain-trained reference on memorization, privacy, and utility.
Paper measures how much mismatch on-policy distillation tolerates
Authors say model-specific FP8 recipes keep quality while holding at least 1.93 times more KV-cache tokens than BF16.
onóna uses twin tied embeddings to cut validation loss
At matched token budgets it lowered loss in 11 of 12 runs and raised average accuracy in 10 of 12.
Information Games model analyzes firm repositioning in language-model space
Model examines competitive capacity adjustments using news articles, analyst reports, and multi-agent simulation for Nasdaq companies 2018-2022.
PLatour Enables Prefill-to-Autoregressive Cross-Model Transfer
PLatour framework shows promise in balancing capabilities and costs but gains may not require true sender-receiver communication.
PEARL durably suppresses misaligned behaviors in language models
PEARL augments GRPO rollouts with intermediate state continuations for durable suppression while preserving accuracy.
Born-Measured Attention provides exact word evidence in transformers
Born measurements enable exact integrated-gradient evidence for word contributions with 76.98% e-SNLI accuracy.
StateBench tests language models on state changes in long documents
Benchmark evaluates how language models track state changes over extended documents.
Kierston simulation for crime investigation
KierstonBench benchmark tests language models on 125,000-resident Allentown simulation for record linkage and investigation tasks.
Research Detects Prompt-Injection Hijacks via Model Probabilities
Monitor using output probabilities detects some hijacks on AG News classifier but misses most when attackers add certainty claims.
RangeSWA provides training-free length extrapolation for diffusion language models
RangeSWA reduces perplexity and repetition in long-text generation using sliding-window attention and positional remapping.
OLMo 2 membership inference study finds limited success
Controlled OLMo 2 1B training with matched document pairs shows reference-ratio AUC of 0.767 but detects only 3.8% members at 1% FPR.
Paper says localized gradients improve language-model data attribution
On the synthetic CATT benchmark, top-token negative log-odds scoring lifts ground-truth recall by 35 to 72 points.
Neural Thickets gains traced to output diversity rather than weight experts
Perturb-Select-Vote on frozen models matches or beats full-weight perturbations on most reasoning tasks and helps vision under domain shift.
STRATA scores each step of temporal clinical hypothesis tests
The benchmark has 8,735 vaccine-report hypotheses; ART narrows a record-selection gap and brings a 31B model near frontier scores.
Multi-View MoE reuses multiple routers to shape expert computation
On a 24-layer 10B-A1B model trained on 708B tokens, nine-task mean rose 1.18 points.
Czech president awards Medal of Merit to Word2Vec co-creator Mikolov
Brno University of Technology graduate Filip Maleňák was also honored for the Záchranka emergency app at the October 28 ceremony.
Paper catalogues 13 patterns for coupling language models with rules
It adds five invariants, five failure patterns, and a decision guide based on the authors’ measurements.