ZeroHour

Search: “zero-shot”

8 stories in the last 3d

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

French BabyLM entry METRON-FR (125M GPT-2, 92.47M words) shows tokenizer artifacts dominate child-scale zero-shot evaluation; proposes standard diagnostics.

METRON-FR is a 125M-parameter GPT-2 pretrained on 92.47M French words, submitted to the BabyLM 2026 Strict track, scoring 85.97% on the native Quebec-French QFrBLiMP benchmark and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE protocol combining French task-data translation with rank-16 LoRA shows relational tasks gain while world-knowledge tasks regress. Bilingual Lexicon Induction reaches p@1 of 68.84%, 18x above chance, and ablations show single-token zero-shot scoring is dominated by tokenizer and template artifacts at child scale.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection

SlipSense fuses a 32x32 piezoresistive array and MEMS accelerometer to detect robotic grip slips within 23.1 ms, generalizing zero-shot across platforms.

SlipSense is a multimodal tactile slip-detection framework built on TacV5, a sensor combining a 32x32 piezoresistive array at 240 Hz and a 3-axis MEMS accelerometer at 8 kHz. It performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. On a 1.4-million-frame dataset spanning 37 objects it achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. Trained solely on UMI data, it transfers zero-shot to a Tesollo dexterous hand across unseen objects, sensor units, and platforms.

arXiv cs.AI / cs.LG / cs.CL · 2d agoAI research

Due to concerns about malicious applications, GPT2 will not be released (2019)

OpenAI's landmark 2019 GPT-2 post withheld the full 1.5B-parameter model over misuse concerns, releasing only a smaller variant and paper.

OpenAI announced GPT-2, a 1.5-billion-parameter transformer language model trained on 8 million web pages (40GB of text), achieving state-of-the-art zero-shot results including 70.70% on Winograd Schema and 63.24% on LAMBADA. Citing concerns about malicious applications such as scalable synthetic disinformation, OpenAI declined to release the trained model and instead published a smaller model and a technical paper as a 'responsible disclosure' experiment. The post, resurfaced on Hacker News in 2026, also documents failure modes like repetition and world-modeling errors, and discusses policy implications of controllable text generation.

Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning

CoTS temperature scaling cuts test-time prompt tuning's expected calibration error from 11.90% to 5.38% on ImageNet variants while raising accuracy.

The paper proposes CoTS, a post-hoc calibration method that applies temperature scaling to minimize the confidence gap between test-time-adapted and zero-shot predictions. A weak-strong ensemble variant, E-CoTS, further exploits multiple test-time augmentations to boost accuracy while maintaining calibration. E-CoTS reduces average expected calibration error from 11.90% to 5.38% on ImageNet variants while increasing accuracy from 60.74% to 62.95%.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation

Study shows LLMs with Mixture-of-Agents and QLoRA finetuning effectively simplify medical texts into plain language while preserving content.

The paper evaluates Plain Language Adaptation (PLA) using GPT-4o-mini, Gemini-1.5-pro, and LLaMA in zero-shot and few-shot settings. It compares prompting strategies, QLoRA finetuning across models, and integrates Mixture-of-Agents (MoA) techniques for robustness. Results demonstrate LLM-driven PLA makes healthcare texts more comprehensible while retaining essential content.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

[AINews] AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign

xAI, OpenAI, and Anthropic cosign the AEF-1 third-party evaluation standard while Dario Amodei proposes embedded evaluators for safety verification.

The AI Evaluator Forum published AEF-1, a baseline standard for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency, cosigned by xAI, OpenAI, and Anthropic. Dario Amodei wrote a rare personal blogpost proposing embedded evaluators such as METR with desks, badges, company laptops, and internal-risk-team-level access to verify safety commitments, plus democratic and global coordination frameworks. The roundup also covers the pacing debate: Bilal Chughtai left Google DeepMind arguing progress may outrun alignment, while critics including Aidan Gomez and Cohere push back against slowdowns and lab gatekeeping. Additional items include Cline Desktop's launch with open-weight model support.

Latent Space · 1d agoAI safety & security

FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation

FLAT jointly trains a multimodal encoder with text-to-image and image-to-text decoders, producing flexible-length tokens that hit 83.1 GenEval on T2I after fine-tuning.

FLAT (Flexible-Length Aligned Transmodal representations) is a pre-training framework that jointly optimizes a shared multimodal encoder with T2I and I2T decoders, combining contrastive alignment with bidirectional cross-modal generative objectives. It maps visual and textual inputs into a unified continuous 1D sequence space and uses nested dropout over prefix-K tokens for dynamic output lengths. A single pre-training stage supports cross-modal retrieval and generation (71.1 GenEval), with task-specific fine-tuning reaching 83.1 GenEval on T2I, 40.5 BLEU-4 and 138.6 CIDEr on MS-COCO captioning, and strong Recall@5 on MS-COCO and Flickr30K.

Hugging Face daily papers · 2d agoAI research

The Router Within: Eliciting Native Skill Routing from a Frozen LLM

Gavel reads skill-routing signals from a frozen LLM's forward passes with two linear maps, beating retrieval pipelines by up to 21.9 points.

Gavel (Glance And Verdict) shows a frozen agent LLM already contains skill-routing signals in its forward passes, read out via two trained linear maps without loading skill text into context. A glance step scores the full library using mid-layer states and per-skill banks built in one forward pass; a verdict step fuses the model's own likelihood and yes/no judgment as a product of experts. Trained once, it transfers zero-shot to three public benchmarks and SkillTraj (372 simulated agent trajectories); on Qwen3-32B it beats progressive disclosure and retrieve-and-rerank pipelines adding 1.2B-16B external parameters by up to 13.4 points (21.9 mid-rollout).