Search: “corpus composition”

32 stories

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

SeverityAll critical 0 high 0 medium 0 low 0 info 3

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining

Climate-ModernBERT domain-adapted encoders reach 76.3 average F1 across nine climate benchmarks, 2.8 points above vanilla ModernBERT-Base.

The authors continue pretraining ModernBERT-Base on three climate corpora - academic text, climate-filtered web data, and synthetic documents - and compare joint mixtures against parameter-space merging of specialized checkpoints. The best model achieves 76.3 average F1 across nine climate NLP benchmarks, a 2.8-point improvement over the vanilla baseline. Academic climate corpora provide the strongest adaptation signal, and parameter-space merging outperforms joint multi-source training while preserving complementary corpus information; all variants are released.

arXiv cs.AI / cs.LG / cs.CLSep 7, 17:41 UTC · 9d agoAI research

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selectionnew

Researchers introduce PANORAMA, a vision-language model grounding caption phrases in pixel masks, alongside the PanoCaps benchmark and gPQ metric.

The paper studies panoptic grounded captioning, requiring VLMs to describe foreground and background regions while grounding each phrase with pixel-level masks. The authors release PanoCaps, a human-annotated benchmark built from panoptic segmentation datasets, plus a phrase-mask matching protocol and a generalized Panoptic Quality (gPQ) metric. PANORAMA formulates grounding as selection from phrase-conditioned mask proposals generated by a pretrained segmenter, achieving the best overall grounding on PanoCaps and matching or exceeding specialized models. Code, data, and models are publicly available.

Hugging Face daily papersupdated Sep 16, 17:59 UTC · 13h agofirst Sep 15, 20:00 UTC · 1d agoAI research 2 sources

GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

GoDeep achieves annotation-free open-vocabulary 3D segmentation by grounding structured image descriptions in language-only embeddings, outperforming CLIP-based lifting on out-of-vocabulary objects.

GoDeep uses a vision-language model purely as a translator, producing structured entity-level image descriptions that are grounded, projected, and aggregated in a general-purpose language-only embedding space, with no 3D training corpus or dedicated 3D encoder required. On ScanNet++ the pipeline is competitive with strong annotation-free baselines, and on a cultural-heritage benchmark a systematic vocabulary correction reverses initial CLIP-based rankings. Language-space embeddings separate genuinely out-of-vocabulary objects more sharply, localize them within scenes, and keep all predictions explainable as discrete text.

arXiv cs.AI / cs.LG / cs.CLSep 8, 17:31 UTC · 8d agoAI research

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CLSep 4, 16:38 UTC · 12d agoAI research

A Hybrid LSTM-XGBoost Framework for Multi-Horizon Stock Return Prediction Across Diversified Equity Portfolios

Hybrid LSTM-XGBoost model predicts multi-horizon returns for 14 US equities, cutting 30-day RMSE to about one-third of a standalone LSTM baseline.

The paper combines a two-layer LSTM (64 hidden units) processing 60-day windows of five market features with an XGBoost regressor over a 78-dimensional hybrid feature vector including 14 technical indicators. It is trained on pooled data for 14 US equities across six sectors using chronological splits and per-stock MinMaxScaling to prevent look-ahead bias, and evaluated at 30, 90, 252, and 365 trading-day horizons. The hybrid achieves test RMSE of 0.0949 at 30 days, roughly one-third of the standalone LSTM, while 97.6% directional accuracy at 365 days largely tracks the base rate of positive returns.

arXiv cs.AI / cs.LG / cs.CLSep 11, 17:51 UTC · 5d agoAI research

Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences

SCPTM graph-augmented variational topic model shows syntax aids topic diversity and descriptor quality but gains stem mainly from the variational encoder.

The Structural Contextual Probabilistic Topic Model represents corpora as heterogeneous document-word graphs with lexical and syntactic edges processed by a Graph Attention Network inside a VAE for mixed-membership topic distributions. Across four corpora, neural gains in document-topic alignment are attributable to the variational encoder rather than syntax, while graph-augmented variants improve topic diversity everywhere. Dependency paths add value on argumentative deliberative texts but are redundant in technical and institutional registers.

arXiv cs.AI / cs.LG / cs.CLSep 7, 17:39 UTC · 9d agoAI research1

Building a Linux GPU Driver for the M4 Mac Mini in One Month

Two developers built a fully OpenGL ES 3.0 compliant Linux GPU driver for the M4 Mac Mini in one month via clean-room reverse engineering.

Niklas and the author reverse engineered Apple's AGX GPU firmware ABI and user-space components in about a month, a process that normally takes years, producing an OpenGL ES 3.0 conformant driver fast enough to run Minecraft at 200fps on an M4 Mac Mini. The work was done transparently using hypervisor traces without examining Apple binaries, following clean-room practices, and included a custom shader compiler, command stream builder, and a full Linux kernel driver for the firmware ABI. The A18 Pro firmware ABI proved significantly more complex than the M1's, with 1.5x as many structs and twice as many pointers. All experiments and provenance evidence were published in public agx-re repositories.

Hacker News · AISep 15, 19:30 UTC · 1d agoResearch HN 40↑ · 5 comments

Unifying Conformal Language Tasks with In-Context Ensembles

Researchers propose Conformal Relevance, which builds conformal score functions via in-context example curation and ensembling to improve conciseness across seven NLP tasks.

The paper targets NLP tasks like summarization and extractive QA that reduce to retrieving content under coverage and conciseness constraints. Conformal Relevance replaces hand-engineered LLM scoring prompts with curated in-context examples and ensembles, maintaining coverage guarantees while improving conciseness with minimal manual input. The authors demonstrate the framework on seven NLP tasks and contribute theory, including a complementarity condition for when ensembling improves worst-case sentence scores and a saturation bound on ensemble gains.

Hugging Face daily papersSep 1, 20:00 UTC · 15d agoAI research1

Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answering

Evaluation of twelve LLMs on 222 clinical questions shows verbatim quotes rarely substantiate claims; claude-opus-5 fully substantiates only 37.1%.

The authors build a standardized harness over four clinical practice guidelines and evaluate twelve LLMs on 222 synthetic clinical questions, measuring citation attachment, verbatim quote production, and claim substantiation. Most models attach verbatim quotes to over 90% of claims from prompting alone, though lightweight models like claude-haiku-4.5 struggle. Quotes frequently fail to substantiate claims: claude-opus-5 quotes 98.0% of claims but fully substantiates only 37.1%, exposing a capability gap for verifiable clinical QA.

arXiv cs.AI / cs.LG / cs.CLSep 14, 17:53 UTC · 2d agoAI research

Type Diversity Enables Transformers to Generalise Compositionally

Researchers show lexical-versus-structural compositional generalization gaps in Transformers stem from type diversity imbalance in datasets, not architectural limits.

The paper argues that Transformers' difficulty with structural compositional generalization is an artifact of low structural type diversity in prior benchmark datasets rather than an architectural limitation. Using Grammatical Framework, the authors create linguistically diverse variants of COGS and SLOG. They find type diversity correlates with compositional generalization equally in lexical and structural test cases, contradicting previous claims that compound divergence explains task difficulty.

arXiv cs.AI / cs.LG / cs.CLSep 11, 17:59 UTC · 5d agoAI research

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face BlogAug 26, 00:00 UTC · 22d agoAI tools & infra1

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papersSep 7, 20:00 UTC · 9d agoAI research1

LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics

LexFlip releases 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving tokens, exposing weaknesses in embedding-based meaning preservation metrics.

LexFlip provides 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of tokens, creating dissociation items that break monotone token-overlap metric validation. The seven embedding and BERTScore metrics tested register only 0.022-0.039 of their identical-to-unrelated range on these edits, versus 0.670 for bidirectional NLI. Against FrJudge, with a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric tested.

arXiv cs.AI / cs.LG / cs.CLSep 4, 15:49 UTC · 12d agoAI research

I Am No One: Style-Aware Paraphrasing for Text Anonymization

Prompt-driven style-aware paraphrasing with LLMs cuts authorship attribution F1 by 60-70% while preserving content quality.

The paper proposes a style-aware, prompt-driven anonymization approach using pretrained LLMs to build compact stylistic profiles from minimal samples and rewrite text to suppress identifiable style markers while preserving meaning. It addresses stylometric re-identification risks in anonymized text, including ASR transcripts of meetings and call-center calls where leakage persists after acoustic anonymization. Across blog and review datasets, the approach reduces authorship attribution F1 by 60-70%, substantially outperforming both DP-based and non-DP baselines while maintaining readability.

arXiv cs.CRSep 11, 02:07 UTC · 6d agoResearch

Objective vs. Search: Decomposing What Makes a Good Tokeniser

New tokeniser study shows search procedure, not optimisation objective, drives bits-per-byte performance across model sizes, vocabulary sizes, and multilingual settings.

The paper disentangles BPE and UnigramLM along two axes: optimisation objective (compression vs log-likelihood) and search procedure (bottom-up merging vs top-down pruning). Two new algorithms, BottomUpLL and TopDownComp, complete the 2x2 design space, and trained language models are evaluated on bits-per-byte and BLiMP across model sizes, vocabulary sizes, and English-only vs multilingual domains. Bottom-up tokenisers consistently achieve lower bits-per-byte in most settings, while BLiMP shows no consistent relationship with design choice.

arXiv cs.AI / cs.LG / cs.CLSep 16, 17:59 UTC · 13h agoAI research

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

French BabyLM entry METRON-FR (125M GPT-2, 92.47M words) shows tokenizer artifacts dominate child-scale zero-shot evaluation; proposes standard diagnostics.

METRON-FR is a 125M-parameter GPT-2 pretrained on 92.47M French words, submitted to the BabyLM 2026 Strict track, scoring 85.97% on the native Quebec-French QFrBLiMP benchmark and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE protocol combining French task-data translation with rank-16 LoRA shows relational tasks gain while world-knowledge tasks regress. Bilingual Lexicon Induction reaches p@1 of 68.84%, 18x above chance, and ablations show single-token zero-shot scoring is dominated by tokenizer and template artifacts at child scale.

arXiv cs.AI / cs.LG / cs.CLSep 15, 16:43 UTC · 1d agoAI research

IndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing

Researchers release IndicTriMix benchmarks and fine-tuned MuRIL and XLM-RoBERTa models for token-level language identification in tri-language code-mixed text.

The paper formulates token-level language identification in code-mixed text as a sequence labeling task and fine-tunes MuRIL and XLM-RoBERTa transformer models for Indian languages. It evaluates on Hindi, Gujarati, and Bengali configurations with manually annotated test sets and proposes two code-mixed generation approaches using parallel trilingual sentences. A public benchmark, annotated test sets, and fine-tuned models are released for reproducibility.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:36 UTC · 6d agoAI research1

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled pure-autoregressive testbed shows task-specific validation losses rank image tokenizers differently, with I2T loss the most consistent signal.

Researchers built a controlled pure-autoregressive testbed and tracked task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They find losses should be analyzed per task because they exhibit distinct scaling behavior and rank tokenizers differently, and that the loss-performance relationship depends on the predicted token space. I2T loss, computed over a shared text vocabulary, correlates consistently with both generation and visual understanding performance after supervised finetuning. Case studies revisit the discriminator, semantic supervision, and vocabulary size as tokenizer design axes.

arXiv cs.AI / cs.LG / cs.CLSep 8, 17:57 UTC · 8d agoAI research1

Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers

Hugging Face details building and using multi-vector late-interaction embedding models with Sentence Transformers for retrieval workloads.

Hugging Face published a guide on multi-vector, late-interaction embedding models (ColBERT-style) supported through Sentence Transformers. The post covers how practitioners can build and use these models for retrieval and RAG pipelines. It is a developer tooling and technique write-up, not a security advisory.

Hugging Face BlogAug 18, 00:00 UTC · Aug 18, 2026AI tools & infra1

Molecular Déjà Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

Audit of 22 frontier models finds widespread verbatim retrieval of published molecular property values, with higher reasoning increasing recall of memorized numbers.

An arXiv audit tests 22 frontier LLMs across 12 molecular regression benchmarks for verbatim retrieval of published values. More than 50% of the LLMs show verbatim retrieval on five datasets, and identical experiments are flagged 89% more often at a high reasoning level than at the lowest one. Suppressing retrieval moves model prediction errors closer together in relative terms, suggesting predictive capability is not determined solely by memorized values.

arXiv cs.AI / cs.LG / cs.CLSep 4, 17:32 UTC · 12d agoAI research1

Studying Without a Syllabus: Task-Agnostic Environment Preprocessing

Paper formalizes task-agnostic environment preprocessing, where agents study unfamiliar environments under a budget to build reusable artifacts for a frozen solver.

The paper formalizes task-agnostic environment preprocessing, where a studying system explores an environment under a budget and produces artifacts like indices, scripts, or procedural guidance for a frozen solver, without task examples or evaluation feedback. The authors compare unaided and archive-equipped meta-agents against fixed synthetic-practice and corpus-processing methods across six heterogeneous benchmarks. A meta-agent variant achieves the highest Avg@3 reward on five benchmarks, while fixed corpus processing remains best on the largest corpus benchmark. Studied artifacts reduce the test-time sampling needed to reach a given score, shifting computation from repeated test-time attempts to a pre-task study phase.

Hugging Face daily papersSep 8, 20:00 UTC · 8d agoAI research

RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

RelateAnything is a 53M-parameter open-vocabulary relation prediction model running at 20 ms/frame, with 2.3-3.5x higher mean recall than comparable open-vocabulary methods.

RelateAnything predicts scored relations between image regions using any predicate vocabulary supplied at inference as text embeddings, with object labels never required as input, so region sources can change without retraining. Training covers 19,103 predicates using positive-unlabeled supervision; the authors release RA-4M (474k images, 4.3M geometrically verified relations over 10,102 free-text predicates) and the OV-SGG-Bench evaluation suite. The 53M-parameter model runs at 20 ms/frame and achieves 2.3-3.5x the mean recall of the strongest comparable open-vocabulary method across cross-dataset and zero-shot benchmarks. Model, corpus, and benchmark are public.

Hugging Face daily papersSep 10, 20:00 UTC · 6d agoAI research1

Robust Coverless Linguistic Steganography via Sentence Embedding Space with Global Resynchronization

Researchers propose a coverless steganographic framework encoding messages as hierarchical clustering paths in sentence embedding space with a Global Resynchronization Mechanism for robustness.

An arXiv paper proposes encoding secret messages as hierarchical clustering paths in the sentence embedding space rather than token space, improving decoding stability against word- and sentence-level textual perturbations. A Global Resynchronization Mechanism (GRM) reframes variable-length bitstreams as discrete symbols anchored to semantic subspaces to prevent bit-slippage. Experiments show substantial robustness improvements while maintaining embedding capacity and resistance to statistical analysis.

arXiv cs.CRSep 4, 10:22 UTC · 12d agoResearch

FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation

FLAT jointly trains a multimodal encoder with text-to-image and image-to-text decoders, producing flexible-length tokens that hit 83.1 GenEval on T2I after fine-tuning.

FLAT (Flexible-Length Aligned Transmodal representations) is a pre-training framework that jointly optimizes a shared multimodal encoder with T2I and I2T decoders, combining contrastive alignment with bidirectional cross-modal generative objectives. It maps visual and textual inputs into a unified continuous 1D sequence space and uses nested dropout over prefix-K tokens for dynamic output lengths. A single pre-training stage supports cross-modal retrieval and generation (71.1 GenEval), with task-specific fine-tuning reaching 83.1 GenEval on T2I, 40.5 BLEU-4 and 138.6 CIDEr on MS-COCO captioning, and strong Recall@5 on MS-COCO and Flickr30K.

Hugging Face daily papersSep 14, 20:00 UTC · 2d agoAI research1

MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education

Introduces MUSE, a twelve-task benchmark evaluating vision-language models on artistic image understanding in situated educational, Southeast Asian contexts.

MUSE is a benchmark assessing large vision-language models on artistic image understanding across twelve tasks spanning visual perception, semantic and affective interpretation, cultural understanding, and compositional reasoning. It decouples image annotation from question generation for controllable difficulty and curates images centering Singaporean and Southeast Asian multicultural contexts alongside Western art. Evaluations of open-source and proprietary models found substantial disparities, especially in affective interpretation and compositional reasoning.

arXiv cs.AI / cs.LG / cs.CLSep 16, 17:26 UTC · 14h agoAI research

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Researchers introduced Brain2Semantics2Text, decoding sentence meaning from non-invasive MEG brain recordings via a semantic bottleneck, improving on prior Brain2Text methods.

The paper proposes Brain2Semantics2Text, a non-invasive speech decoding method that maps sentence-level magnetoencephalography (MEG) responses into a semantic embedding space and inverts those embeddings into natural language. Motivated by evidence that high-level semantic representations are distributed across cortex and evolve on slower timescales, the approach targets meaning rather than phonemes or words, avoiding the need for word-level alignment. The authors report improved sentence-level results compared to prior non-invasive Brain2Text methods despite the low signal-to-noise ratio of neural recordings.

Hugging Face daily papersSep 8, 20:00 UTC · 8d agoAI research2

LLMs and Contextual Integrity

Bruce Schneier highlights two papers: the CIMemories benchmark shows frontier LLMs leak memory attributes up to 69%, and an RL method reduces inappropriate disclosures.

Bruce Schneier discusses contextual integrity in LLMs, referencing the CIMemories benchmark, which uses synthetic profiles with 100+ attributes per user to test whether models with persistent memory disclose sensitive information appropriately. Evaluation showed frontier models exhibit up to 69% attribute-level violations, with GPT-5's violation rate rising from 0.1% to 9.6% across 40 tasks and reaching 25.1% with repeated prompting, showing unstable leakage behavior. A second paper introduces a reinforcement learning framework trained on a synthetic 700-example dataset that substantially reduces inappropriate disclosure while maintaining task performance, with improvements transferring to the human-annotated PrivacyLens benchmark.

Schneier on SecurityAug 18, 10:40 UTC · 29d agoAI safety & security1

Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation

Study shows LLMs with Mixture-of-Agents and QLoRA finetuning effectively simplify medical texts into plain language while preserving content.

The paper evaluates Plain Language Adaptation (PLA) using GPT-4o-mini, Gemini-1.5-pro, and LLaMA in zero-shot and few-shot settings. It compares prompting strategies, QLoRA finetuning across models, and integrates Mixture-of-Agents (MoA) techniques for robustness. Results demonstrate LLM-driven PLA makes healthcare texts more comprehensible while retaining essential content.

arXiv cs.AI / cs.LG / cs.CLSep 15, 16:33 UTC · 1d agoAI research

Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX

Cross-corpus analysis finds gaze patterns modestly correlate with grounding alignment in the MapTask and MUNDEX dialogue corpora.

Researchers mapped HCRC MapTask and MUNDEX annotations into a shared partner/task/away vocabulary and computed gaze features around task-relevant dialogue units. Aligned reference interpretations and understood judgments co-occur with more task-directed gaze, lower gaze entropy, and fewer transitions, clearest for task-leading participants. Effects are small and several weaken when recurring participants rather than dialogues are the unit of inference, so gaze is treated as one contributing cue to grounding.

Hugging Face daily papersSep 15, 20:00 UTC · 1d agoAI research

Do speech foundation models really learn words?

Researchers show via residualization that later layers of HuBERT and wav2vec 2.0 encode word identity and semantics independently of phonetic content.

The study argues that discriminative ability on words does not imply specialized word representations, since good word discrimination can be explained by phoneme encoding alone. By partialling out phoneme information using residualization, the authors show that later layers of HuBERT and wav2vec 2.0 encode words with reasonable fidelity independently of local phonetic content. Applying this disentanglement approach enhances higher-order linguistic information in word discovery tasks, informing analysis of speech foundation models used for recognition and speech tokens.

arXiv cs.AI / cs.LG / cs.CLSep 9, 16:47 UTC · 7d agoAI research

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

Study shows visually grounded token embeddings in a small masked LM persist through training and improve object-property knowledge, but escape standard BabyLM benchmarks.

The paper implements ostensive definition for a small DeBERTa masked language model trained on 10M words, seeding visually grounded tokens with embeddings derived from labeled image regions before training. Visual initialization leaves a persistent, seed-replicated advantage on object-property knowledge (COMPS) and a corpus-tailored Visual-Property Swap benchmark covering color, material, size, and shape, but has no effect on most BabyLM grammar benchmarks. Synthetic grounding of previously unseeded words causally transfers the advantage to exactly those words.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:43 UTC · 6d agoAI research

From Parameters to Answers: How LLMs Retrieve and Use Their Internal Knowledge

Interpretability study traces how Qwen, Llama, and Gemma route query information and internal knowledge across layers when answering questions.

Researchers used layerwise interventions on hidden states to separate query-routing signals from target knowledge in language models, testing Qwen, Llama, and Gemma on country-continent questions with varied answer types. A pair-conditioned request direction strengthens before interventions alter downstream knowledge, opening a causal window while answer-supporting content is still forming. Trajectories differ by model: Gemma shows a partially overlapping mid-layer routing profile, while Llama has no sustained routing-effect window under the same gates.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:39 UTC · 6d agoAI research1