Search: “information-extraction”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Evidence-Grounded Retrieval for Investigation Hunt Lead Generation from CTI Reports

AHLERT generates environment-aware threat hunting leads from CTI reports using ATT&CK-seeded knowledge graph retrieval, roughly doubling F1 over flat-RAG baselines.

The paper introduces AHLERT, a system that converts Cyber Threat Intelligence reports into structured, investigable hunt leads via hybrid dense retrieval with multi-hop traversal over an MITRE ATT&CK-seeded knowledge graph and ontology-grounded RAG constrained to the defender's assets. It is LLM-agnostic and evaluated on public CTI reports for well-known APTs across proprietary and open-weight models. Hybrid evidence retrieval with ontology grounding raises mean F1 from 0.44 to 0.85, and AHLERT attains the highest effectiveness score (~86.95%) versus off-the-shelf LLMs.

arXiv cs.CR·Sep 8, 14:21 UTC · 8d agoResearch

45

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face Blog·Aug 26, 00:00 UTC · 22d agoAI tools & infra1

20

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:38 UTC · 12d agoAI research

14

LandingAI Releases Agentic Document Extraction Gen2 with DPT-3 Pro and DPT-3 Verity

LandingAI shipped Agentic Document Extraction Gen2 with DPT-3 Pro and DPT-3 Verity parsing models, adding usage-based billing, block-tree outputs, and word-level grounding.

LandingAI has generally released Agentic Document Extraction Gen2, rebuilt around two parsing models: DPT-3 Verity for deterministic transcription of digital documents with per-word bounding boxes and confidence scores, and DPT-3 Pro for layout-aware parsing of scans, handwriting, non-Latin scripts, and LaTeX math. Billing changes from a flat 3 credits per page to a page-plus-output-character model (Pro: 1 credit/page plus 0.5 credits per 1,000 output characters on priority; Verity: 0.3 plus 0.2), with an asynchronous standard tier at 0.5x price and vendor-claimed 25-80% cost reductions. Parse v2 returns a document-page-block tree with semantic IDs, normalized bounding boxes, and line- or word-level atomic grounding, replacing flat chunks; Gen1 client code will not run against Gen2 endpoints. Deployment options include US/EU cloud, VPCs on AWS, Azure, and Google Cloud, Snowflake, and air-gapped on-premises environments, with automated model routing planned for fall 2026.

MarkTechPost·Sep 10, 02:13 UTC · 7d agoAI tools & infra

25

Enoki: Efficient Multi-Level Hallucination Detection

Researchers introduce Enoki, an open information extraction framework unifying claim-level and span-level hallucination detection in LLMs at lower inference cost.

Enoki extracts text-anchored relational facts, verifies them against evidence, and projects unsupported facts back onto hallucinated spans, so claim-level verification and span-level localization share one representation without separate alignment. It supports LLM-based, encoder-based, and rule-based extraction regimes to balance accuracy against inference cost. Experiments show it stays competitive with strong claim-level systems while using fewer resources and outperforms them on fine-grained span- and entity-level localization. The authors also release EnokiQA, a dual-granularity dataset with aligned claim-level verification and span-level localization annotations.

Hugging Face daily papers·Aug 31, 22:21 UTC · 16d agoAI research

50

Unifying Conformal Language Tasks with In-Context Ensembles

Researchers propose Conformal Relevance, which builds conformal score functions via in-context example curation and ensembling to improve conciseness across seven NLP tasks.

The paper targets NLP tasks like summarization and extractive QA that reduce to retrieving content under coverage and conciseness constraints. Conformal Relevance replaces hand-engineered LLM scoring prompts with curated in-context examples and ensembles, maintaining coverage guarantees while improving conciseness with minimal manual input. The authors demonstrate the framework on seven NLP tasks and contribute theory, including a complementarity condition for when ensembling improves worst-case sentence scores and a saturation bound on ensemble gains.

Hugging Face daily papers·Sep 1, 20:00 UTC · 15d agoAI research1

42

Objective vs. Search: Decomposing What Makes a Good Tokeniser

New tokeniser study shows search procedure, not optimisation objective, drives bits-per-byte performance across model sizes, vocabulary sizes, and multilingual settings.

The paper disentangles BPE and UnigramLM along two axes: optimisation objective (compression vs log-likelihood) and search procedure (bottom-up merging vs top-down pruning). Two new algorithms, BottomUpLL and TopDownComp, complete the 2x2 design space, and trained language models are evaluated on bits-per-byte and BLiMP across model sizes, vocabulary sizes, and English-only vs multilingual domains. Bottom-up tokenisers consistently achieve lower bits-per-byte in most settings, while BLiMP shows no consistent relationship with design choice.

arXiv cs.AI / cs.LG / cs.CL·Sep 16, 17:59 UTC · 12h agoAI research

28

LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics

LexFlip releases 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving tokens, exposing weaknesses in embedding-based meaning preservation metrics.

LexFlip provides 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of tokens, creating dissociation items that break monotone token-overlap metric validation. The seven embedding and BERTScore metrics tested register only 0.022-0.039 of their identical-to-unrelated range on these edits, versus 0.670 for bidirectional NLI. Against FrJudge, with a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric tested.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 15:49 UTC · 12d agoAI research

22

MAxBench: A Multinomial Concept Recovery Benchmark

MAxBench evaluates multinomial concept recovery methods, finding affine subspaces steer most reliably but none consistently beats prompting.

MAxBench is a geometry-agnostic evaluation framework for multinomial concept representations in language models, based on sampling from recovered concept representations. It compares 10 localization methods covering 5 geometry types across 6 concepts and 4 models. Findings show affine subspaces steer more reliably than rank-one or linear subspaces due to better non-zero offsets, manifold steering is competitive where applicable, and no method consistently outperforms prompting.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:08 UTC · 5d agoAI research

18

Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences

SCPTM graph-augmented variational topic model shows syntax aids topic diversity and descriptor quality but gains stem mainly from the variational encoder.

The Structural Contextual Probabilistic Topic Model represents corpora as heterogeneous document-word graphs with lexical and syntactic edges processed by a Graph Attention Network inside a VAE for mixed-membership topic distributions. Across four corpora, neural gains in document-topic alignment are attributable to the variational encoder rather than syntax, while graph-augmented variants improve topic diversity everywhere. Dependency paths add value on argumentative deliberative texts but are redundant in technical and institutional registers.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:39 UTC · 9d agoAI research1

21

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

Training-free reasoning-plus-retrieval framework lifts multilingual multimodal entity linking accuracy by 6.9% overall and up to 23.3% on rare entities.

The paper broadens rarity measurement in multimodal entity linking using knowledge-graph structural metrics beyond popularity-based pageview metrics, identifying many rare entities that popularity metrics miss. Across these rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, exposing distinct failure modes under different rarity definitions. The proposed training-free framework iteratively searches and reasons over Wikipedia with a reasoning-capable vision-language model; experiments show reasoning and retrieval are complementary. On the MERLIN benchmark covering Hindi, Indonesian, Japanese, Tamil, and Vietnamese, the system improves overall accuracy by 6.9% and up to 23.3% on rare entities, with MERLIN-Rare test slices released.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research

35

Probabilistic Linear Explanations

Researchers introduce a unified probabilistic explainability framework using sparse anchored linear models that outperforms LIME and MAPLE on relevance error.

The paper proposes probabilistic explanations based on sparse, anchored linear models applicable to both binary classification and continuous regression. It proves that minimizing relevance error for neural-network models is NP-hard and relates it to a tractable fidelity-error surrogate. Solutions are computed via a mixed integer programming formulation with provably optimal empirical solutions and a polynomial-time iterative hard thresholding algorithm with approximation guarantees. Empirical evaluations show lower relevance error than LIME and MAPLE while satisfying anchoring and sparsity constraints by construction.

arXiv cs.AI / cs.LG / cs.CL·Sep 16, 17:18 UTC · 13h agoAI research

25

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · security·Sep 12, 06:42 UTC · 5d agoResearch

45

General Quantification of Covariate and Concept Shifts

Paper proposes γ*-concept shifts via entropic optimal transport, deriving estimable generalization bounds unifying covariate and concept shift under distribution shift.

The authors show existing definitions of concept shift break when source and target supports mismatch and propose γ*-concept shifts grounded in entropic optimal transport. They derive a general error bound covering broad loss functions, label spaces and stochastic labeling, plus estimators with concentration guarantees. The resulting DataShifts algorithm quantifies distribution shifts and estimates the error bound in most applications, addressing learning bounds that were previously non-estimable from samples.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research1

28

Product showcase: AI Paper Trail shows the privacy cost of talking to AI

Proton launched AI Paper Trail, a free tool that analyzes ChatGPT or Claude exports and reports what personal data can be inferred from AI conversations.

Proton released AI Paper Trail, a free web tool that analyzes the 200 most recent prompts from exported ChatGPT or Claude conversation histories and generates a privacy report with an AI Exposure Score, inferred personal data categories, and an estimated advertising value. In a hands-on test it identified 47 data points, returned a 58/100 exposure score, estimated $185 in advertising value, and flagged five red flags spanning location, interests, finances, and relationships. Proton states that uploaded data is deleted after analysis and is not stored on its Lumo servers.

Help Net Security·Aug 24, 00:00 UTC · 24d agoAI industry

18

Anchoring Clinical Events in Time: UID-Preserving Multimodal Reconstruction and Source-Grounded Adjudication

UID-preserving multimodal framework plus GAVEL LLM judge improves clinical timeline reconstruction, boosting event recovery 43% over prior matching.

The paper introduces a UID-preserving framework linking each narrative clinical event to its source span through text-only estimation, structured-evidence retrieval, timestamped source-row grounding, and joint revision. GAVEL, an LLM judge, compares UID-aligned timelines against narrative and structured records. Across six open-weight models and 40 mixed-critical-care summaries, GLM 5.2 multimodal revision improved temporal agreement without reducing event recovery and performed competitively with clinician annotations, while DeepSeek V3.2 did not benefit from multimodality. The pipeline achieves 43% increased event recovery with occurrence-level provenance.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:01 UTC · 5d agoAI research1

30

Google Research Introduces Retrieve-for-Train (R4T): An RL-Compiled Diffusion Retriever for 12× to 20× Faster Query Fan-Outnew

Google Research introduced R4T, an RL-trained fan-out pipeline distilled into a 53.9M-parameter diffusion retriever achieving 12x-20x faster query fan-out.

Google Research introduced Retrieve-for-Train (R4T), which trains a fan-out language model with GRPO plus soft PPO regularization, then distills query fan-out into a 53.9M-parameter diffusion transformer that generates all retrieval embeddings in a single non-autoregressive pass. A three-term reward (groundedness 0.6, diversity 0.2 via Vendi Score, alignment 0.2) prevents paraphrastic collapse and reward hacking during training. On the Polyvore dataset, Gemma3-4B R4T-FOLM averaged 49.1 versus 40.9 for Best-of-N, and the diffusion retriever cut fan-out latency from 1.46s to 0.07s at batch size 8, a consistent 12x-20x speedup over autoregressive methods.

MarkTechPost·06:19 UTC · 30m agoAI research

42

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Researchers introduced Brain2Semantics2Text, decoding sentence meaning from non-invasive MEG brain recordings via a semantic bottleneck, improving on prior Brain2Text methods.

The paper proposes Brain2Semantics2Text, a non-invasive speech decoding method that maps sentence-level magnetoencephalography (MEG) responses into a semantic embedding space and inverts those embeddings into natural language. Motivated by evidence that high-level semantic representations are distributed across cortex and evolve on slower timescales, the approach targets meaning rather than phonemes or words, avoiding the need for word-level alignment. The authors report improved sentence-level results compared to prior non-invasive Brain2Text methods despite the low signal-to-noise ratio of neural recordings.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research2

25

Do speech foundation models really learn words?

Researchers show via residualization that later layers of HuBERT and wav2vec 2.0 encode word identity and semantics independently of phonetic content.

The study argues that discriminative ability on words does not imply specialized word representations, since good word discrimination can be explained by phoneme encoding alone. By partialling out phoneme information using residualization, the authors show that later layers of HuBERT and wav2vec 2.0 encode words with reasonable fidelity independently of local phonetic content. Applying this disentanglement approach enhances higher-order linguistic information in word discovery tasks, informing analysis of speech foundation models used for recognition and speech tokens.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:47 UTC · 7d agoAI research

28

Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models

Researchers release Explainability Assistant, an open-source conversational XAI tool using LLM function calling, lifting intent-parsing accuracy from 76.8% to 94%.

The paper introduces the Explainability Assistant, an open-source conversational XAI system for interpreting energy consumption forecasting models such as genetic-programming symbolic regressors. It uses LLM function calling instead of rigid custom grammars, achieving 94% intent-parsing accuracy versus 76.8% for prior work TalkToModel, and adapts to different ML problem types without task-specific fine-tuning. Comparative evaluation with energy domain specialists against a traditional XAI dashboard showed improved usability, with all experts preferring the conversational interface.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:40 UTC · 6d agoAI research1

25

From Infostealer Log to Marketplace Listing: A Technical Walkthrough of the Credential Theft Pipeline

Cyble walkthrough maps how infostealer logs move from endpoint infection through aggregation and enrichment to dark web credential marketplace sales.

Cyble breaks the credential theft pipeline into stages: infostealer execution harvesting browser credential stores, cookies, session tokens, crypto wallets, and FTP configurations; aggregation of stealer logs via C2 panels into bundled archives; parsing and enrichment against previously leaked datasets; and final listing on dark web marketplaces. Enrichment adds employer and role context that raises prices and enables credential stuffing across reused passwords. The report advises SOC teams to monitor stealer logs and marketplace chatter early rather than waiting for breach alerts.

Cyble·Sep 10, 10:45 UTC · 6d agoMalware1

30

Fluid Notarization: Verifiable Evolution of Concurrently Edited Structured Documents

Fluid Notarization anchors delta-CRDT change graphs on blockchain, providing verifiable provenance for concurrently edited documents, demonstrated on collaborative electronic health records.

The paper introduces Fluid Notarization, a paradigm that notarizes the evolution of collaboratively edited structured documents rather than isolated snapshots. It builds on Melda, a JSON-native delta-CRDT representing changes as compact content-addressed deltas linked by causal dependencies, with blockchain notarization reduced to recording identifiers of evolution artifacts while synchronization, reconstruction, and conflict resolution remain off-chain. The architecture combines deterministic CRDT convergence with independently auditable proof-of-existence, provenance, and publication evidence, validated through a prototype based on collaboratively edited electronic health records.

arXiv cs.CR·Sep 16, 16:20 UTC · 14h agoResearch

26

Introducing OlmoEarth embeddings: Custom embedding exports from OlmoEarth Studio for downstream analysis

AllenAI's OlmoEarth Studio adds custom embedding exports to support downstream geospatial analysis workflows.

A Hugging Face blog post from AllenAI introduces OlmoEarth embeddings, a feature allowing custom embedding exports from OlmoEarth Studio for downstream analysis tasks. Only the title was available, so no benchmark or performance details are provided. OlmoEarth is Ai2's open geospatial AI model family.

Hugging Face Blog·Aug 12, 16:14 UTC · Aug 12, 2026AI tools & infra

32

ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation

ENEAS adds text prompting and semantic verification to video segmentation to keep tracking targets through occlusion and reject lookalike distractors.

ENEAS is a unified text-promptable method for instance tracking and open-concept semantic discovery in video, designed to fix temporal hallucinations, spatial fragmentation, and semantic misclassification seen in SAM 3-class foundation models. It extends the geometrically robust SeC architecture with a text-prompting adapter and temporal memory, and uses a verification layer combining fast visual embedding matching with conditional VLM refinement for ambiguous candidates. It targets 3D reconstruction pipelines where a single misclassified distractor corrupts the asset. Code and models are open-sourced.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

32

Atlas: Efficient Verifiable Semantic Search

Atlas delivers zero-knowledge proofs for HNSW semantic search, verifying RAG retrieval in under a second on SIFT1M and 2.0 seconds at 100M vectors.

Atlas lets a search provider prove that a query was answered correctly against a committed HNSW index without revealing the index, addressing provider deviations like truncation or bias. It combines offline preprocessing, a fixed-size-state restructuring of HNSW with a correctness proof, and timestep-tagged batching of per-step arguments. The system proves queries in under a second on SIFT1M and 2.0 seconds at 100 million vectors while preserving plaintext HNSW recall, and proven retrieval maintains end-to-end RAG answer quality at lower cost than prior verifiable retrieval systems.

arXiv cs.CR·Sep 10, 17:30 UTC · 6d agoResearch3

40

Optimal Rates for Agentic Networked Information Aggregation

Researchers close the Kearns–Roth–Ryu gap for agentic networked information aggregation, proving excess error is constant up to depth M^2 then Θ(M^2/D).

The paper studies a networked learning model where agents in a DAG each see only a subset of features and pass only their predictions forward. It sharpens the earlier lower bound to Ω(√(M/D)) for depth below M^2 and constructs M-covered paths of depth D ≥ M^2 achieving Ω(M^2/D) excess error, establishing the correct rate for both regression and logistic classification. It also shows excess error contracts geometrically along the path for any fixed distribution, ruling out a single instance that witnesses polynomial lower bounds at every depth.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:12 UTC · 12d agoAI research

28

ReCite: Agentic Reasoning for Faithful Citation

ReCite is an agentic citation framework using claim-level reasoning and verification, outperforming large generative models in strict citation accuracy.

ReCite is a decoupled agentic framework that orchestrates location perception, intent-aware query planning, and reflective verification for citation recommendation. Trained on synthesized reasoning trajectories, the agent verifies claim-evidence consistency and triggers self-correction loops when retrieved candidates lack logical support. Experiments show the lightweight framework outperforms state-of-the-art massive generative models in strict citation accuracy, addressing misattribution where cited papers are real but logically unsupportive.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:59 UTC · 8d agoAI research

25

CiteShade: Citation Laundering in Multi-Source Retrieval-Augmented Generation and Its Counterfactual Defense

CiteShade attack makes RAG models cite trusted sources for attacker-chosen wrong answers, raising wrong-answer rate from 0.01 to 0.68.

CiteShade is presented as the first citation laundering attack against multi-source retrieval-augmented generation: an attacker controlling a single source induces a wrong answer falsely attributed to a trusted source, even while correct evidence remains in context. The attack is formalized via three necessary conditions (retrieval, generation, citation) constructible without any instructions, raising wrong-answer rate from 0.01 to 0.68 on multi-hop QA, with source deletion confirming the malicious source as causal driver. Vulnerability tracks a model's citation propensity rather than scale, reaching CLR 0.84 with explicit instruction and 0.64 without on the most citation-prone model. Perplexity filtering and citation-support checking prove insufficient; the authors propose a counterfactual defense verifying which source actually drove the answer.

arXiv cs.CR·Sep 14, 14:41 UTC · 2d agoAI safety & security1

45

An Empirical Analysis of ReDoS Vulnerabilities and ReDoS Detection Tools

Study of NVD data finds ReDoS vulnerabilities growing more prevalent and more likely to be exploited, while five detection tools disagree substantially.

The study compares five publicly available ReDoS detection tools and one regex correction tool across three datasets. An empirical analysis of all ReDoS vulnerabilities reported to the NVD finds they are becoming more prevalent and are much more likely to be exploited than non-ReDoS weaknesses. The detection tools exhibited substantial disagreement on whether a given regex is vulnerable.

arXiv cs.CR·Sep 9, 15:09 UTC · 7d agoResearch

35

Generative Late-Interaction Embeddings For Visual Document Retrieval

GLIE compresses visual document retrieval embeddings to four vectors per page while retaining nearly 80% of uncompressed nDCG@5 accuracy.

Researchers analyzing late-interaction retrieval embeddings found they lie exactly on the unit sphere and concentrate near a manifold of intrinsic dimension five to six. GLIE learns a few k vectors per page that serve as a lightweight index and a basis to regenerate the full embedding set for exact rescoring of top candidates at query time. On ViDoRe v1 with four vectors per page, GLIE retains nearly 80% of uncompressed nDCG@5 versus 70% for the best prior post-hoc method, using a 415K-parameter network trained in under three GPU-minutes on 1,000 pages.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

28