Search: “data-curation”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

DataFlex-RL: An Evaluation Platform for RLVR Data Policies

DataFlex-RL benchmark of 13 RLVR data policies on Qwen2.5-7B finds none reproducibly beats uniform sampling under matched GRPO training.

DataFlex-RL is an evaluation platform comparing rollout-selection, reweighting, and mixture data policies for RLVR under a common GRPO recipe. Across 13 configurations and 12 matched seeds with Qwen2.5-7B-Base on 12 math, logic, and science benchmarks, uniform GRPO improved domain-balanced accuracy by 7.76 points, but no alternative policy achieved a statistically significant improvement. A corrected 12-seed Llama-3.1-8B-Base extension found no consistent winner, and math-heavy evaluation summaries were negatively correlated (-0.33) with domain-balanced summaries.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

35

Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization

ModerationBench shows foundation models can nearly triple Bluesky's moderation F1 (0.60 vs 0.22), with instruction- and example-driven guidance performing comparably.

Researchers built ModerationBench, a new benchmark of 4,000 manually annotated in-the-wild posts from Bluesky, to test whether foundation models can reliably operationalize content moderation policies. They systematically compare instruction-driven guidance (reasoning from policy precepts) with example-driven guidance (generalizing from precedents) for Vision-Language Models. Both paradigms achieve comparable peak effectiveness, and foundation models nearly triple the F1 of Bluesky's deployed moderation system on Random Posts (0.60 vs 0.22).

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:30 UTC · 6d agoAI research1

28

Datamimic – don't let your coding agent invent its own test world

Datamimic is an open-source test data generation tool aimed at keeping coding agents from inventing their own test fixtures.

A Hacker News discussion (40 points) highlights Datamimic, an open-source rapiddweller GitHub project for generating realistic synthetic test data. The tool targets AI coding agents, aiming to prevent them from fabricating their own inconsistent test worlds. Only the repository link was shared, so details are limited.

Hacker News · AI·04:58 UTC · 11h agoAI tools & infraHN 40↑ · 6 comments

15

18 ways to check whether data can be trusted for AI

ETSI published TR 104 180 defining 18 data quality metrics, including fairness and privacy, to assess whether datasets are fit for AI.

ETSI's technical report TR 104 180 defines 18 metrics across four groups - intrinsic quality, usability and lineage, fairness, and privacy - each with calculation formulas, plus an open-source tool that scores datasets. Testing on an aircraft engine sensor dataset and a US census dataset revealed a roughly threefold gender gap in high earners (about 31% of men versus 11% of women) and two privacy failures: re-identification via age, race, sex, and country, and sensitive fields stored in plaintext. The working group included Sejong University, EGM, TTA, Daejeon University, and CNIT.

Help Net Security·Sep 7, 00:00 UTC · 9d agoAI policy

30

Molecular Déjà Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

Audit of 22 frontier models finds widespread verbatim retrieval of published molecular property values, with higher reasoning increasing recall of memorized numbers.

An arXiv audit tests 22 frontier LLMs across 12 molecular regression benchmarks for verbatim retrieval of published values. More than 50% of the LLMs show verbatim retrieval on five datasets, and identical experiments are flagged 89% more often at a high reasoning level than at the lowest one. Suppressing retrieval moves model prediction errors closer together in relative terms, suggesting predictive capability is not determined solely by memorized values.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:32 UTC · 11d agoAI research1

42

ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback

ToolLoop introduces a closed-loop synthetic data framework whose 11K examples lift a 4B model to 86.40% on BFCL tool-use evaluation.

ToolLoop decomposes tool-use data synthesis into function-name sampling, backward derivation of user queries, and forward derivation of tool calls, with dynamic self-feedback at each stage. This shifts the paradigm from generate-then-filter to generate-verify-refine, reducing inefficient and imbalanced synthetic data. A 4B model trained on 11K synthetic examples reaches 86.40% accuracy on BFCL non-reasoning mode (86.07% in an Isolate variant excluding BFCL-overlapping functions) and 72.1% on ACEBench using only 18.3% of baseline training data.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:26 UTC · 7d agoAI research1

30

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

Grouped Value Attention stores grouped values and reconstructs content keys via a learned linear map, cutting KV-cache size about 45-47% versus GQA.

GVA stores only grouped values and reconstructs content keys with a learned linear map absorbed into the query at decode time, while a small shared decoupled RoPE channel preserves positional information via a separately cached positional key. At 350M parameters trained on 30B FineWeb-Edu tokens, the 16-dimensional positional variant scores 44.18 average accuracy across five tasks versus 44.36 for GQA and 43.88 for MLA. Custom decoding kernels are in development with an open-source release planned.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

42

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

Training-free reasoning-plus-retrieval framework lifts multilingual multimodal entity linking accuracy by 6.9% overall and up to 23.3% on rare entities.

The paper broadens rarity measurement in multimodal entity linking using knowledge-graph structural metrics beyond popularity-based pageview metrics, identifying many rare entities that popularity metrics miss. Across these rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, exposing distinct failure modes under different rarity definitions. The proposed training-free framework iteratively searches and reasons over Wikipedia with a reasoning-capable vision-language model; experiments show reasoning and retrieval are complementary. On the MERLIN benchmark covering Hindi, Indonesian, Japanese, Tamil, and Vietnamese, the system improves overall accuracy by 6.9% and up to 23.3% on rare entities, with MERLIN-Rare test slices released.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

35

Notes on gotchas while migrating 35kb preprompts from Opus to self-hosted Ollama

Opinion piece urges migrating 35KB preprompts from Anthropic/OpenAI to self-hosted Ollama, citing session privacy risks and safety filters blocking security research.

The author documents gotchas migrating 35KB preprompts from Claude Opus to self-hosted Ollama, motivated by fears that frontier providers train on user sessions, citing the OpenAI Navier-Stokes controversy. The piece argues inference providers cannot audit their own retention or training pipelines and that only self-hosted hardware offers verifiable privacy. It also criticizes frontier safety filters for refusing vulnerability research tasks and calls for models that support exploitability testing in CI/CD pipelines.

Hacker News · AI·Sep 14, 13:59 UTC · 2d agoAI industryHN 41↑ · 13 comments1

15

Evidence-Grounded Retrieval for Investigation Hunt Lead Generation from CTI Reports

AHLERT generates environment-aware threat hunting leads from CTI reports using ATT&CK-seeded knowledge graph retrieval, roughly doubling F1 over flat-RAG baselines.

The paper introduces AHLERT, a system that converts Cyber Threat Intelligence reports into structured, investigable hunt leads via hybrid dense retrieval with multi-hop traversal over an MITRE ATT&CK-seeded knowledge graph and ontology-grounded RAG constrained to the defender's assets. It is LLM-agnostic and evaluated on public CTI reports for well-known APTs across proprietary and open-weight models. Hybrid evidence retrieval with ontology grounding raises mean F1 from 0.44 to 0.85, and AHLERT attains the highest effectiveness score (~86.95%) versus off-the-shelf LLMs.

arXiv cs.CR·Sep 8, 14:21 UTC · 8d agoResearch

45

Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training

Controlled mid-training experiments on Qwen3-8B-Base find each domain has a 10-40% coverage optimum and domain gaps survive alignment SFT.

Using Qwen3-8B-Base (with a 4B replication) across five semantically rule-disjoint KOR-Bench domains, the authors train 30 data allocations spanning the five-domain simplex at five seeds each. All five domains show interior optima in the moderate 10-40% coverage band, and domain gaps persist after a fixed-budget compensatory SFT pass, which raises 116/120 cells yet bridges 0/240 pairs at a 5% threshold. Zero coverage collapses mid-training-only accuracy, though a FineWeb-Edu-only control shows the collapse is partly generic drift. The results argue mid-training data composition requires principled design rather than reliance on later alignment.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:31 UTC · 7d agoAI research1

30

Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences

SCPTM graph-augmented variational topic model shows syntax aids topic diversity and descriptor quality but gains stem mainly from the variational encoder.

The Structural Contextual Probabilistic Topic Model represents corpora as heterogeneous document-word graphs with lexical and syntactic edges processed by a Graph Attention Network inside a VAE for mixed-membership topic distributions. Across four corpora, neural gains in document-topic alignment are attributable to the variational encoder rather than syntax, while graph-augmented variants improve topic diversity everywhere. Dependency paths add value on argumentative deliberative texts but are redundant in technical and institutional registers.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:39 UTC · 8d agoAI research1

21

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face Blog·Aug 26, 00:00 UTC · 21d agoAI tools & infra1

20

From Concept to Context Engine: How Wiz Built AI-Powered Data Discovery

Wiz details the multi-agent pipeline and feedback loops that evolved its bucket scanner into AI-powered data discovery.

Wiz published an engineering write-up on building its AI-powered data discovery capability, tracing the evolution from a bucket scanner to a context engine. The post explains the multi-agent pipeline and feedback loops behind the system. The article is a vendor engineering deep-dive with no disclosed vulnerabilities, incidents, or exploitation activity.

Wiz Blog·Aug 27, 15:52 UTC · 20d agoTools1

22

CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents

Researchers introduce CUA-Universe, a pipeline turning real desktop software into hybrid GUI+CLI agent environments, lifting a 9B model's OSWorld success rate.

CUA-Universe is an environment-to-data pipeline that converts real desktop applications into hybrid GUI+CLI environments, scaling to 16 applications via App-Forge, Task-Weave, and Path-Steer. Training on its harvested trajectories shifted a 9B model toward effective GUI+CLI orchestration, yielding +39.3 points on CUA-Verse, +16.8 points success rate on OSWorld, and +7.84 points on OSWorld-MCP while cutting steps and tokens by up to 57% and 60%. The work addresses the scarcity of scalable hybrid environments for computer-use agents.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:26 UTC · 11d agoAI research1

38

An Evidence-First Multi-LLM Framework for Auditable Critical-Infrastructure Dependency Modeling

Evidence-first multi-LLM framework builds auditable critical-infrastructure dependency graphs while preserving provenance and unresolved cases.

The framework constructs Infrastructure Knowledge Bases and Infrastructure Dependency Graphs from heterogeneous infrastructure documentation using multiple open-weight LLMs that independently extract candidate entities and dependencies from normalized evidence. It separates evidence verification, ontology grounding, entity resolution, dependency alignment, validation, fusion, and human review, projecting the validated IKB deterministically into the IDG without new LLM-generated knowledge. Evaluation across nine infrastructure projects shows entity recovery achieves substantially higher recall than full dependency recovery, and cross-model overlap is much lower for dependencies than entities, indicating models often produce non-overlapping candidate assertions rather than stable consensus.

arXiv cs.CR·Sep 11, 02:33 UTC · 5d agoResearch

28

Distributed and Private Textual Data Synthesis from Embeddings

Researchers propose a distributed differentially private text synthesis method combining DP summaries and secure protocols, removing the need for a trusted curator.

The paper presents a differential privacy and cryptography co-design for synthesizing textual training data without a trusted curator or tightly synchronized user participation. It releases a one-time DP summary in embedding space, identifying frequent semantic regions and their DP centroids to enable training-free offline text synthesis, with semantic support protection to avoid exposing rare user texts. A custom secure protocol enforces end-to-end DP guarantees over distributed user data. Across four benchmarks the approach achieves utility comparable to the state-of-the-art centralized DP synthesis method.

arXiv cs.CR·Sep 9, 12:33 UTC · 7d agoAI research

25

Generative Late-Interaction Embeddings For Visual Document Retrieval

GLIE compresses visual document retrieval embeddings to four vectors per page while retaining nearly 80% of uncompressed nDCG@5 accuracy.

Researchers analyzing late-interaction retrieval embeddings found they lie exactly on the unit sphere and concentrate near a manifold of intrinsic dimension five to six. GLIE learns a few k vectors per page that serve as a lightweight index and a basis to regenerate the full embedding set for exact rescoring of top candidates at query time. On ViDoRe v1 with four vectors per page, GLIE retains nearly 80% of uncompressed nDCG@5 versus 70% for the best prior post-hoc method, using a 415K-parameter network trained in under three GPU-minutes on 1,000 pages.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

28

An Empirical Analysis of ReDoS Vulnerabilities and ReDoS Detection Tools

Study of NVD data finds ReDoS vulnerabilities growing more prevalent and more likely to be exploited, while five detection tools disagree substantially.

The study compares five publicly available ReDoS detection tools and one regex correction tool across three datasets. An empirical analysis of all ReDoS vulnerabilities reported to the NVD finds they are becoming more prevalent and are much more likely to be exploited than non-ReDoS weaknesses. The detection tools exhibited substantial disagreement on whether a given regex is vulnerable.

arXiv cs.CR·Sep 9, 15:09 UTC · 7d agoResearch

35

Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing

Researchers introduce KnowChange, a framework that uses pretrained vision-language models to synthesize realistic change-detection training data for remote sensing.

KnowChange is a knowledge-guided change data synthesis framework that leverages pretrained vision-language models to reason about plausible change locations and class transitions from pre-change scenes and desired change types. It addresses the limited class-transition coverage and inflexibility of handcrafted rule-based synthesis methods, enabling diverse change types in a unified pipeline. Experiments show KnowChange-generated data outperforms existing synthetic datasets in both synthetic-to-real transfer and synthetic data augmentation, despite compact generation scale.

Hugging Face daily papers·Aug 24, 20:00 UTC · 22d agoAI research

25

Security Data Isn’t the Problem. Security Context Is.

Horizon3 blog argues security context, not data volume, is the SOC bottleneck, promoting its NodeZero integration with CrowdStrike Falcon Next-Gen SIEM.

Horizon3.ai published a vendor blog explaining how its NodeZero Proactive Security Platform integration with CrowdStrike Falcon Next-Gen SIEM brings validated exposure findings into existing security operations workflows. The post argues SOCs are now limited by confidence rather than visibility, needing context to decide which issues matter. It cites a global chemical manufacturer that validated exploitable exposures with NodeZero before completing a $2 billion merger.

Horizon3.ai·Sep 15, 13:12 UTC · 1d agoTools

15

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

Study shows visually grounded token embeddings in a small masked LM persist through training and improve object-property knowledge, but escape standard BabyLM benchmarks.

The paper implements ostensive definition for a small DeBERTa masked language model trained on 10M words, seeding visually grounded tokens with embeddings derived from labeled image regions before training. Visual initialization leaves a persistent, seed-replicated advantage on object-property knowledge (COMPS) and a corpus-tailored Visual-Property Swap benchmark covering color, material, size, and shape, but has no effect on most BabyLM grammar benchmarks. Synthetic grounding of previously unseeded words causally transfers the advantage to exactly those words.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:43 UTC · 5d agoAI research

20

UC Berkeley Researchers Release CUA-Lite, an Open Platform Unifying Sandboxes, Data, Evaluation and RL for Computer-Use Agents

UC Berkeley's CUA-Lite is an open platform unifying computer-use agent sandboxes, datasets, evaluation and RL; Lite.OSWorld cuts OSWorld memory 4.1 GB to 0.9 GB.

UC Berkeley researchers released CUA-Lite, an open platform placing agents, environments, traces, and training for computer-use agents behind one action space, one LiteSample schema, and one command across desktop, browser, and mobile. Lite.OSWorld reproduces the OSWorld task suite and evaluators in plain Docker containers (0.9 GB RAM vs 4.1 GB, cold start 23.8s, ~4.6× more parallel instances), with scores matching the QEMU/KVM VM across 13 models. The platform claims 30k+ verifiable tasks, 15+ benchmarks, 10+ agents, and 20+ datasets on Hugging Face including Aguvis, OpenCUA, and ScaleCUA. A documented SFT run lifts Qwen3-VL-2B-Instruct mean episode return from 0.138 to 0.237 on the 332-task lite.osworld split.

MarkTechPost·Sep 6, 06:11 UTC · 10d agoAI tools & infra1

45

Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation

Benchmark Radar provides a living searchable database of 1,283 AI benchmark records and 12,916 score observations drawn from 37 daily discovery sources.

Benchmark Radar combines daily discovery of benchmark papers, repositories, datasets, and releases from 13 direct connectors and 24 first-party feeds into a searchable catalog with model card mentions and score histories. The catalog contains 1,283 source records drawn from 4 benchmark catalogs plus 12,916 numeric observations on 790 records. The release includes a web dashboard with leaderboard, Pareto frontier of score versus usage, saturation and trend views, daily feeds, a CLI, and reproducible analysis. The paper audits the full catalog and examines benchmark saturation and limits of score comparisons.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

25

Type Diversity Enables Transformers to Generalise Compositionally

Researchers show lexical-versus-structural compositional generalization gaps in Transformers stem from type diversity imbalance in datasets, not architectural limits.

The paper argues that Transformers' difficulty with structural compositional generalization is an artifact of low structural type diversity in prior benchmark datasets rather than an architectural limitation. Using Grammatical Framework, the authors create linguistically diverse variants of COGS and SLOG. They find type diversity correlates with compositional generalization equally in lexical and structural test cases, contradicting previous claims that compound divergence explains task difficulty.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:59 UTC · 4d agoAI research

20

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

LimiX-2 scales Contextual Mechanism Networks pretrained via context-conditional masked modeling, beating tabular foundation models on TabArena, TALENT, and BCCO benchmarks.

LimiX-2 is a new tabular model in the LimiX family, developed through model and data scaling guided by previously established scaling laws under the Contextual Mechanism Networks (CMNs) paradigm. It is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models spanning diverse graph structures, functional mechanisms, and observation processes. It outperforms dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO, and its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:30 UTC · 22h agoAI research1

18

Enoki: Efficient Multi-Level Hallucination Detection

Researchers introduce Enoki, an open information extraction framework unifying claim-level and span-level hallucination detection in LLMs at lower inference cost.

Enoki extracts text-anchored relational facts, verifies them against evidence, and projects unsupported facts back onto hallucinated spans, so claim-level verification and span-level localization share one representation without separate alignment. It supports LLM-based, encoder-based, and rule-based extraction regimes to balance accuracy against inference cost. Experiments show it stays competitive with strong claim-level systems while using fewer resources and outperforms them on fine-grained span- and entity-level localization. The authors also release EnokiQA, a dual-granularity dataset with aligned claim-level verification and span-level localization annotations.

Hugging Face daily papers·Aug 31, 22:21 UTC · 15d agoAI research

50

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · security·Sep 12, 06:42 UTC · 4d agoResearch

45

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research1

40

Securing Data in the AI era

Wiz published guidance arguing AI adoption reshapes data risk, urging teams to identify what data is connected to AI services and exposed.

Wiz published a viewpoint on how AI adoption changes the context of data risk. The post argues that understanding what data is connected to AI systems, what is exposed, and why is now critical for managing that risk. No specific incident, vulnerability, or product release is described in the available text.

Wiz Blog·Aug 14, 12:19 UTC · Aug 14, 2026Industry

15