Search: “RHEL”

Red Heron Exploits Gitea RCE to Compromise 13 Organizations Across Six Countries

China-linked Red Heron exploited Gitea RCE CVE-2026-60004 to compromise 13 organizations across six countries, stealing source code.

Acronis Threat Research Unit attributes a China-nexus actor tracked as Red Heron to rapid exploitation of Gitea RCE CVE-2026-60004, scanning 1,386 instances across seven countries plus 477 Taiwan-based systems, with confirmed compromises of 13 organizations in Canada, Argentina, Taiwan, the U.S., Qatar, and Sri Lanka. Within days of the July 2026 disclosure, the actor weaponized a public PoC into an automated Python framework registering accounts, exploiting servers, stealing repositories, and removing traces, then deployed the JITTERLY C++ Linux implant (30+ post-exploitation commands) and the undocumented SIXZUT LD_PRELOAD rootkit. In one Taiwanese environment the actor reached root-level access across a three-node Proxmox cluster, targeting defense, elections, energy, aerospace, telecom, government, and research sectors.

The Hacker News·updated Sep 15, 11:16 UTC · 1d ago·first Sep 14, 16:56 UTC · 2d agoThreat actor in the wild 3 sourcesCVE-2026-60004

18-Year-Old Linux SCTP Flaw Could Let Local Users Gain Root and Escape Containers

The Hacker News·Aug 7, 11:10 UTC · Aug 7, 2026Exploit / PoC in the wildCVE-2026-64564

DataFlex-RL: An Evaluation Platform for RLVR Data Policies

DataFlex-RL benchmark of 13 RLVR data policies on Qwen2.5-7B finds none reproducibly beats uniform sampling under matched GRPO training.

DataFlex-RL is an evaluation platform comparing rollout-selection, reweighting, and mixture data policies for RLVR under a common GRPO recipe. Across 13 configurations and 12 matched seeds with Qwen2.5-7B-Base on 12 math, logic, and science benchmarks, uniform GRPO improved domain-balanced accuracy by 7.76 points, but no alternative policy achieved a statistically significant improvement. A corrected 12-seed Llama-3.1-8B-Base extension found no consistent winner, and math-heavy evaluation summaries were negatively correlated (-0.33) with domain-balanced summaries.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

Top 10 Best Server Security Solutions in 2026

Cyber Security News ranks 2026 server security platforms, favoring Trend Micro for hybrid estates, CrowdStrike for detection, Defender for Azure economics.

Cyber Security News published an editorial ranking of ten server security platforms for 2026, favoring Trend Micro for hybrid estates, CrowdStrike for detection quality, and Microsoft Defender for Servers for Azure-centric economics. The guide stresses that server security licensing differs (per server, per core, or per cloud-hour) and flags hypervisor protection as a commonly missed coverage gap.

Cyber Security News·Sep 10, 08:23 UTC · 6d agoIndustry1

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

PARSER uses parallel reader subagents and an RL-trained lead agent for long-context QA, beating baselines and cutting latency up to 11x.

The PARSER paper decouples reading from reasoning: frozen subagents each read one document chunk in parallel while an RL-optimized lead agent iteratively broadcasts queries and aggregates evidence in scatter-gather rounds. On multi-hop QA with 7K to 896K token contexts, a 4B-backbone PARSER beats the strongest sequential memory baseline by 5.7 points on average and 12.0 points at 896K tokens, and a 9B version surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments show robustness to evidence position, order, and distance perturbations, with inference latency reduced by up to 11x.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research1

12 Best Server Security Solutions Compared (2026): Features & Pricing

GBHackers ranks 12 server security solutions for 2026, naming CrowdStrike and SentinelOne as server EDR leaders and Trend Micro Deep Security top for virtual patching.

The guide scores 12 server security platforms across five weighted criteria, with detection and response depth and Linux parity weighted 25% each. CrowdStrike and SentinelOne lead server EDR, Trend Micro Deep Security is highlighted for virtual patching of unpatchable estates, and Microsoft Defender for Servers is noted for Azure and hybrid economics. It is an editorial assessment comparing features and pricing models rather than a lab test.

GBHackers·updated 09:52 UTC · 8h ago·first Sep 11, 07:24 UTC · 5d agoIndustry 14 sources1

Reducto Releases r-1: A Single Pass Document Parsing Model That Cuts Errors 20% at 1 Cent Per Page

Reducto launched r-1, a single-pass document parsing model claiming 20% error reduction over its legacy agentic pipeline, priced at 1 cent per page.

Reducto announced r-1, the first model in a new parsing family that replaces multi-stage agentic OCR with one full-page pass handling text, tables, figures, layout, formatting, and grounding with page-relative bounding boxes. The company reports a 20% error reduction measured against its own legacy agentic pipelines, plus vendor-run wins over Amazon Textract and Azure Document Intelligence on complex documents. Pricing is a flat 1 cent per page versus 3-6 cents for legacy models; r-1 is available in preview via the V3 Parse API with no open weights.

MarkTechPost·Sep 8, 03:07 UTC · 8d agoModel release1

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:38 UTC · 12d agoAI research

ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR

ThinkPrior builds zero-rollout difficulty priors via an offline verifier-anchored pass, halving silent groups in RLVR and cutting wasted rollouts on Qwen2.5-Math-7B.

In GRPO-based RLVR, groups where all rollouts are correct or all are wrong yield zero advantages and consume about 39% of a run's rollouts under uniform sampling. ThinkPrior initializes a Beta posterior from an external anchor pass's verifier-scored pass rate, selecting prompts by expected learnability before any target-policy rollout, without changing the loss or optimizer. On Qwen2.5-Math-7B across sixteen seeds it more than halves early silent groups and cuts wasted rollouts through step 30 by nearly a fifth, with no detected final-accuracy difference. The ThinkPrior+DAPO composition reduces generated rollouts by 10.6% at an equal 3,840-rollout update budget.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:30 UTC · 8d agoAI research1

ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

ActReview post-trains Qwen3-8B-Base on OpenReview rebuttals to generate actionable peer-review feedback with grounded revision suggestions, benchmarked on 1,000 curated instances.

The paper defines Actionable Peer-review Generation as diagnostic claim generation plus revision suggestion generation and introduces ActReview, a rebuttal-guided post-training framework. From OpenReview review-rebuttal threads the authors build ActReview-40K, aligning reviewer weaknesses with author responses grounded in localized paper evidence, and post-train Qwen3-8B-Base with multi-task SFT followed by GRPO using weakness-specific rubric rewards. They also release ActReview-Bench, a human-curated 1,000-instance benchmark, on which ActReview outperforms prior specialized review-generation models on actionability and grounding while remaining competitive with strong prompt-based LLMs. Human evaluation confirms improved revision usefulness but identifies a remaining gap in technical accuracy.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

TokenRhythm/NeoHorse-1-4B — new model trending #30 on Hugging Face

TokenRhythm releases NeoHorse-1-4B, an Apache-2.0 agentic fine-tune of Qwen3.5-4B claiming +5.93 benchmark macro-average gain.

NeoHorse-1-4B is a roughly 4B-parameter text-only causal language model post-trained by TokenRhythm from Qwen/Qwen3.5-4B for agent harnesses, tool use, coding, and instruction following. It applies routing-guided curriculum SFT and routing-guided on-policy distillation over execution trajectories as an early prototype toward recursive self-improvement (RSI). The release reports a 64.87 macro average across ten benchmarks versus 58.94 for Qwen3.5-4B (+5.93) and is distributed under Apache-2.0, trending #30 on Hugging Face.

Hugging Face trending models·Sep 5, 08:21 UTC · 11d agoModel release1

MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control

MobileVLA-R1 2.0 couples chain-of-thought reasoning with RL for mobile robot control, gaining 10 points on real Unitree G1 tasks.

MobileVLA-R1 2.0 is an RL-enhanced vision-language-action framework that explicitly couples structured embodied reasoning with executable mobile robot control via supervised Chain-of-Thought alignment and reinforcement learning. A reasoning-conditioned action decoder maps multimodal reasoning representations to task-level action targets, decoupling high-level action generation from robot-specific actuation for both locomotion and manipulation. It achieves an average 1.6 point SR improvement on VLN-CE and a 10.0 point improvement in full-task success on real-world Unitree G1 mobile manipulation, with evaluations covering navigation, quadruped control, and real deployments on Unitree Go2 and G1 robots.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

VDiff-Bench: A Challenging Benchmark for Fine-Grained Image Difference Identification

VDiff-Bench, a 1,756-question benchmark, shows multimodal LLMs struggle with fine-grained image-difference identification, scoring as low as 8.7% on low-level changes.

VDiff-Bench is a multiple-choice benchmark of 1,756 four-way questions over image pairs covering 10 change categories including position, motion, color, texture, OCR/text and illumination, with curated hard negatives. Evaluation of 11 state-of-the-art open- and closed-source MLLMs shows fine-grained visual comparison remains brittle: 7-8B-scale open-source models score 52.5-70.6% on semantic changes but only 8.7-33.3% on low-level changes like noise and texture. Notably, Grok 4.3 shows a sharp performance drop on noise and texture differences, falling behind large open-source models like Kimi K2.5 and K3.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

Molecular Déjà Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

Audit of 22 frontier models finds widespread verbatim retrieval of published molecular property values, with higher reasoning increasing recall of memorized numbers.

An arXiv audit tests 22 frontier LLMs across 12 molecular regression benchmarks for verbatim retrieval of published values. More than 50% of the LLMs show verbatim retrieval on five datasets, and identical experiments are flagged 89% more often at a high reasoning level than at the lowest one. Suppressing retrieval moves model prediction errors closer together in relative terms, suggesting predictive capability is not determined solely by memorized values.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:32 UTC · 12d agoAI research1

CARDEA: Auditable Reasoning Grounded in Spatial Evidence for End-to-End Coronary Angiography Interpretation

CARDEA, a vision-language model trained only on public data, matches cardiologists on coronary angiography complexity assessment while exposing auditable bounding-box evidence.

CARDEA is a unified large vision-language model serving as the inference core of an end-to-end coronary angiography pipeline from multi-view videos to study-level diagnosis. It was trained on public datasets through visual alignment, self-distilled Chain-of-Box cold start, and reinforcement learning with verifiable rewards encouraging bounding-box reasoning. It reached 0.91 accuracy on dominance classification under domain shift and 0.90 on complexity assessment, comparable to two interventional cardiologists. RLVR raised zero-shot report generation vessel-severity macro-F1 from 0.513 to 0.686, while supervised imitation alone did not.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

CVE-2026-67211: Apache OpenNLP: OOM DoS via Unbounded Array Allocation in SymSpellModelSerializer

Apache OpenNLP CVE-2026-67211: unbounded map pre-sizing in SymSpellModelSerializer enables out-of-memory denial of service in opennlp-symspell 3.0.0-M4/M5.

CVE-2026-67211 affects the opennlp-symspell artifact in versions 3.0.0-M4 and 3.0.0-M5 only. The SymSpellModelSerializer.create() method performs unbounded map pre-sizing, allowing crafted input to trigger out-of-memory denial of service. Releases 1.x and 2.x are unaffected because the spellcheck extension was introduced in 3.0.0-M4; the issue is resolved in 3.0.0-M6.

oss-security·Sep 11, 19:22 UTC · 4d agoVulnerability 2 sourcesCVE-2026-67211

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.

Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.

MarkTechPost·Sep 13, 17:02 UTC · 3d agoAI research1

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

Clinician-calibrated K-Bench evaluates 125 LLM configurations on 200 high-risk mental health vignettes, exposing wide variation in suicide and violence risk handling.

K-Bench is a clinician-calibrated, protected benchmark evaluating 125 model configurations from 33 base models across 14 providers on 200 multi-turn vignettes covering suicide, self-harm, domestic violence, substance misuse and no-risk presentations. A frozen GPT-4o judge achieved 94.2% exact agreement with clinician consensus across 6,751 eligible comparisons from 151 clinician-rated transcripts. Leading models combined supportive conversation with combined-risk scores above 95, while risk exploration varied substantially among weaker configurations; therapeutic prompting helped weaker models and elevated reasoning produced no average improvement. A continuously updated public leaderboard is hosted at k-bench.ai with protected test materials.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 16:49 UTC · 2d agoAI safety & security

Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation

Referee-Based Quality Estimation flags unreliable polyp segmentations at inference without ground truth, reaching ROC-AUC 0.960 with SegFormer-B0 referees.

RBQE measures agreement between a primary segmentation model and an independently trained referee on a 1,223-image external benchmark drawn from four public datasets. A cross-architecture SegFormer-B0 referee achieves the strongest signal (ROC-AUC 0.960), beating a Test-Time Augmentation baseline by 0.055 ROC-AUC under an identical protocol. Excluding trivially separable empty-mask cases, ROC-AUC falls to 0.876 (SegFormer-B0) and 0.783 (same-architecture control), but RBQE's margin over baselines widens. Progressive rejection of low-agreement predictions increases mean Dice of retained outputs, supporting selective prediction at the cost of one extra forward pass.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:32 UTC · 7d agoAI research

Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

A self-distillation safety framework tunes narrow-boundary refusals in Qwen3-8B, raising target-domain refusal to 84.75% while cutting over-refusal from 15.20% to 5.20%.

The paper formulates narrow-boundary safety, where deployments need refusals within specific topics rather than whole subjects, and proposes an offline self-generated framework with controlled topic generation, escalating retries, and harmful-benign boundary pairs. On political persuasion with Qwen3-8B, the method raised target-domain refusal from 9.47% to 84.75% and cut the mean unsafe-response rate across three broader benchmarks from 26.26% to 0.14%. Verified target-model responses reduced over-refusal from 15.20% to 5.20%, and boundary-pair data cut comply-side over-refusal on held-out pairs from 32.94% to 4.16%. Results show data composition controls the safety-usability trade-off and alignment should be evaluated on both sides of the refusal boundary.

Hugging Face daily papers·Sep 2, 20:00 UTC · 13d agoAI safety & security1

ActReview: Rebuttal-Guided Training Data and Rubric Rewards for Actionable Peer Review Generation

ActReview post-trains Qwen3-8B-Base on 40K rebuttal-derived instances with rubric rewards to generate actionable, grounded peer-review feedback, plus a 1,000-instance benchmark.

The framework builds ActReview-40K from real OpenReview review-rebuttal threads, aligning reviewer weaknesses with author responses and grounding feedback in localized paper evidence. Qwen3-8B-Base is post-trained with multi-task supervised fine-tuning followed by GRPO using candidate-aware, weakness-specific rubric rewards. Experiments show improved actionability and grounding over prior specialized review-generation models, supported by ActReview-Bench, a human-curated 1,000-instance evaluation set. Human evaluation confirms better revision usefulness while noting a remaining gap in technical accuracy.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:30 UTC · 8d agoAI research

Miles v0.1: Production-Level Post-Training

Radix Ark open-sources Miles v0.1, a full-stack RL post-training framework demonstrated with asynchronous agentic RL on GLM-5.2 744B-A40B across 64 GB300 GPUs.

Miles v0.1 is a full-stack, open-source system for frontier-scale reinforcement-learning post-training, built on slime with rollout engines on SGLang and trainers supporting NVIDIA Megatron-LM and PyTorch FSDP backends plus three weight-synchronization transports. It supports full-parameter RL, LoRA RL, on-policy distillation, supervised fine-tuning, true-on-policy rollout-training alignment, and extends to diffusion models. The end-to-end case study ran fully asynchronous agentic RL on GLM-5.2 744B-A40B for terminal-use coding tasks on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. The code is open-sourced on GitHub.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI tools & infra

Unbound: 1.26.1 addresses multiple CVE items

Unbound 1.26.1 fixes critical heap buffer overflow CVE-2026-81642 in DNSKEY digesting that may allow remote code execution.

NLnet Labs released Unbound 1.26.1 on September 16 as a security release fixing several vulnerabilities, most notably CVE-2026-81642, rated CRITICAL, a heap buffer overflow during DNSKEY digesting that can possibly lead to remote code execution. Unbound is a widely deployed DNS recursive resolver and validating resolver. Administrators should upgrade promptly.

oss-security·15:05 UTC · 3h agoVulnerabilityCVE-2026-81642

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

Study shows visually grounded token embeddings in a small masked LM persist through training and improve object-property knowledge, but escape standard BabyLM benchmarks.

The paper implements ostensive definition for a small DeBERTa masked language model trained on 10M words, seeding visually grounded tokens with embeddings derived from labeled image regions before training. Visual initialization leaves a persistent, seed-replicated advantage on object-property knowledge (COMPS) and a corpus-tailored Visual-Property Swap benchmark covering color, material, size, and shape, but has no effect on most BabyLM grammar benchmarks. Synthetic grounding of previously unseeded words causally transfers the advantage to exactly those words.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:43 UTC · 6d agoAI research

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

Training-free reasoning-plus-retrieval framework lifts multilingual multimodal entity linking accuracy by 6.9% overall and up to 23.3% on rare entities.

The paper broadens rarity measurement in multimodal entity linking using knowledge-graph structural metrics beyond popularity-based pageview metrics, identifying many rare entities that popularity metrics miss. Across these rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, exposing distinct failure modes under different rarity definitions. The proposed training-free framework iteratively searches and reasons over Wikipedia with a reasoning-capable vision-language model; experiments show reasoning and retrieval are complementary. On the MERLIN benchmark covering Hindi, Indonesian, Japanese, Tamil, and Vietnamese, the system improves overall accuracy by 6.9% and up to 23.3% on rare entities, with MERLIN-Rare test slices released.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

ZDI-26-587: Ashlar-Vellum Cobalt VS File Parsing Heap-based Buffer Overflow Remote Code Execution Vulnerability

ZDI advisory ZDI-26-587 details a heap-based buffer overflow RCE (CVE-2026-19781, CVSS 7.8) in Ashlar-Vellum Cobalt VS file parsing, requiring user interaction.

The Zero Day Initiative released advisory ZDI-26-587 covering a heap-based buffer overflow in Ashlar-Vellum Cobalt's VS file parsing. A remote attacker can execute arbitrary code when the target opens a malicious file or visits a crafted page. ZDI rated the vulnerability CVSS 7.8 and assigned CVE-2026-19781.

ZDI Published Advisories·Aug 24, 05:00 UTC · 23d agoAdvisoryCVE-2026-19781

GDCM <= 3.2.7: six memory-safety and denial-of-service vulnerabilities, no CVE

Six memory-safety and denial-of-service flaws disclosed in the GDCM DICOM parsing library, affecting versions through 3.2.7.

Researcher Abhinav Agarwal disclosed six vulnerabilities in GDCM (Grassroots DICOM), an open-source C++ library for parsing and processing DICOM files. All six were confirmed against GDCM 3.2.6 using AddressSanitizer and UndefinedBehaviorSanitizer, and source review found the vulnerable patterns through version 3.2.7 and the upstream master snapshot. Potential impacts include heap corruption, process-memory disclosure, stack exhaustion, and process termination in applications parsing untrusted DICOM files. No CVE identifiers have been assigned at the time of disclosure.

oss-security·Sep 10, 03:35 UTC · 6d agoVulnerability

Adversarial Testing of Automated Program Repair Agents for Security Vulnerabilities

SWEADV benchmark shows adversarial issue descriptions make LLM program-repair agents write insecure fixes in 51.7% of cases, evading most detection tools.

Researchers built SWEADV, a benchmark of 750 adversarial issue descriptions derived from 150 SWE-bench Verified repair tasks, covering command execution, deserialization, path traversal, denial of service, and weak hashing attack types. Tested on mini_swe agents backed by GPT-5-Mini, MiniMax-M2.5, and DeepSeek-R, adversarial descriptions induced malicious behavior with successful repair in 51.7% of cases. Detection was weak: LLM-as-judge pre-repair screening reached only 62.3% accuracy, and post-repair detection via static analysis and LLM-as-judge achieved just 39.4% and 55.4%.

arXiv cs.CR·Sep 14, 17:53 UTC · 2d agoAI safety & security1

LongAgent: History-Guided Agentic Search for Longitudinal Outcome Prediction

LongAgent autonomously searches variable sets and temporal windows to predict longitudinal medical outcomes, beating the strongest non-agent baseline on synthetic data.

The paper proposes LongAgent, an agent-based method that searches over combinations of variable sets, temporal windows and aggregation functions for outcome prediction on heterogeneous medical longitudinal data. It uses a history memory of previous searches and numerical evidence to guide exploration. On synthetic data it achieves mean RMSE 1.7376, improving over the best non-agent baseline by 0.0151 (95% CI [0.0045, 0.0260]; p=0.0273), and performs comparably to the best baseline on a real clinical dataset.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 16:51 UTC · 2d agoAI research

Training a coding model to paint watercolours with TRL and OpenEnv

Hugging Face tutorial trains a coding model with TRL and OpenEnv to paint watercolours through generated code.

A Hugging Face blog walkthrough uses the TRL reinforcement learning library and the OpenEnv environment framework to train a coding model. The target task is generating code that produces watercolour-style drawings, serving as a hands-on reinforcement learning training example. No article body was available in the feed, so specifics are limited to the title.

Hugging Face Blog·Sep 3, 00:00 UTC · 13d agoAI tools & infra1

Competence-Gated Pooling of Language Models and Priors for Event Forecasting

Paper proposes a competence gate pooling language model forecasts with external priors, improving Brier score from 0.0771 to 0.0732 across 2,357 binary questions.

The paper defines a language model's relative competence as its marginal value beyond an available external forecast, and derives conditions under Brier loss where model disagreement improves that forecast. A competence gate estimates domain-level source weights from resolved outcomes, shrinks uncertain estimates toward a global weight, and recalibrates the pooled forecast. Across 2,357 resolved binary questions and five language models, it improves the external baseline from 0.0771 to 0.0732 Brier and beats global forecast combinations, though it defers to the market on ForecastBench. Across four Qwen models, verbal confidence failed to identify when the model outperformed the external forecast, while outcome-estimated competence supported better abstention.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research