Search: “token-optimization”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Rare Not Random Using Token Efficiency for Secrets Scanning

Researcher proposes token efficiency (string length divided by BPE token count) as a better post-regex filter than entropy for secrets scanning, validated on CredData.

The post explores whether Byte-Pair Encoding tokenization can replace Shannon entropy as the primary filter for candidate secrets captured by regex in tools like Gitleaks. It defines 'token efficiency' as string length divided by token count under the cl100k_base tokenizer; secret-like strings such as GitHub tokens tokenize into many small tokens and score low, while natural text scores high. Evaluating labeled secrets from the CredData dataset shows a usable separation, with roughly 2.5 suggested as a minimum cutoff versus Gitleaks' 3.5 entropy threshold. The technique is positioned as a post-regex filtering step rather than a standalone detector.

Lobsters · security·Sep 12, 06:42 UTC · 4d agoResearch

45

When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

Elo-per-token analysis shows LLM agents' marginal gains drop below independent sampling at scale; parallel sessions beat one long session.

The paper proposes Elo-per-token analysis, using a Bradley-Terry model to measure how agent performance scales with token budget on open-ended tasks with continuous scoring. Across four agents and four benchmarks with sessions up to 100M tokens, agents initially convert tokens to Elo faster than independent sampling but eventually slow below the linear-in-log-compute reference. The authors define a scaling inflection point and show that splitting 100M tokens across parallel sessions on FrontierCS Polyomino Packing gains +264 Elo over one long session and +355 over ten short sessions. Human contestants on shared AtCoder Heuristic Contest tasks improve superlinearly, indicating headroom over current agents.

Hugging Face daily papers·Sep 13, 20:00 UTC · 3d agoAI research3· 2 reads

48

Silver Rate Is (Almost) Optimal for Gradient Descent Acceleration

New optimization theory paper proves near-optimal lower bounds for gradient descent with predetermined stepsizes, confirming silver-schedule optimality.

The paper studies the limits of accelerating gradient descent using predetermined nonnegative stepsizes in smooth convex optimization, with the key constant p_sil = log2(1 + sqrt(2)). It proves a non-anytime lower bound of Omega(n^(-p_sil - O(sqrt(log log n / log n)))) on the error achievable by any such stepsize schedule. In the anytime setting, it shows every infinite nonnegative schedule must incur error Omega(n^(-2*p_sil/(1+p_sil) - O(sqrt(log log n / log n)))) at infinitely many horizons. Combined with the silver-schedule upper bound of Altschuler and Parrilo (2025) and the anytime upper bound of Zhang et al. (2025), these results determine the optimal polynomial convergence exponents in both settings.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:59 UTC · 8d agoAI research

28

Don't Send What You Don't Need: Question-Guided Token Pruning as a Privacy Defense for Vision-Language Models

QPriv-VL prunes privacy-sensitive visual tokens in federated/split VQA, cutting membership-inference success on VQA-RAD from 0.99 to 0.76-0.79 using ~40% of tokens.

The paper proposes QPriv-VL, a question-guided token-pruning framework for federated, split, and U-shaped split learning that suppresses privacy-sensitive visual patches before transmission. Its Dynamic Threshold Predictor combines cross-modal question relevance with frozen DINOv2-derived sensitivity to compute a per-sample pruning ratio and retention mask in one forward pass, without sensitivity labels. Evaluated on GQA, OK-VQA, VQAv2, SLAKE, VQA-RAD, and PathVQA against FSHA, FORA, iDLG, and attribute-inference membership inference attacks, it matches or beats fixed-ratio pruning. On VQA-RAD it reduces membership-inference success from 0.99 to 0.76-0.79 while preserving competitive accuracy with about 40% of the original token budget.

arXiv cs.CR·Sep 14, 14:48 UTC · 2d agoResearch

35

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

Researchers release OR-Clarify, a benchmark testing whether LLM agents ask clarifying questions before formulating optimization models from incomplete requests.

OR-Clarify evaluates pre-formulation clarification in operations research: each task gives a partial problem description, withholds structured hidden slots, and scores agents via bounded interaction with a simulated user, measuring slot recovery, stopping behavior, silent assumptions, and interaction cost. The authors also propose InterOPT, a two-stage framework that identifies formulation-critical gaps to decide when to ask or stop. In choice-based experiments InterOPT substantially outperforms all baselines in exact slot recovery and remains competitive in the open-ended setting.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research1

35

Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

Researchers release Phi-Bench, a benchmark evaluating frontier LLMs on open-ended, long-horizon engineering and optimization of the LLM infrastructure stack.

Phi-Bench evaluates LLMs on open-ended engineering of the LLM infrastructure stack, derived from optimization problems studied in frontier research and grounded in real-world code repositories. Tasks range from localized kernel-level function completion to long-horizon implementation and end-to-end system optimization. Experiments on frontier LLMs reveal current capabilities and limitations on the path toward autonomous optimization of future AI infrastructure.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research1

28

Graph Machine: Towards Better Pretraining via Edges

Researchers propose Graph Machine, an O(n)-state sparse architecture that replaces 75% of Qwen3-0.6B dense layers with only slight loss change.

The paper introduces the Graph Machine (GM), an architecture that maintains an O(n)-sized state accessed through sparse, dynamic routing via pointer-like edges updated differentiably by a referral mechanism resembling pointer chasing. The authors replaced 75% of dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrained from scratch on 15.7B tokens. Retrieving 2 of 4,096 tokens per KV head in each sparse layer degrades loss only slightly, while retrieving 4 marginally improves loss over the dense baseline.

Hugging Face daily papers·Sep 1, 20:00 UTC · 15d agoAI research

30

Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool

Researchers present SMART, an ML performance-modeling library regenerated by AI coding agents from natural-language design docs instead of code.

The paper describes SMART, a symbolic performance-modeling library whose main branch contains almost no code: the repository is a DAG of self-contained design documents, and coding sub-agents regenerate implementations from only the docs on version updates. Reliability rests on a worked-example doc style used as in-context demonstrations and a minimal operator IR with SymPy cost expressions, offering both fast analytical roll-up and fine-grained modulo-scheduling modes. Regenerated implementations reproduce hand-audited reference models, including DeepSeek-V3 serving on a TPU pod slice, to round-off precision.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:08 UTC · 12d agoAI research1

30

Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation

Researchers propose ERPO, enabling test-time reinforcement learning for code generation via probe-executed consensus rewards, rank masking, and entropy regularization.

The paper introduces probe-driven test-time reinforcement learning (TTRL) for code generation, where output-free probe inputs are constructed from problem statements and candidate programs are executed on them to compute a Probe Consensus Reward (PCR). Because PCR can be gamed through spurious consensus, the authors propose Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which turns low-PCR outcomes into conservative negative updates via rank masking and constrains policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:54 UTC · 8d agoAI research1

38

AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems

AgentGrad introduces intervention-guided prompt optimization for LLM multi-agent systems, achieving state-of-the-art results with 2.5x faster optimization.

AgentGrad is a prompt optimization framework for LLM-based multi-agent systems that addresses limitations in textual gradient extraction and aggregation. It uses sequential intervention to identify the agent whose prompt modification resolves a given failure, then applies agent-level supervision and semantic gradient clustering to build generalized gradients. Experiments report state-of-the-art performance across five MAS benchmarks and a 2.5x average reduction in wall-clock optimization time versus the next-fastest baseline.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

32

LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics

LexFlip releases 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving tokens, exposing weaknesses in embedding-based meaning preservation metrics.

LexFlip provides 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of tokens, creating dissociation items that break monotone token-overlap metric validation. The seven embedding and BERTScore metrics tested register only 0.022-0.039 of their identical-to-unrelated range on these edits, versus 0.670 for bidirectional NLI. Against FrJudge, with a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric tested.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 15:49 UTC · 12d agoAI research

22

When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

Researchers derive an exact law linking learning-rate schedules and weight decay in normalized networks, pinpointing when scale-invariant optimization destabilizes.

The paper shows that normalization makes large parts of neural networks scale-invariant, creating a hidden feedback loop where learning-rate schedules and weight decay interact through the parameter norm to control the effective optimizer step. An exact discrete-time law with a single scalar quantity separates contraction- and expansion-dominated effective learning-rate regimes, and the balance point is intrinsically unstable, so constant learning rate with weight decay produces recurrent behavior instead of a stable equilibrium. A unified homogeneous-optimizer framework explains why adaptive methods stabilize more weakly under normalization. The law is validated with high precision on MLPs, CNNs, and GPT-2 across MNIST, CIFAR, WikiText, and OpenWebText, with code released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 8d agoAI research

30

AdamX: Cosine similarity meets gradient descent

Researchers propose AdamX, a cosine-similarity-based first-order optimizer with variance rectification that matches Adam-class convergence across benchmark datasets and architectures.

The paper introduces AdamX, a first-order optimizer that uses cosine similarity as an adaptive mechanism for controlling update magnitudes, plus a variance rectification scheme for smoother optimization early in training. The method is described as scalable, model-agnostic, and straightforward to integrate into existing pipelines. Empirically, AdamX shows competitive convergence rates measured by epochs to reach performance thresholds under a fixed hyperparameter budget, with code and experiments released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:42 UTC · 6d agoAI research

22

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

BeaconKV introduces training-free KV cache compression using beacon queries, cutting long-reasoning inference memory up to 5.8x while preserving accuracy.

The paper shows recency-based KV cache compression assumptions fail in long-horizon reasoning because Thought Revisiting Tokens (TRT) re-attend to distant context such as early task-solving plans. TRT queries cluster into a small number of similarity groups, which BeaconKV exploits by maintaining compact beacon query representatives to anticipate revisited KV pairs without storing full query history. The training-free method achieves up to 5.8x memory reduction and over 4.3x throughput improvement across four open-source large reasoning models while nearly preserving full cache accuracy.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research1

33

OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design

OptiPrime co-designs HE-MPC protocols with hardware acceleration to remove network communication bottlenecks in private DNN inference, beating Cheetah by up to 5.7x.

OptiPrime is a protocol-hardware co-optimization framework for private deep neural network inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC). It introduces a novel HE protocol for convolutions that reduces the number of transmitted output ciphertexts, addressing the network bottleneck that limits gains from commercial HE accelerators. A lightweight compression system reduces weight plaintext memory traffic by 10x, while a specialized dataflow maximizes on-chip reuse of intermediate ciphertexts. Experiments show up to 5.7x speedup over the Cheetah baseline on CPUs and 4.2x with an accelerator.

arXiv cs.CR·Sep 15, 09:29 UTC · 1d agoResearch

28

Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training

Controlled mid-training experiments on Qwen3-8B-Base find each domain has a 10-40% coverage optimum and domain gaps survive alignment SFT.

Using Qwen3-8B-Base (with a 4B replication) across five semantically rule-disjoint KOR-Bench domains, the authors train 30 data allocations spanning the five-domain simplex at five seeds each. All five domains show interior optima in the moderate 10-40% coverage band, and domain gaps persist after a fixed-budget compensatory SFT pass, which raises 116/120 cells yet bridges 0/240 pairs at a 5% threshold. Zero coverage collapses mid-training-only accuracy, though a FineWeb-Edu-only control shows the collapse is partly generic drift. The results argue mid-training data composition requires principled design rather than reliance on later alignment.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:31 UTC · 8d agoAI research1

30

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Uno pairs autoregressive LLMs with lightweight diffusion weights to draw multiple tokens in parallel, delivering up to 3x lossless speedup without a draft model.

The paper introduces diffusion-augmented LLMs: autoregressive weights trained with the standard next-token objective plus lightweight diffusion weights trained via a Diffusion Distillation phase to emit multiple tokens in parallel. Psi-Spec samplers enable lossless acceleration without the separate draft model required by speculative decoding. The 8B Uno model outperforms the 26B open DiffusionGemma and proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks, with up to 3x throughput gains over the base model at all evaluated batch sizes. Code and checkpoints are released publicly.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

55

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models

Empirical study finds direct whole-file generation beats iterative diff-based editing for Flutter/Dart code models on about 1,790 held-out tasks.

Researchers trained Rainbow-Pony-100M from scratch and fine-tuned Qwen2.5-Coder-0.5B in both direct-generation and diff-based regimes, then evaluated four resulting models on roughly 1,790 Flutter/Dart tasks. Direct generation outperformed diff-based generation on compilation pass rate, bits-per-byte, character-level similarity, and blinded LLM-judge ratings. Diff-based editing is competitive only on short, localized edits in refactoring and error-handling tasks, a property the authors call task locality.

Hugging Face daily papers·Sep 4, 20:00 UTC · 12d agoAI research1

30

RTK reports token savings, but our cost benchmarks disagree

Quesma's $1,500 benchmark found RTK cuts reported token output but changes Claude Code and DeepSeek coding costs by only about 5% on Terminal-Bench 2.1.

Quesma benchmarked RTK (Rust Token Killer), a popular tool with 79k GitHub stars that filters terminal output for AI coding agents, whose README claims up to 90% output reduction. Across 1,740 Terminal-Bench 2.1 attempts running Claude Code with Fable 5.0 and OpenCode with DeepSeek V4 Pro 0813, total costs moved only -5% for Fable and +5% for DeepSeek, with pass rates dropping 1-2%. RTK's own rtk gain metric reported 349.2 million tokens saved (an 89% reduction) across 445 DeepSeek attempts, but this did not correlate with actual cost savings, and cached terminal-output reads cost as little as 1/10 to 1/30 of regular input tokens. A bug in rtk find 0.45.0 caused one agent to loop with 339 consecutive errors, costing roughly 9x the baseline attempt, though the task still passed.

Hacker News · security·Sep 11, 11:15 UTC · 5d agoAI tools & infraHN 28↑ · 10 comments1

42

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

Drift-Constrained Optimization reformulates fine-tuning as update-direction selection, letting Qwen3 models improve target tasks within a behavioral drift budget.

The paper specifies a behavioral drift budget before optimization and shows that update direction is the remaining degree of freedom, reformulating fine-tuning as a direction-selection problem. In a stringent QA-only setting where instruct models must still generate multi-step reasoning at inference, a coarse layer-selective probe reverses the failure of QA-only fine-tuning. Across Qwen3-8B and Qwen3-14B, these directions substantially improve scientific reasoning and multilingual translation, matching or outperforming dedicated translation systems over 100+ languages and giving stronger initialization for reinforcement learning.

Hugging Face daily papers·Sep 11, 20:00 UTC · 5d agoAI research

48

Unifying Conformal Language Tasks with In-Context Ensembles

Researchers propose Conformal Relevance, which builds conformal score functions via in-context example curation and ensembling to improve conciseness across seven NLP tasks.

The paper targets NLP tasks like summarization and extractive QA that reduce to retrieving content under coverage and conciseness constraints. Conformal Relevance replaces hand-engineered LLM scoring prompts with curated in-context examples and ensembles, maintaining coverage guarantees while improving conciseness with minimal manual input. The authors demonstrate the framework on seven NLP tasks and contribute theory, including a complementarity condition for when ensembling improves worst-case sentence scores and a saturation bound on ensemble gains.

Hugging Face daily papers·Sep 1, 20:00 UTC · 15d agoAI research1

42

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research1

40

Differential Privacy Meets Fixed Parameter Tractability: Algorithms and Lower Bounds

Theory paper combines differential privacy with fixed-parameter tractable encoders, improving approximation guarantees for combinatorial optimization and proving new lower bounds.

The paper studies combinatorial optimization under epsilon-differential privacy within the implicit encoder-decoder framework of Gupta et al. (SODA 2010), generalizing it to allow fixed-parameter tractable encoders. This circumvents approximation barriers inherent to polynomial-time algorithms and yields improved guarantees for fundamental combinatorial optimization problems. The authors establish the first representation-independent lower bounds: assuming a non-uniform variant of the Gap Exponential Time Hypothesis, no epsilon-DP encoder-decoder pair can achieve certain approximation guarantees with a subexponential-time decoder for sufficiently small epsilon. Representation-dependent lower bounds are also provided for larger epsilon.

arXiv cs.CR·Sep 11, 07:13 UTC · 5d agoResearch

18

27.5KB language-agnostic WebGPU syntax highlighter

A developer released gpu-lexer, a 27.5KB language-agnostic syntax highlighter that uses a tiny WebGPU model to label code tokens in the browser.

gpu-lexer splits source into words, whitespace, and symbols, then a small WebGPU model uses local and whole-file context to assign nine token classes, working on languages never seen in training. On held-out files, 12.57% of token labels differ from Shiki, though this measures agreement with Shiki rather than objective correctness. In benchmarks against Shiki 4.4.3, Prism.js, Highlight.js, Sugar High, and Starry Night, it highlighted 10 concatenated copies of three.min.js (5.56M characters) about 10x faster on an Apple M4 Pro in Chrome 152. The author frames it as an experiment, not a grammar-equivalent highlighter.

Hacker News · security·Sep 9, 01:14 UTC · 7d agoAI tools & infraHN 21↑ · 5 comments1

15

MAxBench: A Multinomial Concept Recovery Benchmark

MAxBench evaluates multinomial concept recovery methods, finding affine subspaces steer most reliably but none consistently beats prompting.

MAxBench is a geometry-agnostic evaluation framework for multinomial concept representations in language models, based on sampling from recovered concept representations. It compares 10 localization methods covering 5 geometry types across 6 concepts and 4 models. Findings show affine subspaces steer more reliably than rank-one or linear subspaces due to better non-zero offsets, manifold steering is competitive where applicable, and no method consistently outperforms prompting.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:08 UTC · 5d agoAI research

18

TasmScan: Continuation-Aware Taint Analysis for TVM Bytecode with Savelist Abstraction

TasmScan introduces source-free taint analysis for TON smart-contract bytecode, detecting 95.3% of defects with 96.8% precision and 17x speedup.

TasmScan is the first bytecode-level static analysis framework for the TON Virtual Machine, enabling cross-continuation data flow reasoning without source code by modeling savelist semantics through forward register analysis with formal over-approximation guarantees. It lifts bytecode into a typed intermediate representation (TASIR) and performs path-sensitive taint analysis. On a 208-contract benchmark with human-confirmed ground truth it detects 95.3% of defects across five classes at 96.8% precision, and resolves 294,546 dynamic continuation targets with 100% precision across 2,921 registry contracts. It achieves a 17x median speedup over symbolic-execution baselines.

arXiv cs.CR·Sep 15, 11:01 UTC · 1d agoResearch

28

NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction

An 8.9B-parameter latent-space language model using next-concept prediction matches OLMo-3-7B pretraining loss with only 51.3% of the training tokens.

NCP-ArchPreview augments next-token prediction with Next Concept Prediction over a product-quantized concept vocabulary built from hidden states, trained jointly end-to-end. The 8.9B model was trained on 5.73T tokens from the Dolma-3 dataset, the largest latent-space language model demonstration to date. It consumes 51.3% of the tokens to reach OLMo-3-7B's final pretraining loss and outperforms it by 2.45 points on the downstream macro-average, including a 5.99-point GSM8K gain. The learned latent space also enables lightweight domain adaptation via a 17M-parameter VQ module and improves speculative drafting accepted length by 4.17%.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research1

52

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

SAS trains a gated sparse attention selector end-to-end with the language modeling loss, beating distillation-based sparsifiers on reasoning, long-context, and agentic tasks.

The paper proposes Simple Attention Sparsification (SAS), which injects a selector's continuous scores into attention softmax logits so the language modeling loss directly optimizes context ranking instead of distilling dense attention distributions. Key design choices include log-form gates inside the softmax, normalized gates calibrated against the current block, and preserved continuous selector scores. A memory-efficient Triton kernel integrates SAS into FlashAttention-style computation for long-sequence training. SAS outperforms trainable sparse attention baselines across budgets, with the largest gains under tight attention budgets.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:58 UTC · 5d agoAI research 2 sources2

35

Optimizing Credential Blast Radius Through Trust Boundaries and Delegation Under Post-Quantum Authentication Costs

Academic paper models credential blast radius optimization across trust domains under post-quantum latency costs, cutting expected impact by up to 36%.

The paper formulates the joint selection of trust domains and credential-derivation structures under policy and latency constraints as an NP-hard optimization problem, showing the scalarized two-domain direct-issuance case reduces to a weighted minimum cut. In 195 of 230 exhaustive synthetic comparisons, joint optimization produced lower credential blast radius than choosing boundaries first, especially under chained delegation. A trace-derived replay using measured post-quantum authentication costs found the best design reduced expected impact by up to 36% relative to a single domain within the latency budget.

arXiv cs.CR·Sep 3, 23:42 UTC · 13d agoResearch

26

Beyond Solver Verdicts: Generative Reward Models for Autoformalization

Researchers introduce Generative Verification (GenV), a generative reward model achieving 0.961 AUROC in detecting unfaithful autoformalization that preserves solver verdicts.

The paper formalizes Verdict-Preserving-Unfaithfulness (VPU), a failure mode in neurosymbolic autoformalization where an incorrect encoding executes successfully and matches the expected solver verdict, and proves verdict-only verification is bounded to chance-level detection. The proposed Generative Verification (GenV) distills an offline Z3-equivalence oracle into a reference-free, continuous reference-equivalence score within the language model's vocabulary space. The oracle-mined verifier (GenV+HN) achieves 0.961 AUROC, generalizes zero-shot across unseen translators and formal styles, and yields an 11.3-point downstream accuracy gain in agentic test-time compute allocation. Mechanistic analysis with decision-projected logit lenses and sparse autoencoders shows the generative readout extracts precise spatial error coordinates without explicit localization training.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research1

35