Domain-Specific Hallucination Detection in Large Language Models
A multi-signal pipeline detects LLM hallucinations, reaching F1 0.915 on HaluEval and cutting Qwen2.5-0.5B hallucination rates from 85.5% to 37.7% via DPO.
The paper presents a hallucination detection pipeline combining fine-tuned DeBERTa-v3 classification, Monte Carlo Dropout uncertainty, and temperature-scaled calibration. It achieves F1 0.915 and AUROC 0.977 on general-domain HaluEval tasks, with MC Dropout inference raising accuracy to 93.2%. Applying DPO to a Qwen2.5-0.5B generator reduces its hallucination rate from 85.5% to 37.7%, while cross-domain evaluation shows poor general-domain transfer to SciFact (F1 0.52) and PubMedBERT fine-tuning as the strongest adaptation (F1 0.63).
32
Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model
Study shows visually grounded token embeddings in a small masked LM persist through training and improve object-property knowledge, but escape standard BabyLM benchmarks.
The paper implements ostensive definition for a small DeBERTa masked language model trained on 10M words, seeding visually grounded tokens with embeddings derived from labeled image regions before training. Visual initialization leaves a persistent, seed-replicated advantage on object-property knowledge (COMPS) and a corpus-tailored Visual-Property Swap benchmark covering color, material, size, and shape, but has no effect on most BabyLM grammar benchmarks. Synthetic grounding of previously unseeded words causally transfers the advantage to exactly those words.
20