TeleAntiFraud 2.0: A Refreshable, Profile-Grounded, and Audio-Based Benchmark for Telecom Fraud Detection
TeleAntiFraud 2.0 introduces a monthly refreshed audio benchmark with 900 Chinese calls for realistic telecom fraud detection evaluation.
TeleAntiFraud 2.0 is an audio-based telecom fraud detection benchmark built with a Mixed-Tree Anti-Fraud Generation Pipeline and a monthly frozen evaluation protocol. Each frozen set contains 900 Chinese calls (600 fraud, 300 near-domain non-fraud). Classifiers achieving perfect Macro-F1 against ordinary negatives drop to 0.65-0.68 with near-domain negatives, and ASR+LLM evaluations reveal class-prior shortcuts and prediction collapse.
- Monthly frozen benchmark sets with 900 audio calls each
- Near-domain negatives cut classifier Macro-F1 from 1.0 to 0.65-0.68
- Reveals shortcut learning and prediction collapse in fraud detectors
- Dataset, code, and evaluation scripts publicly released
- Products
- TeleAntiFraud 2.0
- Countries
- China
From the original article
1 min read at huggingface.co
Telecom fraud scripts evolve rapidly and are often designed to resemble routine service conversations, creating two key requirements for audio-based telecom-fraud evaluation. First, benchmarks must incorporate newly observed scam patterns without overwriting previously established test sets. Second, they must distinguish fraud from lawful, near-domain calls rather than relying on topic-separated negative examples.
By Huiyuan Liu, Zhiming Ma, Yanxing Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.