A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition
Token-level wildcard paths in CTC cut ASR word error by tolerating localized transcript ambiguity.
The paper argues that verbatim ASR references contain localized pronunciation, spelling, and lexical differences that the acoustics do not uniquely determine. It moves Omni-temporal Classification wildcard arcs from word level to token level so unsupported tokens can be bypassed while the rest of the word stays supervised, and combines both arc types. Across 19 languages, three corpora, and 25 tasks, token-level OTC improves over CTC. A predictive-entropy schedule plus the hybrid graph gives the lowest mean word error rate on every corpus and a 9.45% average relative WER reduction versus CTC.
- Token-level wildcards bypass unsupported tokens without discarding the word.
- Token-level OTC improved over CTC on all 25 tasks.
- Hybrid graph plus entropy schedule cut relative WER 9.45% versus CTC.
- Evaluated across 19 languages and three corpora.
Full article197 words · extracted from arxiv.org · click to collapse
Automatic speech recognition is typically trained assuming that the reference transcript is the only valid labeling of an utterance, yet even nominally verbatim transcripts contain localized differences in pronunciation, spelling, or lexical realization that the acoustics do not uniquely determine. Omni-temporal Classification (OTC) tolerates such noise by adding wildcard paths to the connectionist temporal classification (CTC) alignment graph, but its word-level arcs are too coarse, since bypassing one unsupported token discards supervision for the whole word. We move wildcard arcs to token granularity so unsupported tokens can be bypassed while the rest of the word stays supervised, and we combine token- and word-level arcs as complementary escape paths. Across 19 languages and three corpora, token-level OTC improves over CTC on all 25 tasks. We also replace epoch-indexed relaxation of the wildcard weights with a predictive-entropy-indexed schedule, which performs comparably while reducing dependence on training length. Combining this schedule with the hybrid graph gives the lowest mean word error rate (WER) on every corpus and a 9.45% average relative WER reduction over CTC. Independent validator transcriptions show that token-level models place significantly more wildcard-bypass probability than CTC on disputed characters, indicating that token-level tolerance targets localized transcript ambiguity.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.30160