Label-wording augmentation improves typed-decision model accuracy on unfamiliar words
Original title: Label-Wording Augmentation in a Typed-Decision Encoder: Gains on Unfamiliar Words, Not on Abbreviations or Codes
The augmented release of Ru-Jev outperforms on ordinary words but shows no gain on abbreviations or opaque codes across Russian-language tasks.
Researchers compared two releases of the 307M-parameter Ru-Jev multilingual cross-encoder trained on thirteen Russian-language tasks. The later release adds label-wording augmentation. Holding questions fixed and varying eighteen routing labels, they evaluated ten vocabularies on ordinary, jargon, slang, abbreviation and code wordings.
- Augmented release more accurate by up to 0.24 on ordinary or borrowed words
- Eleven of fourteen gains survive Holm correction
- No usable gain on abbreviations or codes
- Accuracy at or below chance on codes once training labels removed
- Four evaluation hazards reported from release comparison
From the original article
1 min read at arxiv.orgTyped-decision models score options that the caller supplies at request time, so their accuracy depends on label wording that the model's developer does not control. We compare two releases of a 307M-parameter multilingual cross-encoder, Ru-Jev, trained on thirteen Russian-language tasks; the later release adds an augmentation that presents labels in several surface forms.
By Vladimir Beskorovainyi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.