Inferring L1 Backgrounds from L2 English with LLM Token Probabilities
Llama 3.1 token probabilities identify writers' first languages from L2 English, with 70B reaching state of the art.
The paper infers a writer's first language from L2 English by scoring each text as a continuation of a prompt that names a candidate L1, using token probabilities from Llama 3.1 8B and 70B. Evaluation covers TOEFL9, Write and Improve 2024, and BULATS-L1. Token-probability scoring substantially beats direct classification, especially with the 8B model, while the 70B model reaches state-of-the-art performance on all three datasets.
- Predicts L1 from the highest token-probability continuation score.
- Tests Llama 3.1 8B and 70B on three English-learner datasets.
- The 8B model gains most versus direct classification prompting.
- The 70B model reports state-of-the-art on all three datasets.
- Vendors
- Meta
- AI models
- Llama 3.1 8BLlama 3.1 70B
From the original article
1 min read at arxiv.orgNative language identification (NLI) infers a non-native speaker's first language (L1) from their second language (L2) production. We investigate a token probability approach that scores each text sample as a continuation of a prompt prefix specifying a candidate L1, then predicts the candidate with the highest score. We evaluate our approach using Llama 3.
By Ahmet Yavuz Uluslu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.