Detecting GPT-Assisted Writing Using Interpretable Stylometric Features
Random Forest using stylometric features detected ChatGPT-assisted student writing with ROC-AUC 0.87.
The paper tests whether interpretable stylometric features taken only from submitted text can separate independently written student work from ChatGPT-assisted writing. Ninety participants produced both conditions, and eight classifiers were evaluated with same-participant data kept together in validation. On the held-out test set, Random Forest reached an ROC-AUC of 0.87 and an F1-score of 0.84, with a 22.2% false positive rate and an 11.1% false negative rate. SHAP analysis attributed predictions mainly to lexical and grammatical features.
- Study used writing from 90 participants, both independent and ChatGPT-assisted.
- Random Forest reached ROC-AUC 0.87 and F1 0.84 on a held-out test set.
- False positive rate was 22.2% and false negative rate was 11.1%.
- SHAP analysis points to lexical and grammatical features as main drivers.
- Same-participant data stayed together during validation to limit leakage.
From the original article
1 min read at arxiv.orgDistinguishing GPT-assisted from independently authored student writing has become a critical challenge in academia. This paper evaluates the discriminative capability of interpretable stylometric features extracted solely from submitted text.
By Rajesh Kumar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.