Bag of Tricks or Bag of Myths? Reducing Modeling Complexity with Task Knowledge in Explainable Suicide Risk Assessment
Researchers audit 31 NLP techniques for clinician-annotated suicide risk prediction, finding only 5 of 31 comparisons yield reliable gains.
A study of 1,635 clinician-annotated social media posts ran roughly 300 controlled experiments across 7 methodological families, auditing techniques such as model scaling, synthetic data, ensembling, and threshold tuning under severe class imbalance. The proposed system reformulates risk factor prediction as entailment between posts and codebook definitions, using architecturally diverse ensembles with class-balanced training and deployment-consistent calibration. It scores 0.8203 for risk, 0.7953 for evidence, and 0.7045 macro-F1 for factors, ranking third among 53 teams.
- Only 5 of 31 audited techniques produced reliable gains under severe class imbalance
- Risk factor prediction reformulated as entailment against codebook definitions
- Deployment-consistent calibration delivered the largest factor-system improvement
- Final composite score 0.7781, ranking third among 53 teams
From the original article
1 min read at arxiv.orgAssessing suicide risk from social media text is a small-data, high-stakes setting requiring not only severity prediction but also supporting evidence and clinically relevant risk and protective factors.
By Shlok Shelat · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.