Study maps how WildChat users ask LLMs for legal guidance
Original title: Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations
In an enriched sample, 71.1% of legal-content chats sought personalized or practical guidance; annotations are public.
Researchers built a multi-axis taxonomy of legal queries and applied it to 2,122 naturally occurring WildChat-1M conversations, plus a human-reviewed gold set of 200. In the enriched sample, 71.1% of conversations with legal content sought personalized or practical guidance, and 25.3% of guidance-positive chats spanned multiple legal topics. Rights-and-options advice was the most common primary request type. They compare ModernBERT-base and ModernBERT-large with a TF-IDF cascade; annotations, checkpoints, and evaluation code are public.
- Taxonomy covers whether guidance is sought, how it is asked, and the legal domain.
- Gold evaluation set contains 200 human-reviewed conversations.
- 25.3% of guidance-positive conversations cover more than one legal topic.
- ModernBERT-base and ModernBERT-large are compared with a TF-IDF cascade.
From the original article
1 min read at huggingface.co
People ask conversational AI models about their legal rights, obligations, procedures, and personal situations that may have legal consequences. To study this behavior, we first need to distinguish requests for legal guidance from other legally themed discussions and unrelated conversations.
By Amirreza Mohseni, Gerasimos Spanakis · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.