CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles
CineSubBench tests LLMs on narrative, cultural, and safety judgments using subtitles from 1,012 films.
CineSubBench evaluates long-context film understanding from multilingual subtitles rather than scripts or explicit scene structure. It includes 1,012 films with complete coverage in six languages, producing 6,072 tracks and 8.13 million timestamped subtitle entries. Seven tasks span narrative reconstruction, genre prediction, age suitability, ratings under ten national classification systems, and subtitle-grounded language safety. Across nine LLMs, plot premises were recovered more reliably than event-complete synopses, cross-lingual consistency varied, and strong profanity was easier to ground than mild obscenity.
- Covers 1,012 films, six languages, 6,072 tracks, and 8.13 million subtitle entries.
- Seven tasks include plot reconstruction, genre, age ratings, and language safety.
- Nine LLMs recovered premises more reliably than complete synopses.
- Strong profanity was easier to ground than mild obscenity.
From the original article
1 min read at huggingface.co
Large language models are increasingly evaluated in specialized domains such as law, medicine, software engineering, and cybersecurity, yet film remains comparatively underexplored despite requiring long-form narrative integration, multilingual interpretation, and culturally situated audience judgments.
By Mir Tafseer Nayeem, Susmoy Chakraborty, Davood Rafiei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.