Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences
SCPTM graph-augmented variational topic model shows syntax aids topic diversity and descriptor quality but gains stem mainly from the variational encoder.
The Structural Contextual Probabilistic Topic Model represents corpora as heterogeneous document-word graphs with lexical and syntactic edges processed by a Graph Attention Network inside a VAE for mixed-membership topic distributions. Across four corpora, neural gains in document-topic alignment are attributable to the variational encoder rather than syntax, while graph-augmented variants improve topic diversity everywhere. Dependency paths add value on argumentative deliberative texts but are redundant in technical and institutional registers.
- SCPTM combines dependency syntax, GATs, and a VAE for topic modeling
- Document-topic alignment gains trace to variational encoder, not syntax
- Syntax improves topic diversity across all evaluated corpora
- Dependency paths help argumentative texts, add noise in technical registers
- Tags
- bertopiccomputational-social-sciencegraph-attention-networkldasyntaxtopic-modelingvariational-autoencoder
From the original article
1 min read at arxiv.orgTopic modeling is widely used in computational social sciences to identify latent themes in large text corpora. Traditional approaches rely on Bag-of-Words representations and generative models such as LDA, while recent methods like BERTopic operate on dense document embeddings.
By Alessandro Meneghini · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.