Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs
Researchers propose G2MLP, a curvature-guided distillation method that transfers GNN geometry into graph-free MLPs.
GNN-to-MLP distillation keeps a message-passing teacher's accuracy while deploying a graph-free MLP at inference. Existing methods transfer node predictions but miss where the student should preserve graph-induced geometry, causing spectral underfit on sparse graphs and spectral overfit on dense graphs. The authors introduce Graph Geometry-aware MLP (G2MLP), which uses Ollivier-Ricci curvature to allocate supervision between prediction-level and representation-level alignment. Across node-classification benchmarks, G2MLP beats graph-free baselines, reduces the teacher-student rank gap, and also transfers to Graph Transformer teachers and link prediction.
- G2MLP stays a standard MLP and needs no graph at inference.
- Curvature allocates supervision between prediction and representation alignment.
- It reduces spectral underfit on sparse graphs and overfit on dense graphs.
- Gains hold for node classification, link prediction, and Graph Transformer teachers.
From the original article
1 min read at arxiv.orgGNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry.
By Zhewei Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.