Partition-Matched Evaluation of Community Features under Distribution Shift in Android Malware Function-Call Graphs
Study finds Android malware graph communities add little shift-stable signal beyond random partitions.
An arXiv cs.CR paper tests whether mesoscopic community structure in Android function-call graphs stays useful when malware type or family shifts. Using 15,000 MalNet-Tiny, Common, and Distinct graphs, six pre-specified Leiden descriptors, and five optimizer seeds, communities lift Tiny macro F1 from 78.7% to 81.3% but cause a 29.4-point source-only drop on Common. Size-matched random partitions degrade by a mean 27.9 points, and the 95% bootstrap interval for random minus community degradation is [-4.2, 1.2]. With metadata the difference is -0.2 points, so the authors conclude the signature adds IID signal but no repeatable shift-stability advantage.
- Researchers tested Leiden community features on 15,000 MalNet function-call graphs.
- Communities raised Tiny macro F1 from 78.7% to 81.3%.
- Source-only Common evaluation still fell by 29.4 points.
- Size-matched random partitions degraded by about 27.9 points on average.
- Bootstrap intervals show no repeatable shift-stability advantage.
Full article145 words · extracted from arxiv.org · click to collapse
Graph-based Android malware classifiers can lose accuracy under malware-type or family shifts. We test whether mesoscopic organization in function-call graphs provides shift-stable information beyond local degree profiles (LDP), global statistics, lightweight metadata, and size-matched random partitions. Using 15,000 MalNet-Tiny, Common, and Distinct graphs, six Leiden descriptors specified before evaluation, and five optimizer seeds, communities raise Tiny macro F1 from 78.7% to 81.3% but yield 29.4-point source-only Common degradation. One size-matched random partition yields 27.8-point degradation. Across five random partitions, mean degradation is 27.9 points; the 95% two-level bootstrap interval for random minus community degradation is [-4.2, 1.2] points. With metadata, the corresponding difference is -0.2 points with interval [-1.1, 0.6]. Removing modularity raises structure-only Common macro F1 from 51.9% to 53.6%. The tested signature adds IID signal but shows no repeatable shift-stability advantage, demonstrating why mesoscopic graph claims need partition-matched controls and repeated null draws.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.25256