Split MLLM features still leak age, gender, and race
Original title: Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs
With 20 labeled samples per class, Safe-LLaVA split features recover 75-85% of unprotected biometric leakage.
Edge-cloud split inference keeps raw images on the device but sends intermediate multimodal LLM features to a server. The authors say an honest-but-curious server can infer age, gender, and race from those features at any split depth when pipelines are unprotected. Their label-budgeted recovery curve finds stage-wise protections, including Safe-LLaVA response filtering, still leave most leakage recoverable from transmitted features with small labeled sets.
Unconfirmed: Leakage shares are the authors' measurements on their evaluation setup.
- Unprotected intermediate features leak all three attributes at every split depth.
- The evaluation uses two popular multimodal large language models.
- Safe-LLaVA removes biometric content from responses, not split features.
- Twenty labels per class recover 75-85% of unprotected leakage.
From the original article
1 min read at arxiv.orgEdge-cloud split inference runs the early layers of a multimodal large language model (MLLM) on a device and sends intermediate features to a server to complete the task. While keeping raw images local is often perceived as a privacy benefit, an honest-but-curious server can still infer sensitive attributes, such as age, gender, and race, from the received features.
By Junguang Yao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.