CTP-FL: Common-Trajectory Gradient Prediction for Federated Learning
CTP-FL estimates a shared global gradient path at the same per-round cost as FedAvg-M.
The paper proposes Common-Trajectory Predictive Federated Learning (CTP-FL). Each round, clients evaluate K stochastic gradients along one shared path built from the current global model and the previous aggregated direction, then upload their average for a single server update. Compute and communication match full-participation FedAvg-M. For smooth non-convex objectives under full participation, the authors prove an average-stationarity bound of O(sqrt(LΔσ²/(NKR))+LΔ/R), with path length trading forward-looking information against displacement bias.
- Clients share query points so the aggregate estimates the global gradient
- Per-round cost matches full-participation FedAvg-M with one model-sized vector each way
- Stationarity bound needs no bounded client-gradient dissimilarity assumption
- Longer prediction paths add lookahead but increase displacement bias
From the original article
1 min read at arxiv.orgCommunication-efficient federated optimization commonly spends several gradient evaluations between server updates. Existing local-update methods use this computation to advance an independent model on each client. Under heterogeneous data, however, these models evaluate gradients at different locations, making the aggregated update difficult to interpret as a gradient of the global objective.
By Junkang Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.