E-means claims much faster k-means at equal or better loss
Authors report beating Lloyd loss in 1.2 to 3 epochs and 10–45× GPU speedups versus Flash-KMeans.
Stories per day · 30 days
Relatedconvex-optimization2robustness2variational-inequalities2adam1adamw1adversarial-training1algorithm-analysis1android-malware1anomaly-detection1autoformulation1benchmark1clustering1datacenter1decision-making1differential-privacy1distribution-shift1dp-sgd1dual-averaging1
46 stories, newest first · one card per story
Stories tagged #optimization
DW-FedADMM improves federated learning for Android malware detection
DW-FedADMM uses dynamic client-side weighting in federated optimization, achieving up to 97.64% accuracy on Drebin dataset with partial participation.
Adversarial training with large step sizes shows persistent implicit bias
Large constant step sizes in adversarial training preserve implicit bias gap with standard training and exhibit two-stage sharpness dynamics.
GraphMuon optimizes graph neural networks with matrix-aware updates
GraphMuon achieves 1.32x-2.42x speedups over AdamW on large OGB benchmarks.
FPTR Refinement Ensures Feasible Allocations for Wireless Schedulers
Feasibility-Preserving Transactional Refinement maintains validated solutions while refining beam and resource allocations before deadlines.
A²ODA method achieves accelerated convergence in p-norm geometry
Adaptive accelerated optimistic dual averaging provides line-search-free preconditioning for convex optimization.
Adam's implicit bias depends on learning rate schedule
Adam maximizes weighted l2 margin for constant schedules on homogeneous models.
Improved complexity bounds for second-order convex finite-sum optimization
The paper matches a new lower bound of Omega(n + n^{5/7} eps^{-2/7}) using a Taylor-corrected lazy Newton method with Monteiro-Svaiter acceleration.
Paper derives optimal privacy-budget schedules for private SGD
PB-AMD uses adjoint gradients and entropic updates so noise schedules stay positive and spend the certified zCDP budget.
AIR schedule targets effective learning rate, not nominal rate
On 128 billion-token MoE pretraining, AIR lowers terminal training loss by about 0.01 versus cosine at matched cumulative learning rate.
Paper recasts evolution-strategy updates as center-of-mass shifts
The mean-subtracted update matches a reward-weighted displacement for any finite population when rewards are nonnegative.
SWAPE jointly learns parameters and sparse sample weights
Alternating majorize-minimize updates cover convex and difference-of-convex losses while down-weighting anomalous points.
Tutorial presents KKT-HardNet for constrained hybrid modeling
The framework enforces linear and nonlinear constraints via projection and releases code plus engineering benchmarks.
Paper compares predict-then-optimize and generate-then-optimize decisions
Regret bounds show smoothness can offset generate-then-optimize's extra dimensions, while nonnegative-weight methods face a smoothness limit.
Fixed-point bundle method computes Nash equilibria on product simplices
The path-following algorithm reached 10^{-5} precision on all 5600 random games, from 2-player 128-action to 128-player 2-action.
ASTOR turns natural-language soft constraints into optimization code
On the new SoftOR benchmark, the authors report the best aggregate accuracy among compared LLM methods.
How to Use Meta’s Rebalancer for Task Placement & Optimization
A tutorial shows how to use Meta's Rebalancer solver to optimize datacenter task placement.
REASON proposes two-loop residual analysis for learning
The two-loop model beats the fast loop at about half the medium loop's computation, including under unseen corruptions.
Partially Informed Stochastic Optimization via Zeroth-Order Residual Tracking
PISO tracks unknown gradient residuals with zeroth-order probes for partially observed stochastic optimization.
Decentralized SGD under Heavy-Tailed Noise: Optimal Convergence Rates and the Role of Gradient Clipping
Clipped decentralized SGD reaches order-optimal rates under heavy-tailed noise with linear agent speed-up.
Meta AI Open-Sources Rebalancer: A C++ Assignment Solver That Runs About 40 Million Placement Problems a Day
Meta open-sourced Rebalancer, a C++ assignment solver handling about 40 million placement problems a day.
Differential Privacy of Gradient Descent on Perturbed Objectives
Proves explicit privacy bounds for finite gradient-descent iterates on perturbed objectives, removing ambient-dimension factors for generalized linear models.
Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning
ZFO fine-tunes language models by pairing a first-order direction with cheap zeroth-order step search.
SoftServe: A Scalable Quasi-Newton Method for Deep Learning
SoftServe scales positive-definite quasi-Newton updates to large networks and beats Adam, Muon, and SOAP on ill-conditioned tasks.