On the Regularization Landscape for the Linear Recommendation Models
Study shows leading linear recommendation models reduce to nuclear-norm or Frobenius-norm regularization, with two new closed-form low-rank solutions proposed.
The paper unifies top-performing linear recommendation algorithms under a single regularization framework, showing they effectively apply either nuclear-norm or Frobenius-norm regularizers. Nuclear-norm solutions have a rigid structure, are low-rank, and have closed form, while Frobenius-norm solutions are more expressive but full-rank or require hard-to-tune procedures such as ADMM. The authors derive two new low-rank, closed-form solutions that combine the advantages of both regularization families.
- Leading linear recommenders share nuclear- or Frobenius-norm regularization
- Nuclear-norm models are rigid but yield closed-form low-rank solutions
- Frobenius-norm models are expressive but full-rank or ADMM-dependent
- Two new low-rank closed-form solutions combine both advantages
Full article164 words · extracted from arxiv.org · click to collapse
Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.11876