Learning Sparse Decision Trees via Transformer Variational Auto-Encoders
TREVIS uses a Tree Transformer VAE latent space to learn decision trees matching near-optimal predictive performance while improving structural sparsity.
TREVIS learns decision trees optimized for complex objectives by exploring the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). Mapping trees to continuous latent representations replaces the discrete search space with a continuous one, enabling gradient-based optimization through a differentiable surrogate model. Experiments show TREVIS matches the predictive performance of near-optimal algorithms while improving structural sparsity, targeting high-stakes contexts needing transparent decision logic.
- Tree Transformer VAE maps decision trees to continuous latent space
- Differentiable surrogate enables gradient-based tree optimization
- Matches near-optimal predictive performance with better structural sparsity
From the original article
1 min read at huggingface.co
Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity.
By Giacomo Fidone, Alessio Cascione, Riccardo Guidotti · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.