Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression
#Chebyshev Polynomial #Radial Basis Function #Tabular Regression #Machine Learning Models #Tree Ensembles #Smooth Models #Generalization Gap #Surrogate Optimization
📌 Key Takeaways
- Smooth-basis models can compete with tree ensembles in tabular regression tasks
- Transformer models achieved highest accuracy but have practical limitations for real-world deployment
- Smooth models demonstrate tighter generalization gaps compared to tree ensembles
- Researchers developed new implementations of Chebyshev and RBF models compatible with scikit-learn
- Smooth-basis models are particularly beneficial when downstream applications require gradually varying predictions
📖 Full Retelling
Researchers Luciano Gerber and Huw Lloyd published a study on February 25, 2026, comparing smooth-basis machine learning models with traditional tree ensembles for tabular regression tasks, addressing the underutilization of theoretically advantageous models in this field. The researchers developed three novel models: an anisotropic radial basis function (RBF) network with data-driven center placement and gradient-based width optimization, a ridge-regularized Chebyshev polynomial regressor, and a hybrid model combining tree structures with Chebyshev polynomials. All three implementations were released as scikit-learn-compatible packages to facilitate adoption in the machine learning community. The study evaluated these models across 55 regression datasets organized by application domain, benchmarking them against tree ensembles, a pre-trained transformer, and standard baselines, assessing both accuracy and generalization behavior. The transformer model achieved the highest accuracy across most datasets, but its practical limitations—including GPU dependence, inference latency, and dataset size constraints—make it unsuitable for the CPU-based environments common in applied science and industry. Among CPU-viable alternatives, smooth-basis models and tree ensembles performed comparably in terms of accuracy, though the former consistently demonstrated tighter generalization gaps between training and test performance.
🏷️ Themes
Machine Learning, Model Comparison, Tabular Regression, Smooth-basis Models
📚 Related People & Topics
Radial basis function
Type of mathematical function
In mathematics a radial basis function (RBF) is a real-valued function φ {\textstyle \varphi } whose value depends only on the distance between the input and some fixed point, either the origin, so that φ ( ...
Entity Intersection Graph
No entity connections available yet for this article.
Original Source
--> Computer Science > Machine Learning arXiv:2602.22422 [Submitted on 25 Feb 2026] Title: Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression Authors: Luciano Gerber , Huw Lloyd View a PDF of the paper titled Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression, by Luciano Gerber and Huw Lloyd View PDF HTML Abstract: Smooth-basis models such as Chebyshev polynomial regressors and radial basis function networks are well established in numerical analysis. Their continuously differentiable prediction surfaces suit surrogate optimisation, sensitivity analysis, and other settings where the response varies gradually with inputs. Despite these properties, smooth models seldom appear in tabular regression, where tree ensembles dominate. We ask whether they can compete, benchmarking models across 55 regression datasets organised by application domain. We develop an anisotropic RBF network with data-driven centre placement and gradient-based width optimisation, a ridge-regularised Chebyshev polynomial regressor, and a smooth-tree hybrid (Chebyshev model tree); all three are released as scikit-learn-compatible packages. We benchmark these against tree ensembles, a pre-trained transformer, and standard baselines, evaluating accuracy alongside generalisation behaviour. The transformer ranks first on accuracy across a majority of datasets, but its GPU dependence, inference latency, and dataset-size limits constrain deployment in the CPU-based settings common across applied science and industry. Among CPU-viable models, smooth models and tree ensembles are statistically tied on accuracy, but the former tend to exhibit tighter generalisation gaps. We recommend routinely including smooth-basis models in the candidate pool, particularly when downstream use benefits from tighter generalisation and gradually varying predictions. Comments: 32 pages, 6 figures, 11 tables. Submitted to Information Sciences Subjects: Machine Learn...
Read full article at source