Chebyshev polynomial regressors and radial basis function networks match gradient-boosted trees on tabular accuracy benchmarks, according to new research.
Researchers built three smooth-basis models - an anisotropic RBF network, a ridge-regularized Chebyshev polynomial regressor, and a smooth-tree hybrid - and tested them across 55 regression datasets spanning multiple application domains. A pre-trained transformer ranked first on accuracy across most datasets, but it requires a GPU, carries inference latency, and has dataset-size constraints that limit deployment in the CPU-bound environments common in applied science and industry. Among CPU-viable models, smooth alternatives and tree ensembles ended up statistically tied on accuracy. The smooth models edged ahead on one metric: tighter generalization gaps, meaning less divergence between training and held-out performance.
Tabular regression has been tree-ensemble territory for years - XGBoost and LightGBM have survived challenge after challenge, including repeated deep-learning incursions. Smooth-basis models offer a different value proposition: continuously differentiable prediction surfaces suited to surrogate optimization, sensitivity analysis, and other workflows where a gradually varying response is an asset rather than a limitation.
All three models ship as scikit-learn-compatible packages, lowering the bar to try them - though the recommendation amounts to "add these to your candidate pool," a notably modest ask compared with papers that have promised for years to finally unseat the trees.