Introducing TBMs: Tabulating Boosting Machines.

Pricing Frontier‘s new t-boost Open Source predictive modelling tool for insurance pricing.

Closing the gap between explainability and performance.

The logic behind these is that, despite their performance, GBMs fit a lot of noise and are prone to overfitting – if these can be trained in a way that’s transparent and reduces the noise, we could achieve a model that’s explainable and generalises better.

Transparency

If we build trees to be symmetric, then these can easily be represented as tables (a 3 deep, symmetrical tree, can be a lookup table of 8 values). A trained model with hundreds or thousands of tables can be condensed using fANOVA into base, 1-way, 2-way, 3.., k-way tables similar to the rating table structure we are familiar with in insurance.
However despite being tables this results in the model structure being mostly interactions, only the first split is a 1-way effect, every split after it is an interaction unless using the same feature.

Interaction Hurdle

During training, for each round, after the first split, in order for a table to use a new feature, the estimated gain of doing so must be greater than the gain of the first split multiplied by a hurdle (which increases for successive features).
This means that unless the new feature is a very strong interaction, tables will prefer to continue using feature(s) already used in earlier splits. Prioritising main effects and low order interactions, later rounds will have a lower gain for the first split, slowly allowing more interactions into the model.

Pruning

Once all the trained tables have been decomposed into main effect and interaction tables. These can be assessed and sorted by their gain, we’ve effectively separated the signal from the noise and those that don’t carry much signal can be pruned, reducing model complexity without losing performance.

Performance

To validate I took 40 insurance based datasets, built a simple GLM as a baseline along with TabPFN, XGBoost, CatBoost, and LightGBM in a similar style to the tab-arena benchmark. The TBM implementation (t-boost) leads whilst maintaining a tabular structure.

t-boost can now be installed.

This is the implementation of the TBM methodology.

It’s written in Rust with a Python API, so very fast for the amount of work it’s doing.

The code: GitHub – PricingFrontier/t-boost: Implementation of Tabulating Boosting Machines · GitHub, and can be installed with ‘uv add t-boost’.

It’s also now included in Haute so can play around with it there: Playground | Pricing Frontier

Brought to you by Ralph Clayton, Director at Pricing Frontier