Data Presentation Over Architecture: Resampling Strategies for Credit Risk Prediction with Tabular Foundation Models
Tabular Foundation Models are gaining traction for credit risk prediction, but a new study reveals that how data is presented to these models matters far more than which model architecture you choose. Researchers benchmarked five TFMs against classical baselines on real lending datasets, finding that balanced and hybrid sampling strategies outperform uniform sampling by 3-4 AUC points, a gap that dwarfs performance differences between competing TFM families. The finding challenges the assumption that model selection drives performance in imbalanced tabular tasks and suggests practitioners should prioritize context construction over architecture shopping, with optimal results emerging at 5K-10K examples per context window.
Modelwire context
ExplainerThe study isolates sampling strategy as a lever independent of model family, suggesting that practitioners have been optimizing the wrong variable. The finding also pins optimal context window size to a narrow band (5K-10K examples), a concrete operational constraint that wasn't previously quantified.
This connects directly to the distillation work from the same day (Pocket Foundation Models), which solved deployment latency by converting TFMs into gradient-boosted trees. That work assumed TFMs were worth distilling because they were fundamentally superior. This new study suggests the gap was partly an artifact of how data was fed to them, not model capacity. Together, they imply that the practical advantage of TFMs in production may be narrower than the research community has assumed, and that classical methods with better data presentation could close more of the gap than previously thought.
If practitioners applying this resampling strategy to XGBoost or CatBoost on Home Credit and Lending Club datasets match the 3-4 AUC point gains reported for TFMs, that confirms the finding generalizes beyond foundation models. If the gains collapse when context window shrinks below 5K or exceeds 10K on out-of-sample lending data, that validates the window size constraint as a real boundary rather than a dataset artifact.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsTabular Foundation Models · Home Credit · Lending Club
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.