Shrinking expensive health AI models without losing accuracy

Tabular foundation models excel at predicting health outcomes but require heavy computational infrastructure at inference time, blocking deployment in resource-constrained settings. Researchers used knowledge distillation to transfer the predictive power of large foundation models into lightweight tabular models, carefully avoiding context leakage by training teachers on hold-out folds. Across 19 healthcare datasets, distilled students maintained at least 90% of their teacher's AUC performance—sometimes exceeding it—while running 26× faster on CPU and preserving calibration and fairness metrics critical for clinical use.