Two-Stage Constrained Model Selection and Cost-Sensitive Threshold Control for Explainable Diabetes Readmission Prediction
Model architecture and decision threshold jointly determine the behavior of a clinical prediction system. This study presents two complementary analyses of diabetes readmission prediction. Stage 1 applies Pareto enumeration, an area under the receiver operating characteristic curve (AUROC) tolerance, normalized utility, a worst-objective regret diagnostic, and 100,000 preference-weight simulations to five candidates with complete reported metrics. Stage 2 examines archived results for seven classifiers and explores calibration and cost-sensitive threshold selection for a separately refitted stack. In Stage 1, XGBoost and stacking were non-dominated; an absolute AUROC tolerance of 0.0005 selected XGBoost, which maximized utility in 99.88% of five-objective weight draws. In Stage 2, the stack achieved AUROC 0.70349 and average precision 0.66796. Platt scaling marginally reduced the refitted stack’s Brier score from 0.217859 to 0.217845. With illustrative false-negative and false-positive costs of 5,000 and 1,000 units, the test-grid minimum occurred at 0.20. Relative to 0.50, recall increased from 0.56475 to 0.99519 and total cost fell by 56.28%; however, 96.66% of encounters were flagged, and cost was only 3.74% lower than flagging all encounters. Shapley additive explanations (SHAP) described the individual LightGBM model. Preprocessing before splitting, encounter-level partitioning, test-informed calibration and threshold selection, and unresolved cohort provenance limit inference. The framework makes selection preferences and operating trade-offs explicit, but requires a locked, patient-separated evaluation before clinical use.
[1] International Diabetes Federation, 2025, IDF Diabetes Atlas (11th ed.). Brussels, Belgium: International Diabetes Federation.
[2] World Health Organization, 2016, Global Report on Diabetes. Geneva, Switzerland: World Health Organization.
[3] Strack B, DeShazo JP, Gennings C, et al., 2014, Impact of HbA1c measurement on hospital readmission rates: analysis of 70,000 clinical database patient records. BioMed Res Int, 2014: 781670. DOI:10.1155/2014/781670.
[4] Bhuvan MS, Kumar A, Zafar A, Kishore V, 2016, Identifying diabetic patients with high risk of readmission. arXiv:1602.04257. DOI:10.48550/arXiv.1602.04257.
[5] Rubin DJ, 2015, Hospital readmission of patients with diabetes. Curr Diab Rep, 15(4): 17. DOI:10.1007/s11892-015-0584-7.
[6] Song Y, Park J, Kim D, Kim C, 2024, Prediction of hospital readmission for type 2 diabetes patients using machine learning models. J Korean Data Inf Sci Soc, 35(1): 99-110. DOI:10.7465/jkdi.2024.35.1.99.
[7] Mishra V, Tanniru MR, Sreedharan J, 2025, Prediction of 30-day readmission in diabetes management using machine learning. Comput Biol Med, 195: 110616. DOI:10.1016/j.compbiomed.2025.110616.
[8] Artetxe A, Beristain A, Graña M, 2018, Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Programs Biomed, 164: 49-64. DOI:10.1016/j.cmpb.2018.06.006.
[9] Breiman L, 2001, Random forests. Mach Learn, 45: 5-32. DOI:10.1023/A:1010933404324.
[10] Friedman JH, 2001, Greedy function approximation: a gradient boosting machine. Ann Stat, 29(5): 1189-1232. DOI:10.1214/aos/1013203451.
[11] Chen T, Guestrin C, 2016, XGBoost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785-794. ACM. DOI:10.1145/2939672.2939785.
[12] Ke G, Meng Q, Finley T, et al., 2017, LightGBM: a highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems 30, pp. 3146-3154. Curran Associates.
[13] Wolpert DH, 1992, Stacked generalization. Neural Netw, 5(2): 241-259. DOI:10.1016/S0893-6080(05)80023-1.
[14] Fawcett T, 2006, An introduction to ROC analysis. Pattern Recognit Lett, 27(8): 861-874. DOI:10.1016/j.patrec.2005.10.010.
[15] Saito T, Rehmsmeier M, 2015, The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One, 10(3): e0118432. DOI:10.1371/journal.pone.0118432.
[16] Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW, 2019, Calibration: the Achilles heel of predictive analytics. BMC Med, 17: 230. DOI:10.1186/s12916-019-1466-7.
[17] Vickers AJ, Elkin EB, 2006, Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making, 26(6): 565-574. DOI:10.1177/0272989X06295361.
[18] Collins GS, Moons KGM, Dhiman P, et al., 2024, TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385: e078378. DOI:10.1136/bmj-2023-078378.
[19] Moons KGM, Damen JAA, Kaul T, et al., 2025, PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 388: e082505. DOI:10.1136/bmj-2024-082505.
[20] Lundberg SM, Lee SI, 2017, A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30, pp. 4765-4774. Curran Associates.
[21] Lundberg SM, Erion G, Chen H, et al., 2020, From local explanations to global understanding with explainable AI for trees. Nat Mach Intell, 2: 56-67. DOI:10.1038/s42256-019-0138-9.
[22] Fredrikson M, Jha S, Ristenpart T, 2015, Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322-1333. ACM. DOI:10.1145/2810103.2813677.
[23] Tramèr F, Zhang F, Juels A, Reiter MK, Ristenpart T, 2016, Stealing machine learning models via prediction APIs. In 25th USENIX Security Symposium, pp. 601-618. USENIX Association.
[24] Shokri R, Stronati M, Song C, Shmatikov V, 2017, Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy, pp. 3-18. IEEE. DOI:10.1109/SP.2017.41.
[25] Dwork C, Roth A, 2014, The algorithmic foundations of differential privacy. Found Trends Theor Comput Sci, 9(3-4): 211-407. DOI:10.1561/0400000042.
[26] Miettinen K, 1999, Nonlinear Multiobjective Optimization. Kluwer Academic Publishers.
[27] Deb K, Pratap A, Agarwal S, Meyarivan T, 2002, A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput, 6(2): 182-197. DOI:10.1109/4235.996017.
[28] Coello Coello CA, Pulido GT, Lechuga MS, 2004, Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput, 8(3): 256-279. DOI:10.1109/TEVC.2004.826067.
[29] Ehrgott M, 2005, Multicriteria Optimization (2nd ed.). Springer.
[30] Marler RT, Arora JS, 2004, Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim, 26: 369-395. DOI:10.1007/s00158-003-0368-6.
[31] Chankong V, Haimes YY, 1983, Multiobjective Decision Making: Theory and Methodology. North-Holland.
[32] Belton V, Stewart TJ, 2002, Multiple Criteria Decision Analysis: An Integrated Approach. Kluwer Academic Publishers.
[33] Steyerberg EW, 2019, Clinical Prediction Models (2nd ed.). Springer. DOI:10.1007/978-3-030-16399-0.
[34] Zhou Z, Chen X, Li E, Zeng L, Luo K, Zhang J, 2019, Edge intelligence: paving the last mile of artificial intelligence with edge computing. Proc IEEE, 107(8): 1738-1762. DOI:10.1109/JPROC.2019.2918951.
[35] Shi W, Cao J, Zhang Q, Li Y, Xu L, 2016, Edge computing: vision and challenges. IEEE Internet Things J, 3(5): 637-646. DOI:10.1109/JIOT.2016.2579198.
[36] Jibril AU, 2026, An explainable hybrid AI framework for diabetes hospital readmission risk prediction: clinically informed feature engineering, ensemble learning and edge-readiness design. Doctoral thesis manuscript submitted to Skyline University Nigeria, September 2026. Chapter 4; Tables 4.8–4.11. Unpublished manuscript.
[37] Clore J, Cios K, DeShazo J, Strack B, 2014, Diabetes 130-US Hospitals for Years 1999–2008 [Dataset]. UCI Machine Learning Repository. DOI:10.24432/C5230J.
[38] Bergstra J, Bengio Y, 2012, Random search for hyper-parameter optimization. J Mach Learn Res, 13: 281-305.
[39] Snoek J, Larochelle H, Adams RP, 2012, Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25, pp. 2951-2959. Curran Associates.
[40] Riley RD, Snell KIE, Ensor J, et al., 2019, Minimum sample size for developing a multivariable prediction model: part II—binary and time-to-event outcomes. Stat Med, 38(7): 1276-1296. DOI:10.1002/sim.7992.
[41] Lekadir K, Frangi AF, Porras AR, et al., 2025, FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ, 388: e081554. DOI:10.1136/bmj-2024-081554.
[42] Vasey B, Nagendran M, Campbell B, et al., 2022, Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med, 28(5): 924-933. DOI:10.1038/s41591-022-01772-9.
