The effect of machine learning in clinical prediction of septic shock in children: a systematic review and meta-analysis_Chinese Journal of Evidence-Based Medicine

Authors：

ZHEN Kaixuan ¹ , YAN Haolun ² , CHEN Longbiao ³ , YANG Xiaoqing ⁴ ,  WU Jinzhun ⁴

1. School of Pharmacy, Xiamen University, Xiamen 361102, P. R. China;
2. School of Medicine, Xiamen University, Xiamen 361102, P. R. China;
3. School of Information, Xiamen University, Xiamen 361102, P. R. China;
4. Department of Pediatrics, Women and Children's Hospital Affiliated to Xiamen University, Xiamen 361003, P. R. China;

Corresponding author：

WU Jinzhun, Email: 1923731201@qq.com

Keywords：

Septic shock in children; Predictive model; Machine learning; Meta analysis

DOI：

10.7507/1672-2531.202410169

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objectiv To provide a comprehensive overview of model performance and predictive efficacy of machine learning techniques to predict septic shock in children, in order to target and improve the quality and predictive power of models for future studies. Methods To systematically review all studies in four databases (PubMed, Embase, Web of Science, ScienceDirect) on machine learning prediction of septic shock in children before April 1, 2024. Two investigators independently conducted literature screening, literature data extraction and bias assessment, and conducted a systematic review of basic information, research data, study design and prediction models. Model discrimination, which area under the curve (AUC), was pooled using a random-effects model and meta-analysis was performed. Subgroup analyses were performed according to sample sizes, machine learning models, types of predictors, number of predictors, etc. And publication bias and sensitivity analyses were performed for the included literature. Results A total of 11 studies were included, of which 2 were at low risk of bias, 7 were at unknown risk of bias, and 2 were at high risk of bias. The data used in the included studies included both public and non-public electronic medical record databases, and the machine learning models used included logistic regression, random forest, support vector machine, and XGBoost, etc. The predictive models constructed based on different databases appeared to have different results in terms of the characteristic variables, so identifying the key variables of the predictive models requires further validation on other datasets. Meta-analysis showed the pooled AUC of 0.812 (95%CI 0.763 to 0.860, P<0.001), and further subgroup analyses showed that larger sample sizes (≥1 000) and predictor variable types significantly improved the predictive effect of the model, and the difference in AUC was statistically significant (95%CI not overlapping). The funnel plot showed that there was publication bias in the study, and when the extreme AUC values were excluded, the meta-analysis yielded a total AUC of 0.815 (95%CI 0.769 to 0.861, P<0.001), indicating that the extreme AUC values were insensitive. Conclusion Machine learning technology has shown some potential in predicting septic shock in children, but the quality of existing research needs to be strengthened, and future research work should improve the quality of research and improve the prediction effect of the model by expanding the sample size.

1.	袁远宏, 肖政辉. 儿童脓毒性休克早期识别与处理. 实用休克杂志(中英文), 2022, 6(3): 137-140,145.
2.	Weiss SL, Fitzgerald JC, Pappachan J, et al. Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med, 2015, 191(10): 1147-1157.
3.	Gavelli F, Castello LM, Avanzi GC, et al. Management of sepsis and septic shock in the emergency department. Intern Emerg Med, 2021, 16(6): 1649-1661.
4.	施春柳, 黄晓波, 何有美, 等. 儿童早期预警评分对脓毒症患儿病情及短期预后的临床预测. 中国医药科学, 2021, 11(11): 190-193.
5.	张芳. 降钙素原D-二聚体联合小儿危重病例评分在小儿脓毒症诊断及预后的临床应用价值. 中国药物与临床, 2020, 20(5): 757-758.
6.	Zhong M, Huang Y, Li T, et al. Day-1 PELOD-2 and day-1" quick" PELOD-2 scores in children with sepsis in the PICU. J Pediatr, 2020, 96: 660-665.
7.	方建, 刘宾宾, 陈超. 不同临床评分在预测脓毒血症患儿死亡风险中的应用价值. 现代实用医学, 2021, 33(6): 791-793.
8.	Lalitha AV, Satish JK, Reddy M, et al. Sequential organ failure assessment score as a predictor of outcome in sepsis in pediatric intensive care unit. J Pediatr Intensive Care, 2021, 10(2): 110-117.
9.	Cabrita JA, Pinheiro I, Falcão LM, et al. Rethinking the concept of sepsis and septic shock. Eur J Intern Med, 2018, 54: 1-5.
10.	张丽丹, 黄慧敏, 程玉才, 等. 4种小儿危重死亡评分对危重患儿死亡风险的预测价值. 中华危重病急救医学, 2018, 30(1): 51-56.
11.	Senthilkumar G, Madhusudhana S, Flitcroft M, et al. Automated machine learning (AutoML) can predict 90-day mortality after gastrectomy for cancer. Sci Rep, 2023, 13(1): 11051.
12.	Greener JG, Kandathil SM, Moffat L, et al. A guide to machine learning for biologists. Nat Rev Mol Cell Biol, 2022, 23(1): 40-55.
13.	Nichols JA, Herbert Chan HW, Baker MA, et al. Machine learning: applications of artificial intelligence to imaging and diagnosis. Biophys Rev, 2019, 11(1): 111-118.
14.	Debray TPA, Damen JAA, Snell KIE, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ, 2017: i6460.
15.	Fu H, Hou D, Xu R, et al. Risk prediction models for deep venous thrombosis in patients with acute stroke: A systematic review and meta-analysis. Int J Nurs Stud, 2024, 149: 104623.
16.	邓宇含, 刘爽, 王子尧, 等. 基于结构化数据和机器学习模型预测普通人群卒中发病风险的系统评价和meta分析. 中国卒中杂志, 2022, 17(11): 1189-1197.
17.	Newcombe RG. Confidence intervals for an effect size measure based on the Mann–Whitney statistic. Part 2: asymptotic methods and evaluation. Stat Med, 2006, 25(4): 559-573.
18.	Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 1982, 143(1): 29-36.
19.	The pediatric sepsis biomarker risk model. 2024.
20.	Wong HR, Cvijanovich NZ, Anas N, et al. Prospective testing and redesign of a temporal biomarker based risk model for patients with septic shock: implications for septic shock biology. EBioMedicine, 2015, 2(12): 2087-2093.
21.	Le S, Hoffman J, Barton C, et al. Pediatric severe sepsis prediction using machine learning. Front Pediatr, 2019, 7: 413.
22.	Ying J, Wang Q, Xu T, et al. Diagnostic potential of a gradient boosting-based model for detecting pediatric sepsis. Genomics, 2021, 113(1 Pt 2): 874-883.
23.	龚军, 钟小钢, 谈军涛, 等. “网格搜索+XGBoost”算法建立儿童脓毒性休克预测模型. 解放军医学杂志, 2020, 45(12): 1270-1276.
24.	Xiang L, Wang H, Fan S, et al. Machine learning for early warning of septic shock in children with hematological malignancies accompanied by fever or neutropenia: a single center retrospective study. Front Oncol, 2021, 11: 678743.
25.	杨春凤. 吉林省儿童脓毒性休克调查和死亡预测模型的建立与个体化治疗探索. 吉林大学, 2023.
26.	查皓宇, 谭睿, 王浩楠, 等. 儿童重症细菌感染死亡风险预测模型的建立及评价. 中华急诊医学杂志, 2023, 32(4): 489-496.
27.	Scott HF, Colborn KL, Sevick CJ, et al. Development and validation of a predictive model of the risk of pediatric septic shock using data known at the time of hospital arrival. J Pediatr, 2020, 217: 145-151.
28.	Liu R, Greenstein JL, Fackler JC, et al. Prediction of impending septic shock in children with sepsis. Crit Care Explor, 2021, 3(6): e0442.
29.	朱雪梅, 陈申成, 章莹莹, 等. 基于多中心队列数据的机器学习预测重症感染患儿死亡风险和筛选临床特征的研究. 中国循证儿科杂志, 2024, 19(1): 31-35.
30.	Machine Learning in Medicine. Circulation, 2024.
31.	Ahmad A, Imran M, Ahsan H, et al. Biomarkers as biomedical bioindicators: approaches and techniques for the detection, analysis, and validation of novel biomarkers of diseases. Pharmaceutics, 2023, 15(6): 1630.
32.	Li Y, Sperrin M, Ashcroft DM, et al. Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar. BMJ, 2020, 371.
33.	Andaur Navarro CL, Damen JAA, Takada T, et al. Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ Open, 2020, 10(11): e038832.
34.	Carrol ED, Ranjit S, Menon K, et al. Operationalizing appropriate sepsis definitions in children worldwide: considerations for the pediatric sepsis definition taskforce. Pediatr Crit Care Med, 2023, 24(6): e263.

1. 袁远宏, 肖政辉. 儿童脓毒性休克早期识别与处理. 实用休克杂志(中英文), 2022, 6(3): 137-140,145.
2. Weiss SL, Fitzgerald JC, Pappachan J, et al. Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med, 2015, 191(10): 1147-1157.
3. Gavelli F, Castello LM, Avanzi GC, et al. Management of sepsis and septic shock in the emergency department. Intern Emerg Med, 2021, 16(6): 1649-1661.
4. 施春柳, 黄晓波, 何有美, 等. 儿童早期预警评分对脓毒症患儿病情及短期预后的临床预测. 中国医药科学, 2021, 11(11): 190-193.
5. 张芳. 降钙素原D-二聚体联合小儿危重病例评分在小儿脓毒症诊断及预后的临床应用价值. 中国药物与临床, 2020, 20(5): 757-758.
6. Zhong M, Huang Y, Li T, et al. Day-1 PELOD-2 and day-1" quick" PELOD-2 scores in children with sepsis in the PICU. J Pediatr, 2020, 96: 660-665.
7. 方建, 刘宾宾, 陈超. 不同临床评分在预测脓毒血症患儿死亡风险中的应用价值. 现代实用医学, 2021, 33(6): 791-793.
8. Lalitha AV, Satish JK, Reddy M, et al. Sequential organ failure assessment score as a predictor of outcome in sepsis in pediatric intensive care unit. J Pediatr Intensive Care, 2021, 10(2): 110-117.
9. Cabrita JA, Pinheiro I, Falcão LM, et al. Rethinking the concept of sepsis and septic shock. Eur J Intern Med, 2018, 54: 1-5.
10. 张丽丹, 黄慧敏, 程玉才, 等. 4种小儿危重死亡评分对危重患儿死亡风险的预测价值. 中华危重病急救医学, 2018, 30(1): 51-56.
11. Senthilkumar G, Madhusudhana S, Flitcroft M, et al. Automated machine learning (AutoML) can predict 90-day mortality after gastrectomy for cancer. Sci Rep, 2023, 13(1): 11051.
12. Greener JG, Kandathil SM, Moffat L, et al. A guide to machine learning for biologists. Nat Rev Mol Cell Biol, 2022, 23(1): 40-55.
13. Nichols JA, Herbert Chan HW, Baker MA, et al. Machine learning: applications of artificial intelligence to imaging and diagnosis. Biophys Rev, 2019, 11(1): 111-118.
14. Debray TPA, Damen JAA, Snell KIE, et al. A guide to systematic review and meta-analysis of prediction model performance. BMJ, 2017: i6460.
15. Fu H, Hou D, Xu R, et al. Risk prediction models for deep venous thrombosis in patients with acute stroke: A systematic review and meta-analysis. Int J Nurs Stud, 2024, 149: 104623.
16. 邓宇含, 刘爽, 王子尧, 等. 基于结构化数据和机器学习模型预测普通人群卒中发病风险的系统评价和meta分析. 中国卒中杂志, 2022, 17(11): 1189-1197.
17. Newcombe RG. Confidence intervals for an effect size measure based on the Mann–Whitney statistic. Part 2: asymptotic methods and evaluation. Stat Med, 2006, 25(4): 559-573.
18. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 1982, 143(1): 29-36.
19. The pediatric sepsis biomarker risk model. 2024.
20. Wong HR, Cvijanovich NZ, Anas N, et al. Prospective testing and redesign of a temporal biomarker based risk model for patients with septic shock: implications for septic shock biology. EBioMedicine, 2015, 2(12): 2087-2093.
21. Le S, Hoffman J, Barton C, et al. Pediatric severe sepsis prediction using machine learning. Front Pediatr, 2019, 7: 413.
22. Ying J, Wang Q, Xu T, et al. Diagnostic potential of a gradient boosting-based model for detecting pediatric sepsis. Genomics, 2021, 113(1 Pt 2): 874-883.
23. 龚军, 钟小钢, 谈军涛, 等. “网格搜索+XGBoost”算法建立儿童脓毒性休克预测模型. 解放军医学杂志, 2020, 45(12): 1270-1276.
24. Xiang L, Wang H, Fan S, et al. Machine learning for early warning of septic shock in children with hematological malignancies accompanied by fever or neutropenia: a single center retrospective study. Front Oncol, 2021, 11: 678743.
25. 杨春凤. 吉林省儿童脓毒性休克调查和死亡预测模型的建立与个体化治疗探索. 吉林大学, 2023.
26. 查皓宇, 谭睿, 王浩楠, 等. 儿童重症细菌感染死亡风险预测模型的建立及评价. 中华急诊医学杂志, 2023, 32(4): 489-496.
27. Scott HF, Colborn KL, Sevick CJ, et al. Development and validation of a predictive model of the risk of pediatric septic shock using data known at the time of hospital arrival. J Pediatr, 2020, 217: 145-151.
28. Liu R, Greenstein JL, Fackler JC, et al. Prediction of impending septic shock in children with sepsis. Crit Care Explor, 2021, 3(6): e0442.
29. 朱雪梅, 陈申成, 章莹莹, 等. 基于多中心队列数据的机器学习预测重症感染患儿死亡风险和筛选临床特征的研究. 中国循证儿科杂志, 2024, 19(1): 31-35.
30. Machine Learning in Medicine. Circulation, 2024.
31. Ahmad A, Imran M, Ahsan H, et al. Biomarkers as biomedical bioindicators: approaches and techniques for the detection, analysis, and validation of novel biomarkers of diseases. Pharmaceutics, 2023, 15(6): 1630.
32. Li Y, Sperrin M, Ashcroft DM, et al. Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar. BMJ, 2020, 371.
33. Andaur Navarro CL, Damen JAA, Takada T, et al. Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ Open, 2020, 10(11): e038832.
34. Carrol ED, Ranjit S, Menon K, et al. Operationalizing appropriate sepsis definitions in children worldwide: considerations for the pediatric sepsis definition taskforce. Pediatr Crit Care Med, 2023, 24(6): e263.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesThe effect of machine learning in clinical prediction of septic shock in children: a systematic review and meta-analysis

Abstract Full text Figures/Tables Video References Cited by

Format

Content