A non-negligible proportion of patients suffer from cardiac arrest after cardiac surgery, resulting in poor prognoses. Reference Ahmadi and Aarabi1 Small children like neonates and infants are more prone to experience adverse events, such as cardiac arrest, especially for those who have single-ventricle circulation and complex heart anomalies. To prevent such major adverse events in the paediatric ICU, predictive physiological markers have been sought. Reference Duke, Butt, South and Karl2 Serum lactate levels and its trend may indicate haemodynamic instability. Reference Dell’Anna, Sandroni and Lamanna3 Monitoring serum lactate has been recommended for patients undergoing resuscitation, Reference Bakker, Nijsten and Jansen4,Reference Fuller and Dellinger5 and a rapid decrease in serum lactate levels has been identified as an indicator of good outcome/prognosis in an adult population. Reference Han, Kim, Lee, Park, Lee and Lee6 Unfortunately, direct measurement of lactate requires discrete blood sampling, Reference Duke, Butt, South and Karl2 normally not a serious concern in an adult population; however, in the case of pre-mature and at-risk infants, frequent blood draws can be a strain on the patient and can increase the risk of iatrogenic anaemia and infection, Reference Topjian, Clark and Casper7 helping motivate the development of alternative methods for assessing lactate non-invasively. The monitoring of lactate in a paediatric population has been identified as potentially playing an important role in predicting patient outcomes after cardiac surgery. Reference Badreldin, Doerr and Elsobky8–Reference Vida, Tessari and Cristante10
An alternative method to estimate lactate to evaluate haemodynamic stability/instability continuously and non-invasively may assist in improving the standard of patient care. Characteristics of the arterial waveform may share a relationship with haemodynamic instability. Although machine learning for diagnosis and prediction of outcomes in the paediatric population is not widely introduced yet, previous work successfully identified periventricular leukomalacia in neonates after cardiac surgery. Reference Jalali, Simpao, Gálvez, Licht and Nataraj11 This is one of the examples that machine learning plays a significant role in diagnosing the paediatric population who undergo cardiac surgery. Reference Belle, Ansari and Spadafore12,Reference Cannesson, Hofer and Rinehart13 These approaches Reference Topjian, Clark and Casper7,Reference Cannesson, Hofer and Rinehart13 employ the standard paradigm in machine learning known as classification, whereby the learning algorithm is responsible for a group-wise prediction, in this case evaluating a new sample as being an example set of measurements from a patient that is either haemodynamically stable or haemodynamically unstable. This machine learning classification approach provides limited information to the clinicians responsible for patient care. In practice, it is common for specific values of blood lactate (such as >40 mg/dL) to be relied upon as an actionable event provoking preventative treatment for the patient to reduce the risk of cardiac arrest. Reference Duke, Butt, South and Karl2,Reference Shekerdemian, Duke and Toda14 Since serum lactate levels are relied upon clinically, in this manuscript, we propose a machine learning-based regression approach (as opposed to the more traditional classification approach) that supports the direct targeting of serum lactate levels by machine learning in a paediatric population. Thus, in this study, we hypothesise that blood lactate in paediatric ICU patients can be predicted using machine learning applied to arterial waveforms and perioperative characteristics.
Methods
The institutional review board approved this research work as a prospective observational study on 14 November 2016, pursuant to approval number B16-110, and the requirement for informed patient consent was waived due to the non-interventional and low-risk nature of this retrospective study.
Patient characteristics
Forty-eight children, 23 females and 25 males, median age of 4 months (2.9–11.8 months, interquartile range), a median body weight of 5.4 kg (4.3–6.6 kg interquartile range), who underwent heart surgery, were included in Table 1. The most common diagnosis was a ventricular septal defect in 14 patients, followed by a double-outlet right ventricle in 8 patients, a complete atrioventricular septal defect in 6 patients, 3 patients had hypoplastic left heart syndrome, 3 patients had pulmonary atresia with a ventricular septal defect, and 3 patients exhibited tetralogy of Fallot. The most common procedures performed were ventricular septal defect closure in 12 patients, followed by repair of double-outlet right ventricle in five patients, and Fontan operation in three patients.
Patient characteristics.
ASD: atrial septal defect; ASO: arterial switch operation; BCPS: bicaval cavo-pulmonary shunt; BT shunt: Blalock–Taussig shunt; c-AVSD: complete atrioventricular septal defect; DORV: double-outlet right ventricle; d-TGA: d-transposition of great arteries; HLHS: hypoplastic left heart syndrome; IAA/VSD: interruption of aortic arch with ventricular septal defect; IQR: interquartile range; LCA: left coronary artery; MS: mitral stenosis; MVR: mitral valve replacement; p-AVSD: partial atrioventricular septal defect; PDA: patient ductus arteriosus; PAB: pulmonary artery banding; PAPVD: partial anomalous pulmonary vein drainage; PA/VSD: pulmonary atresia with ventricular septal defect; RVOTR: right ventricle outflow tract repair; RV to PA: right ventricle to the pulmonary artery; TA: tricuspid atresia; TOF: tetralogy of Fallot; VSD: ventricular septal defect.
Data acquisition
Perioperative patient variables and patient demographics were obtained from the patient medical record and bedside monitoring system. Patient characteristics were provided as input feature measurements to all our machine learning models alongside cardiac waveform data. Patient characteristics included age, gender, and weight. In order to assess the characteristics and shape of the arterial waveform, a newly developed novel software technology developed commercially (Medical Try Systems, Tokyo, Japan) was used to calculate the area under the cardiac waveform and peak angle at every stroke from the cardiac waveform (see Fig 1 for an illustrative example) and stored for 24–48 hours. Levels of blood drawn lactate are also stored.
Measuring the area under the curve and peak angle of the arterial waveform.
Arterial waveform characteristics
The arterial waveform, acquired from either side of the radial artery, was shown on a bedside monitor, then analogue signals of the arterial waveform were transmitted to a laptop computer with the converter, and a newly designed computer programme “Arterial Waveform Processing System” that rebuilds the arterial waveform to calculate the area under the curve and the peak angle of the arterial waveform from each acquired heartbeat (Fig 1).
Variables used as input are defined by the following equations:
$$P{{mean}} = {{\left( { P {{systolic}} + 2 \times P{{diastolic}}} \right)} \over 3}$$
The area under the arterial waveform curve was determined by subtracting the diastolic pressure (Pdiastolic) from the blood pressure (BP), as is illustrated in equation (2):
$${{Area\ under\ the\ Curve}} = \mathop \sum \nolimits_{n = 0}^N ({{BP}}(n) - P{{diastolic}}) \times {\rm{\Delta }}t$$
The normalised area under the curve was calculated according to equation (3):
$${Area\ {normalized}} = {{{Area\ {under\ the\ Curve}}} \over {P{{systolic}} - P{{diastolic}}}}$$
In various cases, an area (minute) value can also be determined, corresponding to an expanded value of the area for a 1-minute interval of the cardiac waveform. The area (minute) value was calculated according to equation (4):
$$\matrix{
{Area\;minute = Area\;under\;the\;Curve} \cr
{\quad\quad\quad\quad\quad\quad \times {{60,000\;{\rm{ms}}} \over {Stroke\;Time\;Per\;Beat(ms)}}} \cr
} $$
The peak angle of the arterial waveform (see upper part of illustrative Fig 1) can be determined as the angle between an upstroke line and a downstroke line of each waveform cycle.
These extracted cardiac waveform characteristics were automatically calculated in each heartbeat and shown as an average of five beats presented on the computer screen and recorded continuously from the moment when the patient was admitted to the paediatric ICU for 24–48 hours. When the blood gas was drawn, the level of lactate, as well as the timing of the blood drawn, was recorded in the programme. For each patient, the patient demographics including age, gender, weight and height, date of the surgery, diagnosis, and type of surgery were recorded in the programme before or after the admission to paediatric ICU. All records were summarised and analysed in an anonymous way post-operatively.
All the measurements acquired that are provided to machine learning are summarised in Table 2, including patient characteristics, fixed haemodynamics measurements at admission to paediatric ICU, continuous haemodynamic measurements and characteristics of the arterial waveform.
Measurements input to machine learning for lactate prediction.
BP: blood pressure; HR: heart rate; PICU: paediatric ICU.
Machine learning
Model design and approaches
Machine learning has been implemented as a regression problem, allowing the learned models to directly target the patient’s serum lactate and theoretically predict measurements that are directly relied upon clinically as actionable events prompting interventions to reduce the risk of cardiac arrest. Regression approaches potentially provide statistical robustness improvements to the learning machine relative to classification approaches. Patient characteristics and physiological measurements provided in Table 1 were acquired and analysed using specialised software/hardware. Predicting a patient’s blood lactate levels was accomplished using six different regression-based supervised learning algorithms.
Two main approaches were considered: basing the prediction on the currently acquired physiological measurements along with those acquired at admission, as well as a second approach that involved adding the most recent lactate measurement and the time since that measurement as prediction parameters. The second approach supports updating the learning system’s predictive capacity whenever a patient has a new ground truth blood lactate reading acquired.
These machine learning designs (both approaches) are capable of providing lactate estimates at any point in the monitoring process, which enables continuous patient monitoring and live feedback for clinicians. The second model which includes the most recent blood draw as an input parameter is effectively adaptive. The adaptive system’s input includes the most recent blood draw lactate concentration as an input feature measurement alongside the time since the most recent blood-drawn lactate was sampled. We examined which in-lab-validated learning models (six learning technologies) will support each of the two approaches (non-adaptive and adaptive) in terms of the quality and predictability of lactate levels to assess prediction accuracy by mean absolute error (in mg/dL). The performance of different mathematical algorithms was assessed by calculating the mean absolute error which is the mean of the absolute difference between the actual and predicted lactate concentrations. A low mean absolute error suggests superior performance of a particular mathematical model.
Learning models included in the analysis
We compared six different learning models to assess their potential in each of our two approaches (non-adaptive and adaptive learning), and this included Hypertuned random forest, Random forest regressor, AdaBoost regressor, Hypertuned AdaBoost, Decision tree, and Hypertuned decision tree.
Validation
All algorithms were compared with hold-out cross-validation. Specifically, we applied K-Fold cross-validation (K = 5, 10) to ensure that training and testing samples are kept separate where the training set is split into K smaller sets, with K-1 sets included for training and the remaining held-out set relied upon for testing evaluation. The process is repeated such that each of the K sets are held out on different validation runs, and the performance of the learners is averaged across the K validation trials. The six learning models and two different learning approaches (adaptive and non-adaptive learning models) result in twelve different approaches being evaluated in this study. K-fold cross-validation was performed in a fair manner, whereby for each validation run, each of the 12 learning models was trained on identical samples and tested on identical held-out test samples, providing standardisation in the comparisons reported in this study. Furthermore, K-fold randomisation was performed at the patient level instead of the more traditional randomisation based on blood lactate examinations (ground truth data). This validation strategy prevents our learning algorithms from being able to test or validate on held-out blood lactate readings from patients who had other data contributing to the training of the model. This strategy helps ensure that the findings we present in this study are robust across patients and will be much more likely to function appropriately when progressing to prospective clinical evaluation. We performed a correlation analysis across all of the variables included in the study. We also performed an analysis of feature importance to inform the reader and clinicians/users of the technology, regarding the apparent contributions of each measurement acquired to the resultant lactate predictions.
Results
Figure 2 provides the correlation matrix of data where the relationship between two variables is visually illustrated. Figure 2 shows the absolute correlation of independent variables to the target level of lactate (see bottom row), as well as the correlative relationship between all variables. Appendix Table 1 provides the numerical values of the correlations between variables of interest and the target lactate value. In model 1 (non-adaptive), the lactate level at admission had the highest value, in model 2 (adaptive), the previous lactate level had the highest correlation value. Feature importance assessment ranked our leading underlying features as: the initial lactate level, hours after admission, area under the curve, and sharpness of the peak angle of the arterial waveform. In the two different approaches (non-adaptive and adaptive), the performance for the first model (traditional non-adaptive learner) was observed with the hypertuned random forest, which yielded a mean absolute error of 5.60 mg/dL. The performance from the second approach (adaptive to updates in patient blood lactate) was also based on the hypertuned random forest, which yielded a leading mean absolute error of 3.38 mg/dL when predicting blood lactate with updated ground truth. This leading model produced lactate predictions that yielded a 0.73 correlation with ground truth blood drawn lactate readings.
Correlations between variables input to our machine learning models as a hot map. Note that red values are highly correlated variables.
Discussion
A non-negligible proportion of post-cardiac surgery patients suffer from cardiac arrest that necessitates cardiac massage and eventually results in an extracorporeal membrane oxygenation insertion. Poor patient outcomes such as death are possible, and even if the patient’s life is saved, the patient could experience complications such as brain damage. Reference Bhat, Hirsch and Gelehrter15,Reference Lorusso, Raffa and Kowalewski16 Small children like neonates and infants are more prone to experience cardiac arrest, especially for those who have single-ventricle circulation and complex heart anomalies. Intensivists and physicians treat the patients to ameliorate the at-risk haemodynamics based on theory and their experiences; however, there are occasions when it is uncertain what is the best treatment option in very critical situations. To address this problem, multiple tools have been invented and clinically relied upon Reference Davies, Vistisen, Jian, Hatib and Scheeren17–Reference Giomarelli, Biagioli and Scolletta19 where some of the haemodynamic indices are associated with patient cardiac output. However, these devices are not usually suitable for the paediatric population and those with complex CHD. Predicting haemodynamic instability before causing cardiac arrest would be greatly beneficial to the patients to save their lives. Reference Castiñeira, Schlosser and Geva20,Reference Seear, Scarfe and LeBlanc21 Bose and colleagues were successful in performing a single-institutional retrospective cohort study where their model was able to identify impending cardiac arrest at least 2 hours prior to the event with an overall accuracy of 75%. Reference Bose, Verigan and Hanson22,Reference Bose, Verigan and Hanson23 Also, Barker et al proposed a hybrid neural network for the prediction of mortality risk in neonatal ICUs. Reference Baker, Xiang and Atkinson24 As is the case with these examples, machine learning can efficiently model a variety of patient measurements, variables, and even model the passage of time.
This is a preliminary study to apply machine learning for a prediction model of the level of lactate, a well-accepted marker of haemodynamic instability Reference Duke, Butt, South and Karl2,Reference Shekerdemian, Duke and Toda14,Reference Joudi, Fathi, Soltani and Izanloo25–Reference Şahutoğlu, Yaşar, Kocabaş, Aşkar, Ayık and Atay27 aiming for clinical utility in a paediatric population, which has considerable potential to benefit from reduced physical blood draws, unlike the creation of similar technologies for an adult population, Reference Huang, Casaburi and Liao28 for which blood draws pose little concern.
One of the novel features of this technique is that it can predict the level of lactate continuously from the available patient data. This continuous system predicting the level of lactate may potentially be able to reduce the blood draws wherein now the average blood draws in each patient is 8–10 times for 48 hours, the frequency of blood draws can be reduced by estimating the continuous values of lactate in between less frequent blood draws. Furthermore, in the future, this method can potentially strengthen the standard of care by enabling the continuous prediction of lactate concentration and to inform clinicians before adverse events happen and can provide easy to interpret warnings associated with rising patient serum lactate levels. Thus, continuous feedback potentially supports clinicians in assessing the trajectory and dynamic changes of lactate concentrations. As a result of our study, the hypertuned random forest showed the greatest performance with the lowest mean absolute errors. The model exhibited improved accuracy when provided with the most recent level of lactate and the time since that most recent blood draw. One of the novel features of our method is to predict the level of lactate directly instead of grouping the range of patient lactate levels (high, middle, and low) into stratified groups to support a more traditional classification-based machine learning approach. In each heartbeat, variations in the characteristics of the arterial waveform and other influential variables can inform prediction of the patient’s level of serum lactate. This is a novel technique and the most applicable to clinicians’ needs in paediatric ICU where the critically ill patient’s haemodynamics can drastically change with time. Each component of the arterial waveform, diastolic pressure, peak pressure, ejection time, rate of rising of arterial pressure during systole, and mean arterial pressure are based on several interrelated ventriculo-arterial processes. Reference Esper and Pinsky29 There is a great deal of information that can be gleaned from examining the arterial waveform.
As expected and consistent with our clinical impression when treating the patients in paediatric ICU, the characteristics and morphology of the arterial waveform hold significant value. Even if the systolic blood pressure is the same, when the arterial waveform is highly peaked and narrow shaped, the levels of lactate are often elevated, while when the arterial waveform is fat-shaped, the haemodynamics are stable, and the level of lactate tends to be low. Also, the patient’s initial condition when admitted to paediatric ICU is a very important factor that affects assessment of the patient’s haemodynamic stability/instability on the basis of what operations they have recently undergone, and these factors may influence the post-operative course. The results of this study were supportive of this concept. The results of the current study have demonstrated that machine learning has great potential to predict haemodynamic instability by including a series of measurements and variables that are available in modern non-invasive bedside monitoring devices. Limitations of the study include the nature of the retrospective study. A small number of patients with disproportionate patient diagnoses and age may have influenced the study findings. We did not include the variables such as diagnosis and type of procedure for the prediction model because including these features degraded model performance slightly; however, this might be mainly due to the small number and variety of cases. In addition, none of the patients experienced a mortality-related event nor did any undergo an extra-corporeal membrane oxygenation insertion during the study period. Although we could predict the level of lactate with this technology, lactate is not a single indicator for haemodynamic instability/stability, but it is noteworthy that Meredith et al asserted that persistently elevated post-operative lactate is associated with increased morbidity and mortality in the paediatric cardiac population. Reference Allen30,Reference Domico and Allen31 Also, increased lactate production is multifactorial: anaerobic metabolism and hyperglycaemia. Increased lactate load by packed red blood cell transfusions and decreased lactate clearance may all result in hyperlactatemia. Reference Desplanque, Hamaide-Defrocourt, Berkia, Tourneur, Albinni and Bojan32 In children and adolescents, benign lactic acidosis is common without haemodynamic disturbance. Reference Horvath, Moga and Dassenko33 These post-operative behaviours of lactate must be investigated in future studies. The underlying technology created has the potential to be expanded upon to not only predict a patient’s current lactate, potentially in lieu of a blood draw, but to predict a trend curve of a patient’s lactate extending into the future. We anticipate predicting future lactate trends and future trends of other important clinical measurements based on arterial waveform characteristics. We are currently upgrading the system to record more variables which appear on the bedside monitor in a larger cohort by increasing participating institutions. The technology may be able to overcome any shortcomings associated with lactate being a late marker of inadequate systemic oxygen delivery. A large-scale dataset including a range of patient populations and outcomes to train the programme is likely to strengthen the utility and predictability of haemodynamic instability and eventually is expected to play a significant role towards preventing unfortunate outcomes in high-risk paediatric ICU patients.
Rigorous hold-out validation was performed in this study in order to ensure that the results reported are likely to generalise well to future data from patients not yet seen by the learning model. Validation was performed with K-Fold randomisation at the patient level instead of randomising based on each of the ground truth (blood lactate) readings available (there were typically many blood draws per patient). This validation strategy prevents our learning algorithms from being able to test or validate on held-out blood lactate readings from patients who had other data contributing to the training of the model. Thus, this strategy helps ensure that the findings we present in this study are robust across patients and will be much more likely to function appropriately when progressing to prospective clinical evaluation on a new collection of paediatric ICU patients. Future work will investigate the potential of the approaches presented in this manuscript as part of a prospective clinical study. Also, while this preliminary work involved a small cohort of low to moderate risk cardiac surgery patients, future work needs to not only predict future lactate continuously by mathematical algorithms, but more importantly, directly predict the occurrence of major adverse events after cardiac surgery. Future work will incorporate an upgraded data acquisition programme that will include all variables on the bedside monitor towards assessing the safety of the patient’s care and supporting predicting the patient’s haemodynamic instability toward prevention of cardiac arrest. Future work will also include a thorough assessment of the effects of techniques to address unwanted artefacts in the patient data, with detailed analysis of the resultant effects on model performance.
In conclusion, preliminary work to predict the direct level of lactate from post-operative variables, including the area/sharpness of the arterial waveforms, by using machine learning technology achieved promising performance that may represent technology of clinical utility. The hypertuned random forest demonstrated the best performance to predict the level of lactate through statistically rigorous hold-out validation, whereby the predictive accuracy is established on samples that the learning algorithm was not trained on. The most recently acquired lactate levels do not only help improve the model performance but also was the most dominant feature in the learning system. Broad data collection will strengthen the model performance and improve the predictability of haemodynamic deterioration. Future work will involve prospective evaluation of the technologies created in paediatric ICUs.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1047951122000932
Acknowledgements
We would like to thank Medical Try System (Tokyo, Japan) for developing the recording device for arterial waveform analysis and upgrading the system.
Financial supports
This study was partially supported by Fujii Setsuro Medical Research Fund in 2020 to KS, Miyata Cardiac Research Promotion Foundation in 2021 to KS, The Murata Science Foundation in 2021 to KS, and Tateishi Science and Technology Foundation in 2022 to KS. Also, this study was supported by a Natural Science and Engineering Research Council of Canada’s Canada Research Chair grant (grant number 231266) to JL, a Canada Foundation for Innovation and Nova Scotia Research and Innovation Trust infrastructure grant to JL, a Natural Science and Engineering Research Council of Canada Discovery Grant to JL, a St. Francis Xavier University research start-up grant to JL, a Springboard Atlantic Innovation grant and an ESCF Innovacorp grant to JL.
Conflicts of interest
Dr Levman is owner of Time Will Tell Technologies, Inc.
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national guidelines on human experimentation (Ethical Guidelines for Medical and Health Research Involving Human Subjects, Ministry of Health, Labor and Welfare in 2003) and with the Helsinki Declaration of 1975, as revised in 2008, and has been approved by the institutional committees (Approval number: B16-110).
Open access
