In trauma care, as in all of medical care, we try to predict the future. What injuries does my patient have? What will happen if I treat this fracture that way? Is she going to live? How much disability can we expect given this degree of head injury?

Trauma professionals are constantly tapping into their own experience and that of others to predict the future and try to shape it in the best way for their patients. And now more than ever, with the combination of mathematical algorithms and powerful machine learning systems, we’ve been able to move past simple correlations and linear regressions to try to peer into that future.

A group at Washington University in St. Louis previously developed a real time risk score that claims to predict the need for cardiovascular support in ICU patients.  It is called the hemodynamic instability indicator (HII). The exact details of this score are not included in the abstract, and I have not found it published yet so I have no idea how it was derived. The presenters prospectively applied this system to 126 stable patients who were admitted to the ICU and were expected to stay at least 24 hours and survive at least 48 hours. They wanted to determine how well HII predicted an episode of hemodynamic instability.

Here are the factoids:

• The majority were male (64%) acute care surgery patients (55%) with a median age of 60
• Only 60 of the 126 patients had sufficient data to calculate HII in the pre-intervention period of unstable patients (!)
• HII predicted the need for pressors/inotropes with a sensitivity of 0.56 and specificity of 0.76. The authors claim that this was statistically significant (p < 0.01) (???)
• The system got better as the time to intervention for instability grew closer

Bottom line: Machine learning and prediction systems can be tricky tools. They are very good at identifying patterns without anything more than a good training dataset. However, they are only as good as that dataset. It is crucial that the system be trained with and tested against other large sets of data with a variety of patients. Otherwise, you will create a great system for predicting events in 60 year old male acute care surgery patients, and no one else.

Here are some questions for the authors to consider before their presentation:

• Be prepared to describe in detail how you derived the original HII system. How big was the dataset, and what did it look like in terms of demographics?
• What statistics did you use to conclude you had a statistical p value of < 0.01? Your sensitivity and specificity numbers do not look that good. What about negative and positive predictive values?
• You mentioned that the system made better predictions as the episode of instability grew closer. Predicting an adverse event 24 hours in advance vs 5 minutes in advance is very different. How near did the event have to be for good prediction? Did this factor into your significance calculation above?
• Why not use a receiver operating characteristic curve to show your data? It is a much better analysis tool.
• Big picture questions: Why do you expect that you can generalize the results of your HII system to new and disparate datasets? Have you tried it on major trauma patients?

Reference: EAST 2018 Podium paper #5.