Tag Archives: AI

Best Of EAST #17: Artificial Intelligence vs TRISS

The TRISS score is the great grand-daddy of probability of survival prediction in trauma, first introduced in 1981. It is a somewhat complicated equation that takes the injury severity score (ISS), revised trauma score (RTS), and age and cranks out a probability between 0 and 100%. Over the years, this system has been well validated, and its shortcomings have been elucidated as well.

Many authors have attempted to develop a system that is better than TRISS. Years ago, there was the New-TRISS. And back in the day (early 1990s) I even developed a neural network to replace TRISS. In general, all of these systems may improve accuracy by a few percent. But it has never been enough to prompt us to ditch the original system.

The group at the University of California at Los Angeles developed a machine learning algorithm using ICD-10 anatomic codes and a number of physiologic variables to try to improve upon the original TRISS score. They analyzed three years of NTDB data and attempted to predict in-hospital survival.

Here are the factoids:

  • The authors used over 1.4 million records to develop their model
  • Overall, 97% of patients survived, and survivors tended to be younger, have higher blood pressure, and have sustained a blunt mechanism (no surprises here at all)
  • The ROC C-statistic for the false positive rate was better with the machine learning model (0.940 vs 0.908), as was the calibration statistic (0.997 vs 0.814)

Here is the ROC curve for machine (blue) vs TRISS (yellow):

The authors conclude that the machine learning model performs better than TRISS and that it may improve stratification of injury.

Bottom line: This study is one of many attempting to improve upon good old TRISS probability of survival. Why have there been so many attempts, and none that have appeared to “stick?” Here are my thoughts:

  • They are complicated. Sure, the original TRISS equation is slightly complicated, but it’s nothing close to a machine AI algorithm.
  • The inner workings are opaque. It’s not very easy to “open the box” and see which variables are actually driving the survival calculations.
  • The results are only as good as the training data. There is a real skew toward survival here (97%), so the algorithm will more likely be right in guessing that the patients will survive.
  • The improvements in these systems are generally incremental. In this case the ROC value increases from .908 to .940. Both of these values are very good.

In general, any time a new and better algorithm is introduced that shows much promise, someone wants to patent it so they can monetize the work.  Obviously, I don’t know anything about the plans for this algorithm. Somehow I doubt that many centers would be willing to abandon TRISS for an incremental improvement that may not be clinically significant at any price.

Here are my questions for the authors and presenter:

  • Please detail how you selected the variables to enter into the machine learning algorithm. Were they chosen by biased humans who had some idea they might be important, or did the AI comb the data and try to find the best correlations?
  • Be sure to explain the ROC and calibration statistics well. Most of the audience will be unfamiliar.
  • Are you using your model in your own performance improvement program now? If so, how is it helping you? If not, why?

Fascinating paper! Let’s here more about it!

Reference: ICD-10-BASED MACHINE LEARNING MODELS OUTPERFORM THE TRAUMA AND INJURY SEVERITY SCORE (TRISS) IN SURVIVAL PREDICTION, EAST 35th ASA, oral abstract #38.

New Technology: Using AI To Interpret Pelvic X-rays

Look out, radiologists! The computers are coming for you!

Radiologists use their extensive understanding of human anatomy and combine it with subtle findings they see on x-ray shadow pictures. In doing this, they can identify a wide variety of diseases, anomalies, and injuries. But as we have seen with vision systems and game playing (think chess), computers are getting pretty good at doing this as well.

Is it only a matter of time until computer artificial intelligence (AI) starts reading x-rays?  Look at how good they already are at interpreting EKGs. The trauma group at Stanford paired up with the Chang Gung Memorial Hospital in Taiwan to test the use of AI for interpreting images to identify a specific set of common pelvic fractures.

The Stanford group used a deep learning neural network (XCeption) to analyze source x-rays (standard A-P pelvis images) from Chang Gung. These x-rays were divided into training and testing cohorts. The authors also applied different degrees of blurring, brightness, rotation, and contrast adjustment to the training set in order to help the AI overcome these issues when interpreting novel images.

The AI interpreted the test images with a very high degree of sensitivity, specificity, accuracy, and predictive values, with all of them over 0.90. The algorithms generated a “heat map” that showed the areas that were suspicious for fracture. Here are some examples with the original x-ray on the left and the heat map on the right:

The top row shows a femoral neck fracture, the middle row an intertrochanteric fracture, and the bottom row another femoral neck fracture with a contralateral implant. All were handily identified by the AI.

AI applications are usually only as good as their training sets. In general, the bigger the better so they can gain a broader experience for more accurate interpretation. So it is possible that uncommon, subtle fractures could be missed. But remember, artificial intelligence is meant to supplement the radiologist, not replace him or her. You can all breathe more easily now.

This technology has the potential for broader use in radiographic interpretation. In my mind, the best way to use it is to first let the radiologist read the images as they usually do. Once they have done this, then turn on the heat map so they can see any additional anomalies the AI has found. They can then use this information to supplement the initial interpretation.

Expect to see more work like this in the future. I predict that, ultimately, the picture archiving and communications systems (PACS) software providers will build this into their product. As the digital images are moving from the imaging hardware to the digital storage media, the AI can intercept it and begin the augmented interpretation process. The radiologist will then be able to turn on the heat map as soon as the images arrive on their workstation.

Stay tuned! I’m sure there is more like this to come!

Reference: Practical computer vision application to detect hip fractures on pelvic X-rays: a bi-institutional study.  Trauma Surgery and Acute Care Open 6(1), http://dx.doi.org/10.1136/tsaco-2021-000705.