All posts by TheTraumaPro

Best Of EAST #17: Artificial Intelligence vs TRISS

The TRISS score is the great grand-daddy of probability of survival prediction in trauma, first introduced in 1981. It is a somewhat complicated equation that takes the injury severity score (ISS), revised trauma score (RTS), and age and cranks out a probability between 0 and 100%. Over the years, this system has been well validated, and its shortcomings have been elucidated as well.

Many authors have attempted to develop a system that is better than TRISS. Years ago, there was the New-TRISS. And back in the day (early 1990s) I even developed a neural network to replace TRISS. In general, all of these systems may improve accuracy by a few percent. But it has never been enough to prompt us to ditch the original system.

The group at the University of California at Los Angeles developed a machine learning algorithm using ICD-10 anatomic codes and a number of physiologic variables to try to improve upon the original TRISS score. They analyzed three years of NTDB data and attempted to predict in-hospital survival.

Here are the factoids:

  • The authors used over 1.4 million records to develop their model
  • Overall, 97% of patients survived, and survivors tended to be younger, have higher blood pressure, and have sustained a blunt mechanism (no surprises here at all)
  • The ROC C-statistic for the false positive rate was better with the machine learning model (0.940 vs 0.908), as was the calibration statistic (0.997 vs 0.814)

Here is the ROC curve for machine (blue) vs TRISS (yellow):

The authors conclude that the machine learning model performs better than TRISS and that it may improve stratification of injury.

Bottom line: This study is one of many attempting to improve upon good old TRISS probability of survival. Why have there been so many attempts, and none that have appeared to “stick?” Here are my thoughts:

  • They are complicated. Sure, the original TRISS equation is slightly complicated, but it’s nothing close to a machine AI algorithm.
  • The inner workings are opaque. It’s not very easy to “open the box” and see which variables are actually driving the survival calculations.
  • The results are only as good as the training data. There is a real skew toward survival here (97%), so the algorithm will more likely be right in guessing that the patients will survive.
  • The improvements in these systems are generally incremental. In this case the ROC value increases from .908 to .940. Both of these values are very good.

In general, any time a new and better algorithm is introduced that shows much promise, someone wants to patent it so they can monetize the work.  Obviously, I don’t know anything about the plans for this algorithm. Somehow I doubt that many centers would be willing to abandon TRISS for an incremental improvement that may not be clinically significant at any price.

Here are my questions for the authors and presenter:

  • Please detail how you selected the variables to enter into the machine learning algorithm. Were they chosen by biased humans who had some idea they might be important, or did the AI comb the data and try to find the best correlations?
  • Be sure to explain the ROC and calibration statistics well. Most of the audience will be unfamiliar.
  • Are you using your model in your own performance improvement program now? If so, how is it helping you? If not, why?

Fascinating paper! Let’s here more about it!

Reference: ICD-10-BASED MACHINE LEARNING MODELS OUTPERFORM THE TRAUMA AND INJURY SEVERITY SCORE (TRISS) IN SURVIVAL PREDICTION, EAST 35th ASA, oral abstract #38.

Best Of EAST #16: More On TXA

Here’s another abstract dealing with TXA. But this one deals with the classic CRASH-2 use for patients with major bleeding. The original patient showed that TXA improves survival if given within 3 hours of injury. More and more prehospital units (particularly aeromedical services) have been administering TXA enroute to the trauma center to ensure that this drug is given as early as possible.

Many of these same services carry packed cells (or in rare cases, whole blood) so that proper resuscitation can be started while enroute as well. A multicenter group led by the University of Pittsburgh evaluated the utility of giving both TXA and blood during prehospital transport.

Their study summarizes some of the results of the Study of Tranexamic Acid During Air and Ground Medical Prehospital Transport Trial (STAAMP Trial). This study ran from 2015 to 2019 and randomized patients to receive either TXA or placebo during air or ground transport to a trauma center. It included blunt or penetrating patients at risk for hemorrhage within 2 hours of injury who were either hypotensive or tachycardic. Outcome measures included 30-day mortality, 24-hour mortality, and a host of complications.

This abstract outlines a secondary analysis that retrospectively reviewed the impact of using prehospital packed red cells (pRBC) in addition to the TXA/placebo during transport. 

Here are the factoids:

  • There were 763 patients in total, broken down as follows
    • TXA only – 350
    • pRBC only – 35
    • TXA + pRBC – 22
    • Neither – 356
  • Patients who received blood with or without TXA were more severely injured with ISS 22 vs 10-12 in the non-pRBC groups
  • Mortality was higher in the pRBC (23%) and TXA+pRBC groups (29%)
  • TXA alone did not decrease mortality
  • TXA + pRBC resulted in a 46% reduction in 30-day mortality but not at 24 hours
  • packed cells alone decreased 24-hour mortality by 47%

The authors concluded basically what was stated in the results: short term mortality was decreased by pRBC alone, and 30-day mortality with TXA + pRBC. They recommended further work to elucidate the mechanisms involved.

Bottom line: This abstract may also suffer from the “low numbers” syndrome I’ve written about so many times before. The conclusions are based on two small groups that make up only 7% of the entire study group. And these are the two groups with more than double the ISS of the rest of the patients. The authors used some sophisticated statistics to test their hypotheses, and they will need to explain how and why they are appropriate for this analysis. Nevertheless, the mortalities in the blood groups number only in the single digits, so I worry about these statistics.

Here are my questions for the authors and presenter:

  • How do you reconcile the significantly higher ISS in the two (very small) groups who got blood? How might this skew your conclusions regarding mortality? Couldn’t the TXA just be superfluous?
  • How confident are you with the statistical analysis? Could the results be a sampling error given that red cells were given to only 7% of the overall study group?
  • I am having a difficult time understanding the conclusion that mortality was reduced in the blood groups. Specifically, it is stated that 24-hour mortality is reduced by 47% in the blood-only group.  But the mortality is 14% (5 patients)! Reduced 47% from what? I don’t see any other numbers to compare with in the table. Confusing!

Obviously, there must be more information that was not listed in the abstract. Can’t wait to see it!

Reference: PREHOSPITAL SYNERGY: TRANEXAMIC ACID AND BLOOD TRANSFUSION IN PATIENTS AT RISK FOR HEMORRHAGE. EAST 35th ASA, oral abstract #39.

 

 

Reference: PREHOSPITAL SYNERGY: TRANEXAMIC ACID AND BLOOD TRANSFUSION IN PATIENTS AT RISK FOR HEMORRHAGE. EAST 35th ASA, oral abstract #39.

Best Of EAST #15: Prehospital TXA

The world is divided into trauma centers that are TXA believers and those that are TXA nonbelievers. It all depends on how one interprets the CRASH-2 data and subsequent studies. Then came CRASH-3 with TXA use for patients with TBI. This large study found improved survival in patients with mild to moderate head injury when given “early.”

The group at Oregon Health Science University tried to better define this concept of “early.” They examined early vs later administration of TXA in patients with moderate to severe TBI. Note that this degree of head injury is a bit different than CRASH-3 (mild to moderate in CRASH-3 and moderate to severe in this one). This was a multicenter trial that included patients with GCS < 12 and who were hypotensive with SBP < 90. Patients received either a 1g bolus followed by a 1g infusion over 8 hours, or a 2 g bolus only. The authors subdivided these patients into early administration (<45 minutes after injury) or late (45 minutes to 2 hours after injury).

Here are the factoids:

  • There were 354 patients in the early administration group and 259 in the late group
  • All outcomes, including 1 month and 6 month mortality and the extended Glasgow outcome scale were not significantly different between early and late groups (exact numbers were not given)
  • There was no difference in secondary complications between the groups (again, exact numbers or complication types were not given)

The authors concluded that there was no difference in outcomes in early vs later administration of TXA in these head injured patients. They suggest that patients can be given TXA anytime within two hours without loss of benefit.

Bottom line: Essentially, this ends up as a noninferiority study. The biggest question with this type of study is, do you have enough subjects to detect a significant difference? Taken to an extreme, let’s say you have 5 patients who receive a drug who are compared to 5 who did not for some mystery condition. Three who did not get the drug die (60% mortality), but only two who get it do (40% mortality). In relative terms, there was 33% decrease in mortality with the drug. But in absolute terms, it was one patient. Would anyone see this as a significant result with such small numbers?

But now multiply by a thousand, and 300 die without the drug and only 200 die who were given it. The relative difference is the same, but the absolute difference is beginning to look large and significant.

So the smaller study won’t meet the test of significance but the larger one will. The key question in the TXA study here is, do they have enough patients enrolled to show there is no real difference between the groups? I love doing back of the napkin power analyses, and I admit I certainly don’t have all the numbers and probabilities needed for a precise calculation. But the groups sizes in this study (354 vs 259) seem a bit small to achieve significance unless there are large disparities in outcomes. 

I certainly recognize that it’s just not possible to put all the relevant information for a research project into a four paragraph abstract. One would need to be able to submit 12 slide PowerPoint decks. So I’m sure more info will be available as I take in the presentation next Friday.

Here are my questions for the authors and presenter:

  • The study is nicely designed as a randomized, double-blind trial, but how did you blind one vs two doses? Did everyone get an infusion of something, TXA vs saline?
  • Why did you select 45 minutes as the cutoff for early vs late administration? Was this arbitrary or is it based some data?
  • Show us the power analysis that demonstrates the total number of patients in the study is sufficient to show us true non-significance in your results.
  • And I’m sure you will show the actual survival and complications numbers (and type) in the presentation, since they were not available in the abstract.

Reference: THE EFFECTS OF TIMING OF PREHOSPITAL TRANEXAMIC ACID ON OUTCOMES AFTER TRAUMATIC BRAIN INJURY. EAST 35th ASA, oral abstract #40.

Best Of EAST #14: Trauma Center Access

The trauma group at MetroHealth in Cleveland has previously published a paper that examined the impact of Level I trauma centers in close proximity on their surrounding population. They have expanded this work to look at changes in the number of trauma centers of any level over a five year period and the populations that they serve across the US. The group was interested in elucidating the number of centers that opened in previously unserved areas, and the whether these areas were economically disadvantaged.

They used a list of state designated trauma centers maintained by the American Trauma Society. Level I and II centers were grouped together, as were Levels III through V. Census tracts around centers were categorized as “served” if the population surrounding it was within a 30 minute drive time of the center.

Here are the factoids:

  • The number of trauma centers increased by 256 to a total of 2140 in 2019, and 82% of these were Levels III-V
  • Nationwide coverage in terms of census tracts served increased from 75% to 80%
  • The increase in total population served was similar, rising from 76% to 79%
  • 91% of new Level I-II centers were in areas that were already served by other high level centers, and 86% of new Level III-V centers were in already served areas
  • New Level III-V centers were opened in areas with higher poverty than Level I-II centers (16% vs 13%)

The authors concluded that the numbers of trauma centers is increasing over time, but that more Level III-V centers are moving into underserved areas.

Bottom line: The authors have identified a novel way to suggest the financial motivations of opening trauma centers. When trauma systems were first implemented, there was an overall goal to provide coverage for the general population. But only a few states wrote guidelines that would attempt to evenly and equitably distribute new centers within and across counties.

The American College of Surgeons wrote a white paper and created a tool to assist in determining how many trauma centers were needed to serve a given population. Unfortunately, implementation of the tool was left to the states, and their legislatures had little interest in adding it to their system regulations after the fact.

So in some states, it’s like the wild, wild west with new centers opening almost next door to established and storied trauma hospitals. This abstract demonstrates that this phenomenon is real. But unfortunately, Pandora’s box was opened long ago and I don’t see that anything will change to address this situation in the foreseeable future.

Here are my questions for the authors and presenter:

  • Are the trends you identified general ones across the US, or are they focused in particular states?
  • Do you have any information on the impact of this trend on already existing trauma centers?
  • Can you speculate about what can be done to ameliorate this trend going forward?

This is a fascinating abstract about a non-clinical issue that has major implications for existing trauma programs (and especially certain states) well into the future.

Reference: A POPULATION-BASED ANALYSIS OF TRAUMA ACCESS: DO NEW TRAUMA CENTERS PROVIDE NEEDED OR REDUNDANT ACCESS? EAST 35th ASA, oral abstract #8.

Best Of EAST #13: Whole Blood And Hypocalcemia

Hypocalcemia has long been known to exacerbate coagulopathy. Calcium is involved at several points in the coagulation cascade. Once serum levels drop below about 0.25 mmol/L (normal value 1.2-1.4 mmol/L) thrombin generation and clot formation cease. Although levels this low are probably rare, anything between this low and the normal level can significantly lower clot strength.

Trauma patients are more likely to have bleeding issues than most, and trauma professionals do their best to avoid coagulopathy. Unfortunately, the products we use to replace shed blood are preserved with citrate, which binds calcium. Given in even modest to large quantities, transfusion itself can lead to hypocalcemia.

Most blood transfused in the US has been broken down into separate components (packed cells (PRBC), plasma, platelets) and the effect on calcium levels is well known. The trauma group at Oregon Health Sciences University studied the impact on calcium of whole blood transfusions.

They performed a retrospective review of data collected prospectively over a 2.5 year period on patients receiving whole blood. This included the number of transfusions, ionized calcium levels, and calcium replacements administered. Patients were divided into two groups, those who received whole blood only and those who were given whole blood and component therapy. Outcomes evaluated were ionized calcium levels, hypocalcemia correction, and death.

Here are the factoids:

  • During the study period, 335 patients received whole blood, but only 67% met inclusion criteria
  • About half (103) received a median of 2 units of whole blood (only!)
  • The authors do not state how many component units the whole blood plus component therapy group received
  • There was no difference in calcium levels based on average ISS in the two groups, although ISS does not differentiate injuries that bleed very well
  • Hypocalcemia occurred in only 4% of whole blood patients vs 15% of whole blood + components, which was significant
  • Hypocalcemia within the first hour was significantly associated with death in the first 24 hours and 30 days, although the standard deviation or SEM of this value was large
  • Whole blood only patients received less calcium replacement, and failure to correct was associated with 24 hour mortality
  • Median time to death in patients that “failed to correct” was 7.5 hours after admission

The authors conclude that hypocalcemia rarely occurs in whole blood only resuscitation, and that adding components increases its incidence and overall mortality. They state that aggressive calcium supplementation should be prioritized if component therapy is used.

Bottom line: There’s a lot to “unpack” here! Packed red cells are preserved with 3g of citrate per unit, whereas whole blood units contain only half that amount (1.66g to be exact). One would expect that one unit of packed cells would have twice the anticoagulant effect as a unit of whole blood.

This study is a blended model, where every patient got some whole blood, but some got components as well. Why? Is there a blood refrigerator in the ED stocked with whole blood, and when it is exhausted there is a switch to components? This model makes it more difficult to tease out the impact of the components given. Perhaps it could be done by matching patients with a given amount of whole blood. That is, comparing patients with 3 whole blood with those who received 3 whole blood + 2 PRBC.

There was no room in the abstract to explain why one third of patients were excluded from the study. This needs to be provided to ensure that the remaining two thirds are representative and can legitimately be analyzed. 

The number of units of whole blood per patient was low, with a median of two units given. Is it surprising that these patients did better than ones who received many more? Remember, from a citrate anticoagulant perspective, hanging two units of whole blood is the same as giving just one unit of PRBC.

This abstract raises a lot of questions, and the most important ones deal with how it was designed and the exact numbers of product given. Only then can we be confident that the rest of the associations described are significant.

Here are my questions for the authors and presenter:

  • Why did you choose the whole blood vs whole blood + components for your study? Wouldn’t it have been cleaner to do whole blood only vs components only? Perhaps all of your patients get whole blood? It seems like this might make the results more difficult to tease out.
  • How is whole blood made available for your trauma patients, and did this have an impact on your study? Do you have a limited number beyond which component therapy is used?
  • What were the inclusion criteria? These were not stated in the abstract, but a third of patients were excluded from the study based on them.
  • Could excluding a third of patients have skewed your results, and how?
  • How many component units were given along with the whole blood in the combination group? This was not provided in the abstract and will have a major impact on outcomes if the median total product numbers are significantly higher.
  • What does “failed to correct” mean? Were the patients not responding to large amounts of administered calcium, or were they not receiving large amounts of it?

I am very interested in the fine details in this abstract and will be listening intently to the presentation!

Reference: WHOLE BLOOD RESUSCITATION IN TRAUMA REGULATES CALCIUM HOMEOSTASIS AND MINIMIZES SEVERE HYPOCALCEMIA SEEN WITH COMPONENT THERAPY. EAST 35th ASA, oral abstract #6.