Category Archives: Philosophy

Why Is So Much Published Research So Bad?

March 15, 2019 The Trauma Pro Leave a comment

Welcome to two days of rants about bad research!

I read lots of trauma-related articles every week. And as I browse through them, I often find studies that leave me wondering how they ever got published. And this is not a new phenomenon. Look at any journal a year ago. Five years ago. Twenty years ago. And even older. The research landscape is littered with their carcasses.

And on a related note, sit down with any serious clinical question in your field you want to answer. Do a deep dive with one of the major search engines and try to get an answer. Or better yet, let the professionals from the Cochrane Library or other organization do it for you. Invariably, you will find hints and pieces of the answer you seek. But never the completely usable solution you desire.

Why is it so hard? Even with tens of thousands of articles being published every year?

Because there is no overarching plan! Individuals are forced to produce research as a condition of their employment. Or to assure career advancement. Or to get into medical school, or a “good” residency. And in the US, Level I trauma centers are required to publish at least 20 papers every three years to maintain their status. So there is tremendous pressure across all disciplines to publish something.

Unfortunately, that something is usually work that is easily conceived and quickly executed. A registry review, or some other type of retrospective study. They are easy to get approval for, take little time to complete and analyze, and have the potential to get published quickly.

But what this “publish or perish” mentality promotes is a random jumble of answers that we didn’t really need and can’t learn a thing from. There is no planning. There is no consideration of what questions we really need to answer. Just a random bunch of thoughts that are easy to get published but never get cited by anyone else.

Bottom line: How do we fix this? Not easily. Give every work a “quality score.” Instead of focusing on the quantity of publications, the “authorities” (tenure committees and the journal editors themselves) need to focus in on their quality. Extra credit should be given to multicenter trial involvement, prospective studies, and other higher quality projects. These will increase the quality score. The actual number of publications should not matter as much as how much high quality work is in progress. Judge the individual or center on their total quality score, not the absolute number of papers they produce. Sure, the sheer number of studies published will decline, but the quality will increase exponentially!

Tomorrow, the big picture view on how to detect bad research.

Philosophy, Pop quiz

Pop Quiz: Do We Really Need To Do All That? The Answer

February 11, 2019 The Trauma Pro Leave a comment

The scenario involved an elderly woman who fell from standing at her care facility 12 hours earlier. They want to send her to your trauma center for evaluation because she seems a bit different from her baseline. You have well defined practice guidelines for patients with head injuries that dictate what type of monitoring and diagnostics they receive.

What do you need to know to determine what you should do? Thanks for all of you who sent in suggestions.

Here are my thoughts:

Which scans should she get? Usually, you would obtain an initial head CT and, due to her age, a cervical CT regardless of her physical exam due to the high miss rate in these patients. But now the fun begins. Your subarachdoid / intraparenchymal hemorrhage (IPH) practice guideline would have you admit for neurologic monitoring for 12 hours, obtain a TBI screen, then discharge without a followup scan if the screen was passed. But in this case, the clock started 12 hours ago and the guideline would be finished with the exception of the TBI screen. So an initial scan and a TBI screen in the ED are all that are needed. The observation period is already over and the patient could potentially be discharged from ED if a SAH or IPH were found.
Your subdural guideline mandates all of the above plus a repeat scan at 12 hours. But once again, the clock has already started. Do you just get an initial scan, which also serves as the 12 hour scan? Or do you get yet another one? If the neuro exam is normal, I vote for the former, and your evaluation is complete after the TBI screen. If the neuro exam is not quite normal, then admission for continuing exams and a repeat scan are in order.
Does the patient need to be admitted, and for how long? Hopefully, you’ve figure this out in the previous bullet. The clock started running when she fell down, so in cases where the physical exam is normal, only the first CT is needed and ongoing monitoring is not. Thus, she could return to her care facility from the ED after the scan.
What other important information do you need to know? Of paramount importance is her DNR status and her/her family’s willingness to have brain surgery if a significant lesion is identified. It is extremely important to know the latter item. If there is never any patient or family intent to proceed to surgery, is there any point to obtaining scans at all? In my opinion, no. The whole reason to obtain the scan and monitor is to potentially “do something.” But if the patient and/or family will not let us “do something,” there is no reason to do any of this. It is crucial that the patient and family understand the typical outcomes from surgery given her age and degree of frailty. This is most important in patients who are impaired with dementia or a high-grade lesion if found from which there is minimal chance of recovery. In most such cases, even if surgery is “successful,” the patient will never recover enough to return to their prior level of care. This should be weighed heavily by the family and care providers.
Should a patient with DNR or “no surgery” orders even be sent to the ED? Theoretically, no. There is no need from the standpoint of their future care. They are not really eligible to have any studies or monitoring done. However, the facility may try to insist for their own liability issues, but this is not really a valid clinical reason.

I hope you enjoyed this little philosophical discussion. Feel free to agree/disagree through your comments or tweets!

Philosophy, Pop quiz

Pop Quiz: Do We Really Need To Do All That?

February 8, 2019 The Trauma Pro Leave a comment

Here are some philosophical musings to keep you thinking over the weekend.

You are the trauma surgeon on duty one evening, and you receive a call from the emergency department. They have received a mildly demented elderly woman who fell at her nursing home 12 hours ago. The staff believes that her mental status is slightly “off” from what it usually is.

Your trauma program has a well-defined practice guideline for elderly TBI care (not on anticoagulants) that involves an initial CT scan, and then a repeat scan after another 12 hours if anything but a simple subarachnoid hemorrhage (SAH) is present. For just SAH, only serial neuro checks are performed for 12 hours and a TBI screen is performed prior to discharge.

Here are my questions for you:

What scan(s) do you need to perform given that 12 hours have already passed since her injury?
Does the patient need to be admitted? For how long?
What other important information do you need to know?
Should the patient have been sent to the ED at all?

I am very interested in your input on these questions. I’ll discuss them in detail in my next post. Please leave comments below, tweet, or email your responses and I’ll see how much we think alike. Or not!

Pet Peeve, Philosophy

First, Read The Paper. THEN THINK ABOUT IT!

September 5, 2018 The Trauma Pro Leave a comment

This is a perfect example of why you cannot just simply read an abstract! And in this case, you can’t just read the paper, either. You’ve got to critically think about it and see if the conclusions are reasonable. And if they are not, then you need to go back and try to figure out why it isn’t.

A study was published a few years ago regarding bleeding after nonoperative management of splenic injury. The authors have been performing an early followup CT within 48 hours of admission for more than 12 years(!). They wrote this paper comparing their recent experience with a time interval before they implemented the practice.

Here are the factoids. Pay attention closely:

773 adult patients were retrospectively studied from 1995 to 2012
Of 157 studied from 1995 to 1999, 83 (53%) were stable and treated nonoperatively. Ten failed, and all the rest underwent repeat CT after 7 days.
After a “sentinel delayed splenic rupture event”, the protocol was revised, and a repeat CT was performed in all patients at 48 hours. Pseudoaneurysm or extravasation initially or after repeat scan prompted a trip to interventional radiology.
Of 616 studied from 2000-2012, after the protocol change, 475 (77%) were stable and treated nonoperatively. Three failed, and it is unclear whether this happened before or after the repeat CT at 48 hours.
22 high risk lesions were found after the first scan, and 29 were found after the repeat. 20% of these were seen in Grade 1 and 2 injuries. All were sent for angiography.
There were 4 complications of angiography (8%), with one requiring splenectomy.
Length of stay decreased from 8 days to 6.

So it sounds like we should be doing repeat CT in all of our nonoperatively managed spleens, right? The failure rate decreased from 12% to less than 1%. Time in the hospital decreased significantly as well.

Wrong! Here are the problems/questions:

Why were so many of their patients considered “unstable” and taken straight to OR (47% and 23%)?
CT sensitivity for detecting high risk lesions in the 1990s was nothing like it is today.
The accepted success rate for nonop management is about 95%, give or take. The 99.4% in this study suggests that some patients ended up going to OR who didn’t really need to, making this number look artificially high.
The authors did not separate pseudoaneurysm from extravasation on CT. And they found them in Grade 1 and 2 injuries, which essentially never fail
472 people got an extra CT scan
4 people (8%) had complications from angiography, which is higher than the oft-cited 2-3%. And one lost his spleen because of it.
Is a 6 day hospital stay reasonable or necessary?

Bottom line: This paper illustrates two things:

If you look at your data without the context of what others have done, you can’t tell if it’s an outlier or not; and
It’s interesting what reflexively reacting to a single adverse event can make us do.

The entire protocol is based on one bad experience at this hospital in 1999. Since then, a substantial number of people have been subjected to additional radiation and the possibility of harm in the interventional suite. How can so many other trauma centers use only a single CT scan and have excellent results?

At Regions Hospital, we see in excess of 100 spleen injuries per year. A small percentage are truly unstable and go immediately to OR. About 97% of the remaining stable patients are successfully managed nonoperatively, and only one or two return annually with delayed bleeding. It is seldom immediately life-threatening, especially if the patient has been informed about clinical signs and symptoms they should be looking for. And our average length of stay is 2-3 days depending on grade.

Never read just the abstract. Take the rest of the manuscript with a grain of salt. And think!

Reference: Delayed hemorrhagic complications in the nonoperative management of blunt splenic trauma: early screening leads to a decrease in failure rate. J Trauma 76(6):1349-1353, 2014.

Philosophy

Why People Don’t Change Their Minds Despite The Data

January 15, 2018 The Trauma Pro Leave a comment

Has this happened to you?

Your (emergency physician / neurosurgeon / orthopaedic surgeon) colleague wants to (get rib detail xrays / administer steroids / wait a few days before doing a femur ORIF). You question it based on your interpretation of the literature. You even provide a stack of papers to them to prove your point. Do they buy it? Even in the presence of randomized, double-blinded, placebo-controlled studies with thousands of patients (good luck finding those)?

The answer is generally NO! Why not? It’s science. It’s objective data. WTF?

Sociologists and psychologists have shown that there is a concept that they call the Backfire Effect. Essentially, once you come to believe something, you do your best to protect it from harm. You become more skeptical of facts that refute your beliefs, and less skeptical of the items that support them. Having one’s beliefs challenged, even with objective and authoritative data, causes us to hold them even more deeply. There are plenty of examples of this in everyday life. The absence of weapons of mass destruction in Iraq. The number of shooters in the JFK assassination. President Obama’s citizenship.

Bottom line: It’s human nature to try to pick apart a scientific article that challenges your biases, looking for every possible fault. It’s the Backfire Effect. Be aware of this built in flaw (protective mechanism?) in our psyche. And always ask yourself, “what if?” Look at the issue through the eyes of someone not familiar with the concepts. If someone challenges your beliefs, welcome it! Be skeptical of both them AND yourself. You might just learn something new!

Home of the Trauma Professional's Blog