We can't predict HAI with ICD-9 codes and it's only going to get worse

I'm getting ready for a chat with a reporter concerned with issues surrounding HAI surveillance. During my preparation, I thought again through the issues of code-based algorithms (e.g. ICD-9) and I've come to the conclusion that they are useless for assessing the burden of HAIs and HAI trends and it's only going to get worse.

One area we (and many others) have looked at is the utility of ICD-9 code-based algorithms (ie administrative codes) for detecting HAIs efficiently. A key metric frequently reported by researchers is the sensitivity of a specific code or code algorithm, which is great if the purpose of the algorithm is to improve the efficiency of detection by manual methods. Thus, if the sensitivity is high-enough, you could use the code-based algorithm to reduce the number of charts that require an IP's review. If you are using codes in this way, great!  I have no problems with that.

However, many are now using code-based algorithms to track trends in specific HAIs and measure the burden of disease. My general feeling on these is that they should be completely avoided for several reasons:

1) No matter how sensitive the algorithm is, all we care about here (since we are not validating with manual review) is the positive predictive value (i.e. the proportion of all code-positive patients that actually have the HAI of interest)

2) The PPV is very low for almost all HAI algorithms

3) If we are doing our job and lowering the incidence of HAI per admission in our hospitals the PPV by definition will only get worse (given a fixed sensitivity and specificity)

To show you why I have these concerns I have constructed two 2x2 tables evaluating an excellent hypothetical code-based algorithm for UTI with a sensitivity and specificity set at 95%.  In this first 2x2, I have evaluated the performance of the algorithm when the HAI has a 5% incidence per admission (i.e., 5% of the admissions had a UTI). You can see that such a great algorithm with a high-prevalance of disease, has a poor PPV of 50% - like flipping a coin.

Now, assume we have done an amazing job and cut our HAI rate down to 1%.  Given the same hypothetical algorithm, our PPV is now a horrible 16%. Thus, as we get better at preventing HAIs, we get worse at detecting them using code-based algorithms. Are you comfortable saying UTIs are increasing or decreasing or are associated with a certain level of excess costs, when only 16% of the UTIs in your estimation are actual (true positive) UTIs?  Me neither.

HAI Rates are a Red Herring

"Fictional" Hospital CMO: "Why should I care about hand hygiene or environmental cleaning if I have no CLABSI or CAUTI in my hospital?"

Don't take this the wrong way, since I'd never want a patient to develop a CLABSI or VAP, but I think our focus on device infections is actually harming patients in the long run. If we convince ourselves, like that CMO quoted above or QI and patient safety folks, that we can just prevent device infections (never mind define them away) and everything will be fine, we are missing the bigger picture. The bigger picture is antibiotic resistance and I've yet to see any evidence that our antibiograms are improving.

When did hospital epidemiologists forget we were infectious disease physicians?

In September's ICHE Kerri Thom and colleagues in Maryland published a sobering reminder that resistant pathogens are increasing, particularly Gram-negative pathogens. They (COI alert, I'm a co-author) completed an Acinetobacter baumannii prevalence survey of all mechanically ventilated patients in the state of Maryland. They swabbed intubated patients in 40 of 57 hospitals and collected sputum and/or perianal swabs from  92% of all eligible patients in those hospitals.  What they found was staggering.  Fully 34% of patients were colonized or infected with Acinetobacter baumannii with 16% in acute care settings and an astounding 63% in long-term care settings carrying the pathogen. Even more worryingly, many strains were highly drug resistant with 46% of isolates in long-term care described as "extensively drug resistant," meaning there were no effective therapies - polymixin anyone?

Why does this matter?  Resistant pathogens cause untreatable infections and result in terrible situations like patients being removed from organ transplant waiting lists. These pathogens also carry resistance genes and serve as reservoirs for emerging resistance in other pathogens like E. coli. So, while I'm sure these Maryland hospitals all report zero CLABSI or CAUTI, I guarantee that they all have patients infected and dying of Acinetobacter baumannii and other MDR-Gram negative pathogens. Until we make investments in the science behind hand hygiene improvement, environmental cleaning and other methods for transmission prevention and until we invest in antimicrobial discovery, patients will increasingly die of these untreatable infections.

Luckily, when a kidney transplant patient dies of MDR-Acinetobacter sepsis, the hospital CMO can still sleep at night. At least the patient didn't die of a CLABSI.  I'm sure the patient's family will find comfort in that.


red herring image source: misocrazy

Working the refs

As the Miami Heat and Oklahoma City Thunder prepare to do battle in the NBA finals, we should recognize the importance of home court advantage and the pressure placed on referees by coaches and players. There are data that home teams benefit from fewer fouls, and everyone knows coaches and players who are good at “working the refs” to influence their calls. There are also data that as the pressure mounts (game 7 in a 7 game series), referee influence is even greater.

Now, as a mental exercise, picture your infection preventionists (IPs) in referee outfits. It shouldn’t be surprising that as the pressure mounts for hospitals to eliminate reportable HAIs, the refs (those who “make the calls” as to whether an event meets the NHSN criteria for an HAI) are under increasing pressure. We’ll soon publish survey data revealing how commonly hospitals use “consensus methods” (e.g. adjudication panels that include clinical leaders and/or hospital administrators) or even allow clinicians to “veto” HAI calls. These approaches all drive HAI rates lower, as I know of no hospitals where clinicians or hospital administrators bring cases to the IPs to ask why they failed to report them as HAIs. They are also corrosive of the prevention culture, and contribute to IP burnout.

However, even if all hospitals stopped these practices immediately, the increasing pressure to demonstrate HAI elimination would remain a problem. The celebrations units have when they reach “zero” for a period of time, and the massive disappointment when a single VAP or CLABSI ruins the celebration, are akin to the crowds and coaches during game 7, cheering a call for the home team and booing a call for the opposition. And in those centers where financial rewards and penalties accrue to unit directors based upon HAI rates, the pressure is even greater. 

The answer? Eliminate subjectivity in HAI definitions, and move to objective definitions that are amenable to electronic reporting. These HAI events may no longer correlate well with the infectious disease syndromes we diagnose and treat at the bedside, but as long as they are associated with important adverse outcomes (length of stay, mortality) and are preventable, they should suffice (see VAC vs. VAP).

Postmodern infection prevention

My bad! A few weeks ago I was handing out kudos to APIC for removing all the "getting to zero" propaganda from their website. Was I an idiot or what?! Now zero has become APIC's new vision--Healthcare without infection. All of us would like healthcare without infection. All of us would like automobiles without vehicular accidents, too. But most intelligent human beings who have ever driven a car know that we'll never eliminate automobile accidents no matter how many safety features the auto industry designs. Importantly, we've learned that there are many things we can do to make driving safer. The same holds for healthcare.

I think the zero kool-aid that APIC keeps drinking is really part of post-modernism, the philosophical paradigm that holds there is no absolute truth. Postmodernism is inherently anti-science. David Gorski, a physician who blogs at Science-based Medicine, writes:

"To the post-modernist “scientific medicine is no more valid a construct to describe reality than that of the shaman who invokes incantations and prayers to heal, the homeopath who postulates “healing mechanisms” that blatantly contradict everything we know about multiple areas of science, or reiki practitioners who think they can redirect “life energy” for therapeutic effect. In the postmodernist realm all are equally valid, as there is no solid reason to make distinctions between these competing “narratives” and the “narrative” of scientific or evidence-based medicine."

Stephen Colbert talks about truthiness,"truths that a person claims to know intuitively "from the gut" in that it "feels right" without regard to evidence, logic, intellectual examination, or facts." According to Colbert, “It used to be, everyone was entitled to their own opinion, but not their own facts. But that's not the case anymore. Facts matter not at all. Perception is everything. It's certainty.”

A few years ago, a colleague, also a hospital epidemiologist at an academic medical center, sent me an email that encapsulates the effect of postmoderism on our field. He wrote:

“I used to think that the increased attention on HAIs would be a really good thing, despite the hassles. But I sense that the tide has turned strongly and decisively against the academic, ID-trained hospital epidemiologist. No one defers to that training or expertise anymore–they bow down instead to Toyota models and non-ID trained, self-styled patient safety gurus who preach buzzwords. At my hospital we now report to hospital administration only through someone who spent most of his career in the automotive industry. He's a nice guy and all, but come on, nobody at the table when the “hospital leadership group” discusses infection data has any ID or infection control training!”

If we could be magically transported back a half century with the corpus of evidence about healthcare associated infections that exists today, it might be possible to come close to eliminating HAIs in the hospital of 1962. But advances in medicine continue to make patients more immunosuppressed and devices have become increasingly more invasive. We continue to bypass every one of the body's natural barriers to infection. This is why, as I have said previously, I have yet to meet an infectious diseases physician who believes that HAIs can be eliminated. During my travels I have found that most infection preventionists agree. This leads me to believe that there is some disconnect between APIC leadership and the IPs in the trenches. And it's a good thing I'm not a cynical person! If I were, I might think that APIC's vision would be a great way for the organization to "partner" with industry to prevent infections through some good old fashioned quid pro quo.

I decided when I was a fourth-year medical student that I wanted to be a hospital epidemiologist. I very much wanted to spend my career studying the problem of HAIs and designing ways to reduce them. I still find it fascinating over two decades later. I would love to see the day when there are no HAIs. But I live in the reality-based community that embraces modernism, a place where science is the tool to explain what we observe in the world. So I'll freely admit that I believe in microbiology, epidemiology, vaccines, climate change, and anything else that valid evidence reveals to be true. I also believe that APIC is unfortunately spinning ever further into a parallel, postmodern, anti-science, truthy universe.  

Photos: Institute for Science in Medicine;  BWOG