Are "One-Offs" Becoming Routine

one–off

play

adjective \ˌwən-ˈȯf\

After eight years investigating hospital outbreaks, and about 15 years trying to make the best possible use of surveillance data while at CDC, I still struggle with the tensions inherent in mixing surveillance and performance measurement. The past decade has been a roller-coaster of thrills and perhaps some spills in terms of attention, resources, refinement, and usefulness of HAI surveillance led by CDC; yes, you could probably blame me for several aspects of NHSN reporting you may find unsatisfying (take your pick – perhaps I will expand another day). However, I having recently retired from CDC and am transitioning to Emory Healthcare and Emory University. Although It has been almost eight months. It has been a fascinating transition. The learning curve is steep, and not just for re-entering clinical medicine (that is another story), but also navigating the pathway which integrates the business of healthcare delivery, quality of healthcare delivery, and research opportunities. Slightly easier was learning how to navigate the Emory Parking situation (took 3 months). Much easier was recognizing that the performance quality metrics linked to HAI prevention are getting a lot of attention and a lot of action. It only took a few sessions listening to the quality improvement teams reporting on their target HAIs to understand two things. First, the C suite leaders really care. I had assumed this while at CDC, but it was illuminating to see up close how hard these teams worked to influence HAI prevention. Second, it was becoming somewhat routine to report out on “exceptions to the rules” of HAI reporting. There are many names for those scenarios when an HAI is justifiably reported, but either considered not preventable with evidence based prevention practices or not clinically the infectious event represented by the HAI. While at CDC we routinely heard about these: CLABSIs that “shouldn’t really be counted”, MRSA BSIs that really “weren’t ours”, CAUTIs that really don’t represent an infection. Now these reported HAIs were being called “one-offs.”

The NYT reports the term ”one-off” comes from earlier industrial beginnings with the quantity of items produced in manufacturing process, such as taking one-off, two-off, or twelve-off the line to sample or give-away. However nowadays it can refer to any exception of the rule – such as a recent one-off boxing match that really should not ever have happened. 

In an HAI paradigm where we aim for 0 infections, one-offs may either be unavoidable (not preventable) or wrongly attributed to the device, location, procedure. I have historically known of these in terms of byproduct of using proxy measures. I had previously published an editorial on the value of proxy measures of infection as a tool for quality improvement (Meaningful measure of performance: A foundation built on valid, reproducible findings from surveillance of health care-associated infections).

In that editorial, we outlined necessary steps to reduce the inaccuracies inherent in using such an approach. Now that progress has been made in HAI prevention since 2010/2012, many of these HAI events that conspicuously remain and continue to plague our patients, often don’t fit neatly into the intent of the surveillance definitions. Left with these “one-offs,” it is often difficult to know what to do more to prevent them. Surgical patients with fistulas and central lines that don’t have an infection related to insertion or maintenance processes, neutropenic patients that don’t quite meet the definition of MBI-BSI, I have even heard of tissue transplantation related bacteremia categorized as CLABSI. No doubt, changes have occurred since 2011 to improve CAUTI reporting, and neutropenia-related bacteremia. However, the pace is slow. The one-offs are starting to pile up. Perhaps improved risk adjustment of HAI data will mitigate the influence of the one-offs on healthcare facility performance measures. Until then, kudos to the quality folks and infection control teams making prevention progress. However, I hope we can reward them soon with improved performance measures. Perhaps there are surveillance lessons that can be learned from these one-offs after all. 

If you are interested in sharing one-off stories I have started a registry here - maybe we can fill in some gaps and accelerate the process of changes in surveillance methods.

Don't let authors control for factors in the causal pathway!

A downside of being an epidemiologist who thinks about antibiotic resistance 24-7 is that eventually it's hard to read the literature for fear of seeing another paper where authors make a critical mistake.  It's unfortunate, because the review process should catch these common errors and at least make the authors mention limitations in their discussion sections.  A case in point is a recent paper by Chen et al. in CID that sought to assess the impact of MRSA SCCmec type and vancomycin MIC on treatment failure. In this case, treatment failure was defined as all-cause 30-day mortality, persistent bacteremia, or recurrent bacteremia. They provide nice graphs showing higher mortality for hospital associated-MRSA strains (SCCmec type I, II, III) and for strains with vancomycin MICs > 2. So far, so good. But then, I looked at their multivariable model.

As expected, they controlled for septic shock or severe sepsis in their model. They should not have done this. If it were a comparative effectiveness study assessing various treatment outcomes, it would be appropriate to collect severity of illness including evidence of sepsis before treatment initiation and control for it in the model. However, Chen's study was only looking at outcomes based solely on strain differences. Thus, they should never have controlled for shock or sepsis. How else would a bacteremic patient die if not through sepsis? For a longer description of why controlling for factors in the causal pathway is suboptimal, see my post from 3/19/2012. I have pasted the key sections below. Note: We have been writing letters and review articles in CID pointing this issue out since the year 2000.

____
March 19, 2012: My first ever publication, and in some ways still my favorite, was a letter to the editor of CID that I wrote in 2000 pointing out a common flaw in outcome studies of infectious diseases. In the letter, I discussed a paper that looked at the outcomes (death) associated with methicillin-resistance in patients with S. aureus bacteremia. In the analysis, the authors controlled for septic shock in their regression model. I pointed out that shock is in the causal pathway between infection and death and, therefore, should not be controlled for in regression in models. This would be like controlling for car accidents when looking at the association between cell phone use and death. In infectious diseases, if you remove shock from the causal pathway, it is hard to see how you might otherwise die.

The error of controlling for intermediates is frequently repeated in ID outcome studies when, for example, authors control for illness severity using the APACHE score. If the APACHE is measured after the infection manifests, this variable would be in the causal pathway and should not be controlled for in the regression model. The APACHE should be measured before the infection manifests, as we did here. Jessina McGregor and JJ Furuno (both now at Oregon State) published a nice systematic review on optimal methods for ID outcome studies in CID back in 2007. Wouter Rottier (with Marc Bonten) just published a meta-analysis looking at the impact of confounders and intermediates (factors in the causal pathway) on ESBL-bacteremia outcomes. (JAC, March 5, 2012) I highly recommend that you read these studies prior to undertaking an ID outcome study.

If you're criticizing methods, your methods better be strong

This morning I innocently set out to review a new study in ARIC that looks at methodological issues in C. difficile outcomes studies assessing the association between infection and excess hospital stay.  The authors make a very important point in the paper: "The studies did not collect data concerning the time of onset of CDI; therefore, it is not possible to exclude the possibility of reverse causality, in which longer lengths of hospitalisation may have increased the risk of CDI." They then go on to discuss time-dependent bias and recommend the use of multi-state models...we've made this point before.

What caused me the emotional anguish was their Table 1 that lists the 16 studies included in the review. In column two, three of the studies are listed as retrospective case-control studies...what??!!  How can you do a case-control study with an outcome being hospital length of stay?  I'm pretty sure you can't, at least not easily.

You see, a case-control study requires identifying an outcome and looking back for risk-factors associated with the outcome. For example, you could look at 50 people that died (and 100 that didn't die) and see how many had CDI, to determine if CDI was a risk-factor for death.  For length of stay, I can't even make up a good way to do a case-control study.  Would you find patients who stay >14 days and compare them to patients that stay <14 days? The three studies listed MUST have been cohort studies, so why did the authors of the ARIC paper seeking to teach us about proper methods for outcomes studies list them as case-control studies? Perhaps they pasted them from the individual studies' methods sections?

Just to confirm this, I looked at all three papers: (Ananthakrishnan (2008), Bajaj (2010), Pepin 2005).

Pepin: from the Methods  - "We compared mortality and total length of hospital stay among inpatients in whom nosocomial CDAD developed and among control subjects without CDAD."  This is a cohort study - exposed and unexposed to CDAD, looking forward to outcomes.  Just because they incorrectly use the word control, doesn't make this a case-control study. (Strike 1)

Ananthakrishnan: "Our primary case group (C difficile–IBD group) included patients who had a primary diagnosis of C difficile colitis and a secondary diagnosis of either Chrohn's Disease (CD) or Ulcerative Colitis (UC). Patients admitted to hospital with a primary diagnosis of CD or UC without a diagnosis of C. difficile colitis formed one of our comparison groups (IBD group)." Another cohort study - defined by the exposure to C. difficile and not by the outcome. (Strike 2)

Bajaj: "Among the cohort of hospitalized patients with any diagnosis of cirrhosis, co-existing diagnosis of C. difficile was associated with significantly greater in-hospital mortality."(Strike 3)

The error of incorrectly describing cohort studies as case-control studies is very common - enough to be a pandemic.  The journal CID is one of the worst offenders. One of my favorite examples is this recent VAP treatment cohort study described as a case-control study in the title!  When I've mentioned the problem to CID editors, they suggest I write a letter.  You can't write letters for each issue.

OK: Cohort - exposed/unexposed look forward in time to the outcome. Case-control - find outcomes and look back for exposures. Got it?  EOR

Optimal Epidemiological Methods for Infection Prevention Studies

I've spent the last 12+ years writing about epidemiological methods for conducting risk-factor, outcomes and intervention studies of hospital-acquired infections. Instead of always reviewing the latest and greatest studies, I thought it might be fun to look back at some of the epi-methods papers that many of my colleagues and I have published since 2000. I think we've made some important contributions to infection prevention research, so it's kinda fun to look back at these. Over the next month or so, I hope to review other epi-methods topics that I think are particularly relevant to the study of hospital-acquired infections.

A decade ago, Anthony Harris and Yehuda Carmeli (and other folks) outlined optimal control-group selection in risk-factor studies for antibiotic resistant infections. (see here, here and here) Prior to these important studies, authors would frequently use patients infected with the susceptible organism as the control group. For example, when looking at risks for MRSA they would select MSSA controls, which is incorrect.  Unfortunately, many authors still select the wrong control group and unknowingly publish conditional odds-ratios.  I will discuss this more in a later post.

My first ever publication, and in some ways still my favorite, was a letter to the editor of CID that I wrote in 2000 pointing out a common flaw in outcome studies of infectious diseases. In the letter, I discussed a paper that looked at the outcomes (death) associated with methicillin-resistance in patients with S. aureus bacteremia. In the analysis, the authors controlled for septic shock in their regression model. I pointed out that shock is in the causal pathway between infection and death and, therefore, should not be controlled for in regression in models. This would be like controlling for car accidents when looking at the association between cell phone use and death. In infectious diseases, if you remove shock from the causal pathway, it is hard to see how you might otherwise die.

The error of controlling for intermediates is frequently repeated in ID outcome studies when, for example, authors control for illness severity using the APACHE score. If the APACHE is measured after the infection manifests, this variable would be in the causal pathway and should not be controlled for in the regression model. The APACHE should be measured before the infection manifests, as we did here. Jessina McGregor and JJ Furuno (both now at Oregon State) published a nice systematic review on optimal methods for ID outcome studies in CID back in 2007. Wouter Rottier (with Marc Bonten) just published a meta-analysis looking at the impact of confounders and intermediates (factors in the causal pathway) on ESBL-bacteremia outcomes. (JAC, March 5, 2012) I highly recommend that you read these studies prior to undertaking an ID outcome study.

Anthony Harris and I have also written extensively on the appropriate use and analysis of quasi-experimental studies looking at interventions to prevent hospital-acquired infections. In a trilogy of CID review articles, we reviewed the optimal quasi-experimental designs (2004), the frequency of each design's use (2005) and appropriate statistical analysis of time-series data (2007). If you're planning on doing a non-randomized study of any infection prevention intervention, please look these papers over. Following the optimal methods outlined in these reviews will improve your studies and also increase the chances that your intervention study's results will make the grade and be included in future systematic reviews, such as Cochrane reviews.

Image Reference: DA Grimes, Lancet 2002;359:57-61