Using NHSN C. difficile Infection Rates? Mind your denominator!

Over here in the US hinterland we're completing a systematic review of MDRO outcomes for CDC in cooperation with investigators in Salt Lake City. At the moment we're tackling C. difficile and are busily pouring through the literature. We've come across many good studies, such as an ICHE paper from early 2013 by Gase and colleagues from the New York State Dept. of Health that compared NY State CDI surveillance to NHSN in 30 hospitals. The authors noted an 80% agreement between the methods and thus recommended that NY State adopt the NHSN LabID method because of ease of implementation.

Building on that study, Haley and colleagues also from the NY State Dept of Health completed an analysis of the sources of bias in NHSN "Hospital Onset" CDI rate calculations using data from 124 NY hospitals. Their findings were published in the January 2014 issue of ICHE and were accompanied by a nice editorial by two of my former Maryland colleagues Jessina McGregor and Anthony Harris. The NY authors looked at how auditing, including outside labs, age adjustment and exclusion of "patient days not at risk in the denominator" would improve the calculation of hospital-onset CDI rates. As you can see by the portion of Table 2 that I pasted below, most of the corrections had minimal impact on the average hospital-onset CDI rates.  However, "exclusion of patient-days not at risk" had a huge impact on the calculated HO-CDI rate. The correct rate after controlling for all factors was 11.6/10,000 patient days; however, excluding auditing or outside labs, or age adjustment had minimal impact, whereas not excluding patient days not at risk from the denominator led to a rate that was 45% lower (6.4/10,000 pt-days).

The reason that eliminating "patient-days not at risk" from the denominator had such a huge impact is that the CDC NHS definition excludes CDI cases that occur in the first three days from the numerator but does not exclude patient-stays less than three days from the denominator. For example, a patient that stays only two days would not be at risk from contributing a HO-CDI case to the numerator but contributes their patient-days to the denominator.

This has several important implications.  One, not removing the patient days not at risk results in reported CDI rates that were much lower than they actually are. This occurs since many if not most patients have stays that are shorter than 4 days.  Second, as the authors state, "HO-CDI rates at hospitals with shorter LOS are biased downward more than the rates at hospitals with longer LOS." It seems to me that this artificially hurts the rates at tertiary-care and academic medical centers more than it would smaller community hospitals. We always hear how academic hospitals are falling behind, but it may have something to do with how rates are calculated, especially if we are including the wrong patient-days in the denominator.  It seems like this would be an easy fix - hospitals could just exclude the first three days from their patient-day calculations.  I hope this happens.

Failure of Freakonomics

Andrew Gelman (Columbia) and Kaiser Fung (NYU) have an interesting article in the Jan/Feb 2012 issue of American Scientist that is worth a read. They review the popular Freakonomics franchise of Steven Levitt and Stephen Dubner. Freakonomics and SuperFreakonomics have set the standard for the popular statistics/economics genre, from which Gelman and Fung have both benefited. (click on their names to see their books)

However, Gelman and Fung have identified a "tendency in the Freakonomics body of work to present speculative or even erroneous claims with an air of certainty."  Overall, I'm not so worried about the small errors they outline, but the reasons for the errors are concerning and the solutions are important ones to consider in any scientific discipline, including healthcare epidemiology.

One major problem they identify is Levitt and Dubner's reliance on linear informal social networks. For example, in the original Freakonomics, the network was "Levitt did the research, Dubner trusted Levitt, the Times trusted Dubner." However, as time pressures built and the need for more unique stories increased in SuperFreakonomics the network devolved into "Levitt trusts brilliant stars such as Myhrvold or Oster, Dubner trusts Levitt, and we the readers trust the Freakonomics brand."

The solution offered was that they should "leave friendship at the door."  I think this is something all scientific disciplines could benefit from.  It is clear that editorial boards, grant review committees and annual meeting planning committees are all at risk from reliance on a "linear" closed social network (in the past called "old boys' network"). They suggest that building more "non-linearity" into their research and evaluation would protect the process from what I might call a "friendship" bias.  Excellent advice, perhaps difficult to put into practice, but worth the effort.

link: American Scientist Jan/Feb 2012

Antibiotic exposure a risk for pediatric MRSA

There is a nice case-control study in this month's Archives of Pediatric and Adolescent Medicine by Schneider-Lindner et al. in Quebec. Cases were kids 1-19 yo with outpatient MRSA and controls were matched on age and practice. The primary exposure was receipt of antibiotics between 30 and 180 days prior to MRSA infection. In the abstract, they even said they excluded antibiotic prescriptions within 30-days of MRSA infection to prevent protopathic bias. How cool is that! They said protopathic bias in the abstract! Oh, protopathic bias occurs when a treatment is unknowingly prescribed for an early manifestation of the disease, which has not yet been diagnosed.

They found that 53% of cases and 14% of controls had prior antibiotic exposure. So, 47% of MRSA cases in children occurred without recent antibiotic exposure. After multivariable analysis, the adjusted OR for exposure to any antibacterial was 3.5 (95% CI 2.6-4.8). As an example of a nice dose-response curve, the ORs increased with the number of prescriptions with a OR of 2.2, 3.3, 11.0 and 18.2 for 1, 2, 3, and more than 4 prescriptions, respectively.

I was going to write some additional comments in this space and even thought about writing something obnoxious like "Antibiotics Cause Resistance! Thank you Captain Obvious" But then I came across these nice comments in Reuters Health by honorary Canadian and famous co-blogger Dan, which I will post below.

• "This just provides more evidence to support redoubling our efforts to decrease antibiotic use," Dr. Daniel J. Diekema, who was not involved in the new work, told Reuters Health.

• "It (MRSA) remains a major public health problem, but the dramatic increase that we saw during the last decade seems to have leveled off in many areas and may be decreasing in some," said Diekema, an expert in infectious diseases at the University of Iowa in Iowa City.

• "In general, only a minority of people who carry MRSA go on to become infected," Diekema explained.

• "Observational studies like this really can't prove causality," said Diekema. But he added that it was biologically plausible that antibiotic use would fuel the growth of resistant bacteria.

Schneider-Lindner et al. Arch Pediatr Adolesc Med, August 1, 2011

Reuter's Health, August 11, 2011

Gettin’ “meta” with it!

Talking about bias is so pre-millennial. The kool kids are more interested in metabias. Metabias, by definition, only rears its ugly head when groups of studies are examined. Meta-epidemiological studies (e.g. meta-analyses) can uncover risk factors for bias that don’t seem to be associated with a process active in an individual study. That’s metabias! We’ve blogged about several specific forms of metabias before, including publication bias and citation bias. A new form of metabias is described in this week’s Annals of Internal Medicine: it turns out that single-center trials consistently report larger treatment effects than multicenter trials, even after controlling for sample size and other factors. It’s not clear why this is the case, nor is it possible to determine which study type (single-center or multicenter) gets closer to the “truth” (though I strongly suspect the answer is multicenter). Given that many HAI prevention studies happen to be single center studies, this problem bears further scrutiny…..and a willingness to fund larger (more expensive) multicenter trials.

Accentuate the negative

Dan has posted previously on how difficult it is for authors to get negative studies published.  Perhaps this is the real reason why the STAR*ICU study took 4+ years to make it to press.  I suspect that if the study was completed the exact same way that it was but found a benefit for barrier precautions, it would have appeared in press around 2009 or even earlier.  Just a guess.

Mike has posted at least twice on Ben Goldacre and his blog/book called Bad Science (part 1 and part 2).  Ben has a new post in the Guardian that discusses how medicine, academia and popular culture all favor positive, eye-catching and potentially spurious trial results and ignore important negative studies.  His discussion centers around a paper published last year that seemed to provide evidence of precognition - you know it before it actually happens.  That "positive" paper received tons of press, while a new negative study can't see the light of day.  I think this sort of bias plays a large role in infection prevention research - it is so much easier to publish a positive quasi-experimental study showing a benefit than a negative study.  This is why it was so great that after 4+ years of waiting the STAR*ICU study, which was a negative study, was published at the same time as the VA study, which showed a benefit.  This way, we could have a rational discussion with the positive/negative evidence receiving "almost" equal billing.

Ben Goldacre "Backwards step on looking into the future" Guardian 4/23/2011

MRSA active surveillance: More sources of bias

A few days ago, Dan blogged about some sources of bias in the MSRA active surveillance debate--publication bias, citation bias, and editorial bias. To that I would add another form of bias, which I'll term "guideline committee bias." In 2003, the SHEA Guideline for Preventing Nosocomial Transmission of Multidrug Resistant Strains of Staphylococcus aureus and Enterococcus was published. This guideline has been used repeatedly by those outside the infection control community to argue that all hospitals should perform active surveillance for MRSA, a position actively espoused by Drs. Farr and Jarvis, two authors of the guideline. The guideline lists 7 authors, each of whom is a reputable healthcare epidemiologist. However, what is not apparent to many readers is that of the 7 guideline authors, 3 were relatively senior and 4 were relatively junior. All 4 of the relatively junior authors were former trainees of 2 of the senior authors. So it leads me to wonder whether there were 7 independent minds sitting at the table when the guideline was developed. Moreover, one must speculate as to whether the outcome of the process was predetermined and the individuals chosen to make that outcome happen. All trainees are heavily influenced by their mentors--the entire process of mentoring centers around the handing down of a thought process for approaching problems. In my own case, though my professional decision making is now independent of my mentor, I would venture to guess that we agree almost always on our approaches to problems in infection control. The proverbial apple doesn't fall far from the tree.

There's one last form of bias that is worth mentioning and was noted by Wachter et al in an editorial in the Annals of Internal Medicine last year. They criticize a CMS policy that required that the first dose of antibiotics for pneumonia patients be given within 4 hours of arrival to the emergency department. This policy, which was not well thought out, had unintended consequences that ultimately required that the rule be revised. They write:

In addition to financial conflicts of interest, caution should be exercised when individuals are both key investigators and policymakers, particularly when the stakes are high. In the case of TFAD [time to first antibiotic dose], several key researchers had positions with CMS and IDSA and helped drive the conversion of their own studies into performance standards. None of us can be entirely impartial when judging the merits of our own research.

Over five years have elapsed since the SHEA guideline was published. It's time for a new guideline, and SHEA should ensure that the process is open, fair, and includes experts with varying viewpoints on how to control multidrug resistant pathogens.