SENTRY at 20: So many bugs!

The SENTRY Antimicrobial Surveillance Program was begun at the University of Iowa in 1997, moving a few years later to JMI Laboratories (also in Iowa!). Since inception, it’s been an industry-funded platform that performs central laboratory testing of clinical isolates of bacteria and fungi from centers around the world. The isolate submission process has been consistent over time, involving submission of organisms from consecutive episodes of infection at specified body sites (more information here and in the many publications that have come from SENTRY). 

So SENTRY has been operating for >20 years, and there are hundreds of thousands of isolates characterized. The major trends are reported in an OFID supplement and in recent publications in JAC and AAC. The supplement articles and the AAC report on trends in 20 years of bloodstream infection (BSI) isolates are open access—take a look if interested! 

I’ll focus briefly just on the AAC report (which I first-authored, so that’s shameless self-promotion right there)—two major points, each of which confirms on a large scale what regional surveillance programs have reported:

• S. aureus and E. coli dominate the BSI landscape—together account for >40% of all episodes reported to SENTRY. Continued focus on prevention, detection and treatment of these two bad actors is critical, and IMO should include vaccine approaches, despite the disappointments to date.
• There’s an interesting divergence in proportion of BSI caused by important resistance phenotypes among Gram-positives (MRSA, VRE, DRE, etc.) versus Gram-negatives (ESBL, CRE) over the second decade of surveillance (2005-2016). The Gram-positive resistance phenotypes are stable-to-declining, whereas Gram-negative phenotypes steadily increase (as proportion of BSI episodes) over the entire surveillance period.

The decline in MRSA as a proportion of all SA BSI is particularly striking, occurring as it does at the same time worldwide (and at all body sites, both healthcare- and community-onset, as more detail in the OFID report of all SENTRY S. aureus confirms). As Eli and I discussed in this JAMA editorial almost 10 years ago*, this is not easily explained by hospital-based infection control interventions. The waxing and waning of epidemic clones of MRSA is more likely to be informative. There is so much we still don’t understand about an organism (S. aureus) that lives in relative harmony with 20-30% of the human population when it isn’t causing horrendous, difficult-to-treat infections.

Finally, the scope and number of isolates collected by SENTRY and similar programs represent an underutilized public health resource. Regulatory requirements for drug development and approval mandate surveillance for AMR. Better partnerships between public health authorities and the industry sponsors of such surveillance programs could enhance surveillance and response, particularly in the genomic era when ready access to large isolate collections can be so powerful. Some of this is already happening, but more could be done.

*still behind a paywall after 10 years?  What gives?

Global antibiotic resistance surveillance a.k.a. GLASS: one step closer to saving the world

New surveillance data released a week ago by the World Health Organization (WHO) illuminates a step towards a coordinated and standardized way to perceive the AR problem worldwide. Most of the report focuses on healthcare-related bacteria, but it is not limited to this. Probably a hidden gem includes a standardized characterization of each countries capacity and reporting infrastructure.

In the first report from the WHO's Global Antimicrobial Resistance Surveillance System (GLASS), 22 countries submitted data on 507,746 isolates with antibiotic susceptibility testing results (range 72-167,331 per country). To no surprise, Escherichia coli, Klebsiella pneumoniae, Staphylococcus aureus, Streptococcus pneumoniae, and Salmonella spp are the most commonly reported resistant bacteria; most of these reports do come from sentinel laboratories reported to their country designee. Although some countries report high values of “percent resistance”, often these reflect a single laboratory.

The direction GLASS is going is forward! Which is great. Its strength right now is to provide a one stop shop and platform to quickly read how a country representative describes their AR surveillance effort.

You can choose a country and view data here:Tableau feature of GLASS

This may be helpful for grant writing, collaborations, and consultations; however, for a variety of reasons, the actual results are a bit data-penic, and somewhat of a convenience sample. This is not to fault the GLASS effort at all - countries clearly with the capacity to have insight, visualization, and concrete values on the magnitude of their resistance problem didn’t submit data to this data call.  Although the reasons may vary and all be very valid, it does make the report less of a comprehensive source of global variations in the magnitude of the problem. The power of standardize reporting of “representative” laboratories can (and I believe will) help local health and government leaders prioritize efforts in country. Hopefully additional countries (including the U.S.) will contribute to future data calls. Full report is here:GLASS 2017

Regardless, WHO now provides all of us with an interactive feature (Tableau) to view country specific infrastructures (I counted 37 countries) and data (from 22 countries, although I have not checked all 22).

I don’t envy WHO staff trying to herd cats (cats as in those of us having [or having had] a say in AR surveillance reporting) to try to get all countries to report out data to them in a standard way, but I do applaud their efforts and hope the data submissions become more comprehensive – and thus the interface more usable.

GLASS, which was launched in 2015 to help achieve the goals of the WHOs Global Action Plan on 
Antimicrobial Resistance (AMR) includes establishing country-level surveillance of antibiotic use and resistance. There is a lot of momentum building globally on advancing in-country surveillance and innovation around tracking and transmission interruption. These include the Global AMR collaboration Hub in Germany (focusing on new drug development), SEDRIC (surveillance and epidemiology of drug resistant infections consortium), a new initiative funded by Wellcome Trust aimed to provide technical expertise and knowledge to address barriers with a focus on bringing new technology to bear on big data. In addition the Gates Foundation has contributed in numerous ways, including Grand Challenge Grants

Wrong answer

This morning I stumbled upon this piece, Wrong Answer (free full text here), by Rachel Aviv in The New Yorker. It's an old article from 2014, but a wonderfully written, compelling, sad tale. It's the story of how high stakes standardized testing of middle school students in economically disadvantaged neighborhoods in Atlanta led to cheating by teachers. It focuses on Damany Lewis, a superb teacher totally committed to his students, who tirelessly worked to improve his students' math knowledge and was successful in doing so, but not successful enough to hit an unreachable goal. Responding to increasing pressure to raise testing scores, he and other teachers began to change the answers on students' tests. We all know that cheating is unethical, and at first glance I bet most of us would argue to punish those involved, but read this entire piece (warning: it's long), and you're likely to soften your stance. The consequences of not meeting unreasonable targets were so severe that the teachers felt compelled to cheat in the best interest of their students.

Now take this article from the education setting into the world of healthcare epidemiology, and if you're like me, there will be chills going down your spine as you read it. It should be required reading for anyone who works in healthcare quality or the key stakeholders in this space, from those at the front lines to those who work in professional societies, and to those who create policy at the state or national level. There are also lessons here for patients and patient advocates.

Donald Campbell

What happened in Atlanta shouldn't surprise us. In 1979, Donald Campbell, a psychologist, published a paper, the crux of which has become known as Campbell's law. I wasn't aware of this until I read Aviv's article. It states: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor." Moving this to our world, you can delete the word "social" in Campbell's law and take a look at Dan Sexton's commentary, Casablanca Redux, from 2012. Here's an excerpt:

Our informal discussions with other hospital epidemiologists, our experience in evaluating the source of infection in hundreds of bacteremic intensive care unit (ICU) patients, and common sense have led us to suspect that many hospitals do not accurately report their true rates of CLABSI, using current NHSN definitions. In some cases this may reflect an unwillingness of local staff to accept these definitions as accurate or fair; in other situations it may reflect an unconscious desire to hedge or reduce their rate of CLABSI to avoid criticism and negative consequences from their local supervisors in the press, clinicians, or the general public who review their publicly reported data... If clinicians inappropriately or illogically fear or anticipate negative feedback about the rate of CLABSI in their institutions, they may consciously or subconsciously fail to obtain blood culture results for every patient with a possible or likely BSI. Simply put: no culture equals no infection, using standard definitions of CLABSI. 

At some level, we are all complicit in this. And depending on the action, it may not be the wrong thing to do. In fact, it may benefit the patient. For example, better diagnostic stewardship in the form of appropriately ordering fewer urine cultures not only lowers CAUTI rates but reduces antibiotic utilization with several resultant benefits. Still it's important to note that the primary impetus for this was to lower HAI rates. We take into consideration how a new diagnostic test may impact HAI rates and may even allow that to impact the decision to implement (see an excellent paper by Dan on this here). We may allow clinicians to censor infections that infection preventionists have detected even though the cases meet NHSN definitions. And at the extreme, hospitals may engage in practices that may harm patients in order to reduce publicly reported HAI rates. In a recent publication on how physicians in training view quality initiatives, a dirty secret was elicited from a resident during a focus group at an academic medial center: “There’s like the central line infection protocols…. If you suspect that anybody has any type of bacteremia, you don’t do a blood culture, you just do a urine culture and pull the lines … we just don’t even test for it because the quality improvement then like marks you off.” 

While reading Rachel Aviv's paper, I wondered: Do we ever ignore results (i.e., infection rates) that seem too good to be true like the educational administrators in Atlanta did? Do we critically analyze surprisingly good results to the same degree as we do surprisingly bad results? In Damany Lewis' case did the end justify the means? Is there ever a situation where I could be pushed to a similar point as Lewis?

The Atlanta school system harmed students and teachers in a thoughtless quest to improve quality. There were no winners. Sadly, the response to the cheating scandal was to raise the stakes for test scores even higher. With pay for performance the same is happening in health care.

We need better HAI metrics

Over the past few months, my Chief Medical Officer and I met with Department Chairs to discuss incentive-based quality and safety metrics. In these meetings we agree upon metrics tied to financial incentives for achieving specified targets. The goal is to achieve a win-win--we improve patient care and the department benefits. We typically include at least one metric that is a healthcare associated infection.

With one of our surgical chairs, we were discussing surgical site infection rates. We all agreed that these are excellent metrics on which to focus. The more contentious issue was the target rate to be achieved. We suggested a 10% reduction from the current rate. His counter-argument was that his department's SSI rates were already very low and further reduction may not be achievable. He suggested that national benchmarks be used. That's a great idea, except that we no longer have national benchmarks. NHSN used to provide mean pooled infection rates with percentile scoring using data that were updated at regular intervals. This is no longer the case. Now NHSN provides SIRs (standardized infection ratios) calculated by comparing the observed number of infections to the expected. SIRs could be very helpful in this case, but in reality they aren't since the expected numbers of infection are derived from data collected from 2006 to 2008. So using the SIR, all I could tell the surgical chair is how his current SSI rates compare to other programs nationally a decade ago. If the expected number of infections is derived from data that are a decade old, it defeats the entire purpose of the SIR. For this purpose, the SIRs that NHSN produces are worthless.

Hospitals have two needs with regards to quality metrics: internal trending (are we getting better or worse over time?), and benchmarking (how do we compare to other hospitals?). The SIRs that are currently being produced can be used for internal trending, but not benchmarking. This leaves hospitals completely in the dark vis-a-vis their comparative performance. CDC is in the process of establishing new baselines for expected infection rates. This will be helpful for a year or so, but then the expected data will become old and benchmarking will again be flawed.

There seems to me to be an easy fix: establish two SIRs. The static SIR can use a fixed data set to derive the expected number of infections. This will allow hospitals to be able to internally trend their performance over time. The dynamic SIR would use data from the previous year, updated annually, to allow for comparative performance. This could be easily accomplished.

While we have seen some improvement in NHSN metrics, the overall trend, in my opinion, is that NHSN is moving towards metrics of lesser value (e.g., lab-based automated metrics), and I get the sense that they're not particularly interested in the viewpoint of hospitals. In the value-based reimbursement era, hospitals need valid comparative performance data more than ever, yet CDC appears out of touch and moving in a completely different direction.

CAUTI SCHMAUTI ! (part 3)

I've blogged before about the waste of time, effort and resources being utilized to prevent CAUTI (see here and here), and a new paper in Infection Control and Hospital Epidemiology adds fuel to my fire. This two-year study was performed in the adult ICUs at the Mayo Clinic and analyzed 105 CAUTI episodes. In 97% of cases fever was the primary indication for obtaining the urine culture, but on analysis 2/3 of the patients with CAUTI had alternative diagnoses to explain the fever. Thus it appears that CAUTI is highly over diagnosed. Moreover, preventability is relatively low and secondary bacteremias are uncommon. The authors "question the utility of surveillance for this low-frequency, low-morbidity HAI, which does not serve as a valuable patient-centered outcome." And they conclude: "CAUTIs, as currently defined by NHSN (even with the 2015 definition changes), are not clinically relevant, and efforts to reduce CAUTI may be better directed at other more serious healthcare infections."

  

The paper is accompanied by an excellent editorial by Dan Livorsi and Eli Perencevich. They thoughtfully dissect all the problems with the CAUTI metric and offer some alternatives. They note that it is debatable whether a NHSN-defined CAUTI represents an episode of preventable harm. And they remind us that the opportunity cost is significant.

It's time to end the war on CAUTI.

Graphic: Living with a Catheter

Dollars, denominators, and risk adjustment

Because not everyone who reads this blog reads the comments, I wanted to highlight these particularly insightful observations about Mike’s post on denominators for CLABSI (emphasis mine):

"The thought experiment works with the assumption these two ICUs are indistinguishable except for the frequency of CVC use. Historically, I think the justification for comparative rates using CVC denominators was a no-brainer. These devices were critical to saving lives, and the variations in device utilization probably reflected differences in patient populations, even within similar types of locations. Accounting for the overwhelming primary risk (the CVC) made sense, since these devices were critical to care. The problem you’re outlining now is very real -- as the clinical environment has proven that a lot of the variations in CVC use may in fact be personal preference. Just like the argument with CAUTIs (where foley use is deemed less critical to care) to use a patient-day denominator is strong, we may be at a time where the CLABSI argument is as strong. Improving the classification of ICU types, by more objective criteria than currently used in NHSN (i.e. the 80% rule), would really advance the comparative metric substantially, and likely provide more valid risk adjustment with patient-day denominators than we currently have with these archaic classification schemes (e.g., "med-surg icu"). Advancing the use of composite administrative data to classify patient locations to a more objective, reliable, and granular level, based on fractions of patient-days that have key underlying diagnosis, procedures, etc. is greatly needed."

Given the millions of dollars that are now at stake based upon a hospital’s performance on healthcare-associated infection (HAI) metrics, it’s hard to overemphasize the pressure that is now being placed on the NHSN definitions, and the importance of ensuring that the definitions keep pace with evolving approaches to patient care. When I was a medical resident (yes, way back then), the presence of a CVC was a good indicator of severity of illness and likely served well as built-in risk adjustment for the broad categories of ICU. The same cannot be said now; the device utilization ratios (and percentile ranks compared across NHSN units) vary markedly between different ICU types in our hospital, and do not correlate well with illness severity. And as we’ve learned with CAUTI, the device days that are most amenable to reduction (the “low hanging fruit”) are always the lowest risk device days.

Denominators matter

Let's perform a thought experiment. At St. Eligius Hospital there are two ICUs. These two ICUs have the same number of beds, the same number of patient days (12,000/year), and the same case mix index. In fact, they're essentially identical, except that ICU A has an annual CLABSI rate of 2.7/1,000 central line days and ICU B has a CLABSI rate of 5.0/1,000 central line days. Which ICU is better performing with regards to CLABSI? Well, without any other data to consider, we'd be greatly tempted to conclude that ICU A is the better performer since it's CLABSI rate is nearly one-half that of ICU B. Now, let's add another piece of information: ICU B focused on reducing central line placement as a safety intervention--so at year's end, ICU A had 7,500 central line days and ICU B had 3,000 central line days. This means that ICU A finished the year with 20 CLABSIs, and ICU B had 15. Now it's clear that ICU B is the better performer despite having the higher rate.

This is not just a theoretical problem. During my first rotation on the Infectious Diseases Consultation Service at the University of Iowa last year, I was struck by the low prevalence of central lines in the medical ICU. Turns out my perception was spot on--when I looked at our NHSN data, I saw that 3 of our 5 adult ICUs have central line utilization ratios less than the 15th percentile nationally. This is not an accidental occurrence; clinicians in those ICUs have worked hard to avoid placement of devices that are associated with infection. The problem is that the central lines that do get placed in these units are concentrated in a group of patients that are sicker and more likely to develop CLABSI, since the less sick patients will be managed without a central line. Moreover, the denominator is reduced. And the result is higher CLABSI rates. Here, no good deed goes unpunished.

But there's an easy fix. Instead of using device days as the denominator, use patient days. In our thought experiment, we would see that ICU A would have a CLABSI rate of 1.7/1,000 patient days and ICU B would have a rate of 1.2/1,000 patient days. The better performer (ICU B) will now have the lower rate, as expected. Makes sense, no? CDC should move to address this given the financial penalties hospitals now face based on CLABSI rates. Changing the denominator would provide an incentive for hospitals to aggressively reduce device insertion. And since NHSN has collected patient days for decades, there would be no loss of long-term trending. Lastly, use of patient-days as a denominator produces a patient-centered metric. Think about it: do we really care at what rate catheters become infected? No! Our focus should be on what rate of and how many patients become infected, which is also more intuitive for providers at the sharp edge of patient care.

HAI surveillance definitions update: The good, the bad, and the ugly

Here's an update on NHSN healthcare associated infection (HAI) case definitions.

First, the good:  As of January 1, CDC has modified the definition for catheter associated urinary tract infection (CAUTI). This was sorely needed to improve specificity.

The new CAUTI definition can be found here and a video on the changes can be viewed here. In summary, there are 3 major changes:

• A positive culture requires >100,000 CFUs
• Yeast have been eliminated from the definition
• Urinalysis is no longer part of the definition

The new definition will better align with working clinical definitions used by physicians to diagnose and treat CAUTI.

Now, the bad:  Unfortunately, we still have the problem of CLABSI surveillance only allowing the denominator to include one central line per day even though more than one central line may be present. This punishes academic medical centers where the sickest patients receive care. A recent study in Infection Control and Hospital Epidemiology from the University of Rochester sheds some light on this issue. Investigators there performed a case control study to evaluate the risk of multiple central lines on development of CLABSI. They compared patients with 1 central line to those with more than 1. They found that even when controlling for chemotherapy, hemodialysis, use of TPN, length of stay, age, acute and chronic illness (using APACHE and Charlson indices, respectively), patients with more than 1 central line are 3.4 times more likely to develop CLABSI.

Finally, the ugly:  Although CDC developed a definition for CLAMBI (central line associated mucosal barrier injury bloodstream infection) and hospitals are using it, these infections will still be publicly reported as CLABSIs. The end result is that hospitals with large populations of oncology patients are forced to report falsely elevated CLABSI rates. Since there is agreement that CLAMBI is not preventable and actually not causally associated with central lines, this situation is both ridiculous and harmful.

Over the past several weeks, there have been numerous reports in the media regarding hospitals penalized by CMS in the HAC reduction program. Interestingly, over half of the academic medical centers found themselves in the 25% of hospitals that were financially penalized. It's not surprising given that the cards are clearly stacked against them by the NHSN surveillance methodology. As the stakes get ever higher, the need for more precision in the methodology is imperative.

Gratuitous Hand Hygiene Post: Automated Surveillance Technology

Today, May 5th, is Hand Hygiene Day. If you're looking for links to hand hygiene campaign material, the CDC page is a great place to start. Instead of the usual hand hygiene video or bundle of interventions, I wanted to highlight a recent study our group completed that was just published in AJIC.

The study was led by Melissa Ward, one of Loreen Herwaldt's star research coordinators.  Melissa, spent many months scouring the medical literature for evidence that automated hand hygiene surveillance systems are accurate, effective and cost-effective. After an initial 3,463 article abstracts were identified, she painstakingly reviewed each one to find 42 articles that were original science and evaluated at least one aspect of at least one surveillance technology.  Four types of systems were identified: electronically assisted/enhanced direct observation, video-monitored direct observation, electronic dispenser counters, or automated hand hygiene monitoring systems.

To save you some time, here are the main conclusions from the study: Few articles assessed the accuracy of these electronic monitoring systems and those that did reported "little to no hand hygiene compliance rate differences between direct observation and automated or electronically assisted systems, including electronically enhanced direct observation." However, two studies found differences favoring the automated approach, so we concluded that "more research should be done to validate the accuracy of these systems."

Most importantly there was little evidence that "these systems can improve hand hygiene compliance" and there is currently "limited evidence to recommend the adoption of one type of system or approach as no high-quality quasiexperimental studies, cluster-randomized trials or decision-analytic modeling studies have been completed or published that would allow the comparative effectiveness of system types or individual systems to be assessed."

So what is my gratuitous (second definition: "free") advice: "Facilities should pilot test systems compared to gold-standard, directly observed compliance surveillance before they are widely implemented."

WHO: Antimicrobial Resistance

Last week the WHO released a report covering global surveillance for antimicrobial resistant bacterial pathogens. The report starts off by highlighting the major gaps in knowledge about the magnitude of the MDR-bacterial problem and suggests that the post-antibiotic era is a very real threat. While the levels of resistance in the report are very alarming, the authors also note that worldwide surveillance lacks coordination, so it's likely we're only seeing the very tip of the iceberg.  One interesting aspect of Dr. Fukuda's introduction was his acknowledgment that TB, malaria and HIV have much better surveillance systems and should serve as models for MDR-bacterial surveillance.

The report focuses on "nine" bacteria-drug combinations: E coli vs 3rd gen. cephalosporins and fluoroquinolones,  K. pneumoniae vs 3rd gen. cephalosporins and carbapenems, MRSA, S. pneumoniae vs. penicillin, nontyphoidal Salmonella and Shigella vs. fluoroquinolones and N. gonorrhoea vs. 3rd gen. cephalosporins.

I think the take home point is summed up in the reports Figure 1, which I've pasted below.  No new antibiotic classes since 1987. We can safely say that the bacteria didn't take a 30-year break while we rested on the laurels of the prior generation(s).

The report is 232 pages long, so you have two options if you want to learn more: (1) Head over to the WHO website and read the whole thing or (2) Listen to my 15-minute interview on Iowa Public Radio from last week. Just click on the audio player below or head over to IPR's page and listen there.



For additional reading on the US burden of antimicrobial resistant bacteria:

1) Sievert DM et al. ICHE January 2013 (2009-2010 NHSN Summary)

2) CDC 2013 Antibiotic Resistance Threats Report

Using NHSN C. difficile Infection Rates? Mind your denominator!

Over here in the US hinterland we're completing a systematic review of MDRO outcomes for CDC in cooperation with investigators in Salt Lake City. At the moment we're tackling C. difficile and are busily pouring through the literature. We've come across many good studies, such as an ICHE paper from early 2013 by Gase and colleagues from the New York State Dept. of Health that compared NY State CDI surveillance to NHSN in 30 hospitals. The authors noted an 80% agreement between the methods and thus recommended that NY State adopt the NHSN LabID method because of ease of implementation.

Building on that study, Haley and colleagues also from the NY State Dept of Health completed an analysis of the sources of bias in NHSN "Hospital Onset" CDI rate calculations using data from 124 NY hospitals. Their findings were published in the January 2014 issue of ICHE and were accompanied by a nice editorial by two of my former Maryland colleagues Jessina McGregor and Anthony Harris. The NY authors looked at how auditing, including outside labs, age adjustment and exclusion of "patient days not at risk in the denominator" would improve the calculation of hospital-onset CDI rates. As you can see by the portion of Table 2 that I pasted below, most of the corrections had minimal impact on the average hospital-onset CDI rates.  However, "exclusion of patient-days not at risk" had a huge impact on the calculated HO-CDI rate. The correct rate after controlling for all factors was 11.6/10,000 patient days; however, excluding auditing or outside labs, or age adjustment had minimal impact, whereas not excluding patient days not at risk from the denominator led to a rate that was 45% lower (6.4/10,000 pt-days).

The reason that eliminating "patient-days not at risk" from the denominator had such a huge impact is that the CDC NHS definition excludes CDI cases that occur in the first three days from the numerator but does not exclude patient-stays less than three days from the denominator. For example, a patient that stays only two days would not be at risk from contributing a HO-CDI case to the numerator but contributes their patient-days to the denominator.

This has several important implications.  One, not removing the patient days not at risk results in reported CDI rates that were much lower than they actually are. This occurs since many if not most patients have stays that are shorter than 4 days.  Second, as the authors state, "HO-CDI rates at hospitals with shorter LOS are biased downward more than the rates at hospitals with longer LOS." It seems to me that this artificially hurts the rates at tertiary-care and academic medical centers more than it would smaller community hospitals. We always hear how academic hospitals are falling behind, but it may have something to do with how rates are calculated, especially if we are including the wrong patient-days in the denominator.  It seems like this would be an easy fix - hospitals could just exclude the first three days from their patient-day calculations.  I hope this happens.

One way of Getting to Zero

Favorite quote: "I can disagree with gravity but it doesn't mean that it no longer applies to me."

 

HICPAC lays down the letter of the law

This week's Annals of Internal Medicine has a paper entitled "Public Reporting of Health Care–Associated Surveillance Data: Recommendations From the Healthcare Infection Control Practices Advisory Committee." It's a somewhat misleading title, as the paper isn't about public reporting, it's about what happens in hospitals before HAI data are publicly reported. The primary message is this: if a case meets criteria for HAI sensu stricto, it's an HAI, dammit! Don't go asking a doctor for her clinical opinion on whether it's an infection and then erasing that case from your line list if she says, "doesn't look like an infection to me."

I have mixed feelings about this. On the one hand, in order to allow for valid inter-facility comparisons of infection rates, everyone has to play by the same rules. I think we all get that. But it's incredibly frustrating to review a case that is clearly not an HAI, yet be forced to label it as such. And it's not an uncommon occurrence. Last week, my IPs brought me a case of a patient admitted with pneumonia and because the patient's condition worsened after admission, we were forced to label the case a possible VAP. This morning, an IP from another state emailed me a case of a patient who was admitted with an infected wound, went to the OR, and subsequently developed a surgical site infection, which had to be categorized as an HAI. The IP notes, "the same patient, without the surgery, would have a community acquired wound infection and would not be counted as an HAI."

Even more commonly, hospitals with large oncology populations see many cases of bloodstream infections with enteric flora in patients who just so happen to have a central line. While I give CDC credit for now allowing us to classify cases as mucosal barrier injury related bloodstream infections, it's of little value, as these infections are still publicly reported as CLABSIs. And to add insult to injury, those poor IPs in Pennsylvania have to send patients a letter telling them they suffered an HAI that wasn't an HAI.

All of these problems with post-ascertainment veto and adjudication are occurring because the stakes are high. Most of the time, this is done in good faith, I believe. There is a big push in hospitals to hold staff accountable for adverse events, and it really stings to have the finger pointed at you for an event that was not preventable, or maybe not even an event. It undermines the credibility of IPs and hospital epidemiologists with clinicians when you call a single positive VRE blood culture in a neutropenic leukemic patient a CLABSI. To mitigate that, I find myself appearing at committee meetings to explain that while this case technically meets the criteria for CLABSI, all evidence tells us this is an infection not related to the central line, and is in fact, not preventable. Some hospitals keep two sets of books--the official publicly reported set, and the internally "correct" set.

So while I agree with HICPAC in spirit, this paper only addresses a part of the problem. It ignores the fact that we need definitions with more specificity, and those definitions are needed now. The mucosal barrier injury infection definition has been validated, so let's use it for public reporting. Task IPs to send in descriptions of cases where the definitions are not working, catalog them, categorize them, and start fixing the definitions in a timely manner. Allow for relatively rapid tweaking of definitions instead of acting as if definitions are carved in stone and represent some absolute truth. While we will never be able to have perfect case definitions in the murky world of medicine, fixing the underlying problem to the degree that it can be fixed would decrease the drive for post-ascertainment veto and adjudication. Or how about embracing adjudication but have it occur at a central level? In a world of electronic communication it wouldn't be that difficult.

And one last thing: I think that any official statement from HICPAC should be in the public domain, not behind a journal paywall. Perhaps this paper is also posted somewhere on CDC's website, but I was not able to locate it, if it is indeed there.

Huge increase in hand hygiene compliance to...32%!?!?

There is a lot of pressure on hospitals to improve hand hygiene compliance and it seems even more more pressure from industry to install expensive electronic hand hygiene monitoring systems. This investment might be worth it if (A) the systems accurately measure hand hygiene compliance and (B) are cost effective. One additional benefit claimed regarding these electronic hand hygiene surveillance systems (but not yet proven in the peer-reviewed literature) is that these "surveillance" systems actually improve compliance over the long haul.

Last year at IDWeek, John Boyce and colleagues presented results of a quasi-experimental study analyzed using high-quality statistical methods that showed that installation of an electronic "RTLS-based" monitoring system was associated with a 36% decline in entry compliance rate (p=0.191) and 32% decline in the exit compliance (p < 0.001). These declines in directly observed hand hygiene compliance may have been driven by the fact that the badge accuracy was only 60%. Note: it would be nice to see this study published in the peer-reviewed literature. 

Now there is new report out of the 2013 APIC conference assessing a similar system installed at the John Peter Smith Hospital in Fort Worth, Texas. The authors reported a "huge" increase in compliance from 16.5% to 31.7% over a 3-month study period. After the study period the compliance declined to 25.8%.  Some comments: (1) When your hand hygiene compliance rate is 16.5%, you could do almost anything to improve it. I would imagine having your hospital epidemiologist dress up as a clown while carrying around a bottle of hand rub would do it and doing nothing would probably do it too. That's called regression to the mean; (2) I know a 92% relative increase seems large, but when the high point is 32% compliance, I don't think headline writers and companies should get too excited and claim a "Huge increase." Humility and a call for action should probably take precedence over excitement; (3) A 19% relative decline in compliance from 31.7% to 25.8% after 3 months should give them pause, even if it's above baseline.

To sum up the current evidence, electronic hand hygiene monitoring systems may be more accurate than directly observed compliance, although the jury is still out. However, there is no evidence that these systems effectively or cost-effectively sustain hand hygiene compliance improvements. In fact, based on these two abstracts, the evidence appears just the opposite. There needs to be a lot more research and work put into these systems.

Image source: Uzbekistan Global Hand Washing Day (2012)

Surveillance under pressure

There’s a great success story now published online in ICHE. The CDC, using National Nosocomial Infection Surveillance (NNIS) and National Healthcare Safety Network (NHSN) data, estimates that 100-200K central line associated bloodstream infections (CLABSIs) have been prevented since 1990 through implementation of evidence-based prevention practices. This accomplishment should be celebrated as a demonstration of the real progress that has been made in hospital infection prevention. As Mary Dixon-Woods and our fellow blogger Eli point out in an accompanying editorial, however, these results are also a time to reflect on how much surveillance has changed since 1990.

The CLABSI surveillance that we once performed exclusively to guide local prevention efforts is now used for much different purposes, with rates reported publicly and soon to have a real impact on each hospital’s bottom line. The pressure to bring CLABSI rates to “zero”, by any means necessary, gets passed along from hospital administrators to unit directors and infection prevention programs, turning CLABSI rates into what Mary and Eli correctly describe as a “reactive measure.” To quote their editorial,

“the more that organizations are incentivized by the prospect of shaming or financial penalties to decrease sensitivity—and thus not to find cases—the less certain it is that they are reporting a valid assessment of their infection rate”

It is instructive to examine what happens in other professions when intense pressure is brought to bear on a metric. Five minutes on The Google is enough to inform about what happens when law enforcement is under pressure to lower crime rates, or when teachers are under pressure to improve student test scores. Are police officers and teachers more inclined to “cheat” than are those tasked with counting infections in hospitals? Do officers who misclassify a burglary as a theft after receiving a call from a commander really have nothing in common with the IP program that misclassifies a primary CLABSI as secondary after a call from a unit director or hospital administrator?

This increased pressure is also felt at the CDC and NHSN, as a metric that was initially designed for one purpose is now appropriated for very different purposes. We recently performed an email survey of over 50 prominent hospital epidemiologists to gather their opinions about the direction of surveillance over the next decade. The results can be found here. Some of the highlights:

• Over 75% of those surveyed thought it likely or extremely likely that their local surveillance efforts will erode to focus only on those linked to payment policies or state/federal requirements.
• All thought that HAI surveillance metrics linked to payment policies and state/federal mandates would continue to grow to include more outcome and process measures.
• Respondents felt that pay-for-performance metrics were most likely to drive practice change (moreso than public disclosure of data, use of data by practitioners, or release of national summary statistics).
• Fewer than half thought it was likely that fully automated metrics from existing data elements would replace manual review of records for HAI determination.
• About half of respondents thought that the increased attention to HAI prevention from payment policies, mandates and public reporting has made patients safer (13% thought it hadn’t, and 40% thought the jury was still out).
• Almost 80% of respondents thought that infection prevention experts and clinical providers should have a much larger role in developing and modifying state/federal reporting requirements.

In other words, there is real concern in the HAI prevention community that the increased attention to HAIs, the drive to “zero”, the link to payment policies and public reporting requirements, is a double-edged sword. It has resulted in some tangible successes (CLABSI reductions being a prime example), but threatens to undermine our ability to respond nimbly to emerging local priorities by consuming all of our time and energy, and by producing data that no longer accurately reflect the true rate of adverse outcomes. To quote again from Mary and Eli:

"Undermining our surveillance system to serve ill-designed demands for accountability means that it may no longer be useful for monitoring and driving patient-safety improvements. That would truly be a shame."

Shouldn't evidence guide our selection of hand-hygiene surveillance systems?

It's amazing how little evidence is required before infection prevention interventions are adopted. A current example of this is the installation of automated hand-hygiene surveillance systems that track healthcare worker room entry and hand hygiene compliance. Hospitals are committing significant resources, both financial and person-time, to implement these systems with minimal evidence that they sustainably improve compliance or are accurate and cost-effective. To channel Jerry Maguire, before hospitals "show them the money", shouldn't they ask companies to "show me the meta-analysis"?

With that in mind, I really enjoyed reading the study in February's ICHE by Luke Chen and colleagues at Duke describing the implementation of an electronically-assisted, directly-observed hand hygiene surveillance system. The investigators recognized that directly observed compliance remains the gold standard, but also realized that economic and time costs, along with potential biases such as the Hawthorne effect, limit its utility. Thus, they set out to improve on the gold standard by modifying it to address potential biases and reduce costs.

Beginning in 2009, non-secret (they wore ID badges) auditors began monitoring compliance in 40 wards/clinics. They observed two moments of compliance, before/after room entry. All data was entered into wireless-PDAs that were linked to a centralized server allowing instantly updated 30-day tracking reports.

Overall, the compliance rate reported was 88% after 100,000 observations. That's pretty good. What's more, they reported compliance by the order of observation. For example, they calculated the average compliance for the first opportunity observed, the second opportunity and so on. The reason they did this, is they hypothesized that the Hawthorne effect wouldn't kick-in until the observer was seen by the healthcare worker and that this would become more likely the longer the direct observer remained on the ward. What did they find? Look for yourself:

What do you see? They found compliance for the first five observations was less than 86% while the average over the 35th to 43rd observations was 95%. We see an actual Hawthorne effect. Excellent. After seeing this, and for other reasons as well, they created standard operating procedures that observers used. The most important change to their procedures was they limited observations to 10 minutes or 10 total opportunities before moving to the next ward. Following these changes, compliance was reported to be 86%.

Of course there are limitations to this study in that it was a single-center study and lacked economic data to help guide its broader adoption. The authors acknowledged these and are addressing them already in a future study. My sense is that this method will be more effective and cost-effective than fully-automated systems and also keep infection preventionists visible on the wards to identify and address other important issues, which they can't do from behind their computer screens. But unlike many pushing for a new hand hygiene monitoring system, I'm going to start collecting the data and let the evidence guide my decision making.

Image: Luke Chen, MBBS MPH

Did Google Flu Trends Miss the Mark This Year?

There is a nice review of how various influenza-like illness surveillance systems performed during the 2012-2013 influenza season by Declan Butler at Nature.com. There is some good news and some bad news. The good news is that the timing of the start and peak of the epidemic appear pretty well-aligned between the three methods. However, it appears that the "Google Flu Trends" peak was much higher than the CDC and "Flu Near You" peaks. Reasons for the possible divergence include timing of non-influenza ILIs in the community, a bad norovirus season and excess media coverage including CDC's alert of a bad/early season in early December 2012.

I think these reasons and others given in the article have a lot of face validity and certainly follow what many were feeling as the epidemic progressed. One thing I haven't seen in the coverage is whether the "peak" count of ILI is all that important. I'm more interested in the timing of the epidemic's beginning and the peak and not so interested in an absolute count. I suspect that the absolute number of infections helps in future years, but the timing is more important in the current year. I hope someone will comment or covertly tell me why the peak amount matters so much. It's not like ILI is the greatest definition anyway.

Another assumption I haven't seen discussed is whether Google Flu Trends actually got it wrong or was it the other methods which missed the mark. When you have an imperfect definition, it's hard to call any of these three methods a gold standard. Any attempt to pick a winner seems a bit arbitrary. If the intent is to match the CDC so rates can be compared from year-to-year, that's one thing. But what if we are burdening ourselves with a suboptimal gold standard. A lot of sunk costs go into any legacy system, but it's at least worth wondering occasionally if the legacy is worth continuing.

Image source: http://www.nature.com/news/when-google-got-flu-wrong-1.12413