Upon Further Review: Reexamining the Illinois MRSA Active Surveillance Mandate

“Thinking without the positing of categories and concepts in general would be as impossible as breathing in a vacuum” – Albert Einstein (1949)

A couple weeks ago, Scott highlighted a study in CID by Lin et al. that sought to estimate the benefits of the 2007 Illinois state-wide mandate of MRSA active surveillance cultures in ICU settings. The post was titled “Good Intentions Does not Always Mean Good Policy” and concluded “There may be many reasons the hospitals in Illinois overall are seeing an estimated 30% decrease in their hospital-onset MRSA BSI (as most states are) since the 2010 NHSN baseline, but admission screening isn’t one of them.” 

I would like to list several reasons why I think we should reconsider the study authors’ conclusions. And if you skip to the end, you will read why I think this study might make more valid conclusions about the lack of benefits of CHG bathing.

Let’s start with validity from Shadish (2001) et al: “We use the term validity to refer to the proximate truth of an inference. When we say something is valid, we make a judgement about the extent to which relevant evidence supports that inference as being true of correct.” Cook and Campbell (1979) outlined four components of validity: statistical conclusion validity, internal validity, construct validity and external validity. I’ve written about their validity typology here, if you’re interested. 

Now let’s review the Illinois study methods. They included data from 25 ICUs and completed eight, one-day point prevalence surveys AFTER the mandate was initiated (twice annually 2008-2011 and annually in 2012 and 2013). There was no concurrent control group.

Thus, this quasi-experimental study design has very low internal validity. It has no measurement before the intervention (sometimes called historical controls) and no concurrent controls. Shadish labeled this design as a “one-group post-test only design” and summarized its limitations with “this design is rarely useful.”

The Illinois study also lacks statistical validity, since it is underpowered to detect a benefit of active surveillance. Shadish lists low statistical power as the first threat to statistical validity since “the experiment may incorrectly conclude that the relationship between treatment and outcome is not significant.” Since the Illinois study is a negative one – claiming active surveillance for MRSA didn’t work, power is particularly important. If you jump ahead to my ICAAC abstract, you will begin to see why the study is likely very underpowered. Point prevalence is very insensitive to changes in acquisition or transmission, so you would need very large studies to see a benefit.

I’ll admit that a statewide study has strong external validity – that is generalizability. 

But let’s focus on construct validity. Construct validity - what Einstein was hinting at in the quote above - describes whether a study measures what it claims to be measuring. For example, if a study claims that active surveillance for MRSA and isolation doesn’t prevent MRSA transmission, that study better measure MRSA transmission. 

Let’s pause here. Now some might say, we don’t care about transmission, we just care about MRSA prevalence or MRSA infections or MRSA CLABSI (or even deaths in Avengers movies). Yet, active surveillance for MRSA doesn’t work like that – it prevents transmission between patients. 

But what if measuring MRSA point prevalence was good enough at detecting MRSA transmission? Thought experiment: what if we found PCN allergy alerts in the EMR annoying and so we eliminated them? To see if this was safe, we then checked to see if anyone was having a PCN allergic reaction every St Patrick’s Day for the next 5 years. Good enough, right? Probably not. Why would we be comfortable saying yearly point prevalence is an adequate way of measuring the benefits of MRSA transmission? I don’t think we should and here’s why:

Back in 2002 I presented an abstract at ICAAC titled: “Point Prevalence and Clinical Culture Positivity of Vancomycin Resistant Enterococci are Poor Estimates of Infection Control Intervention Impact.” This study was based on the VRE model that we eventually published in CID (2004). Anyway, we modeled VRE transmission in the ICU under a condition where active surveillance compliance on admission increased from 60% to 100% and I assumed that isolation prevented 71% of transmissions. This is a math model, so we know that the intervention worked– but could we detect it? The answer was yes but only if we used admission/discharge screening cultures.  If we used point prevalence, like the Illinois study, we would falsely claim that the intervention didn’t work 54% of the time. However, if we used admission/discharge cultures, we would correctly determine that the intervention worked 96% of the time. Here is our conclusion:

“Point prevalence or clinical culture positivity often failed to detect a benefit due to stochastic fluctuations in prevalence and high prevalence of VRE in patients entering the ICU. Studies to assess the benefits of active surveillance for VRE should measure new incident cases. Relying on point prevalence or clinical culture positivity to assess the benefits of infection control interventions may underestimate the magnitude of their benefit and may be responsible for a persistent bias against the broader institution of active surveillance. The benefits of active surveillance and other infection control interventions are probably underestimated.”  

Not bad for 16 years ago – replace VRE with MRSA and you can see why the study by Lin et al cannot be used to evaluate the benefits of active surveillance. Our conclusion was partially driven, as we said, by high rates of VRE colonization on admission. What did Lin et al say about their study: “we assessed MRSA prevalence in a region where MRSA is widely endemic both in the community and within healthcare facilities.” Thus, the MRSA situation in Illinois fits closely with what we modeled.

In conclusion – the study by Lin et al should not be used to claim active surveillance was ineffective in Illinois or elsewhere. When drawing inferences, it is important to remember all four components of validity described by Cook and Campbell 40 years ago. We still need to figure out why MRSA has declined by 30% in Illinois and elsewhere. I do wonder why we are so quick to claim CLABSI bundles, CHG bathing (see below) or other interventions have been driving these MRSA reductions and not active surveillance. If I had to guess, it’s something yellow and not the data.

-----

Side note: In this same study, the number of hospitals using CHG bathing in their ICUs went from 5 (20%) to 17 (68%). It appears the authors could use these same data and methods to show that CHG doesn’t prevent MRSA in ICUs settings. Interestingly since CHG bathing works at the level of transmission and also on individual patients’ decolonization, point prevalence data would have higher construct validity for evaluating CHG. The study might even be better powered to detect a benefit of CHG.

Paid Sick Leave: Avoiding Contagious Presenteeism and Preventing Influenza

The US of A is the ONLY industrialized country without nationwide access to paid sick leave. Reread that sentence several times to let it sink in, it's OK, I'll wait for you. Opponents of paid sick leave say that mandatory sick leave increases the cost of labor and hurts job creation. In addition, they suggest  that paid leaving increases shirking behavior and "noncontagious absenteeism". However, there are potential economic benefits for paid leave, particularly in regards to encouraging workers to not work when they've acquired an infectious disease - "contagious presenteeism."

Before wider adoption of mandatory paid sick leave in the US, it is likely that the health and economic value of such mandates will need to be estimated. To that end, economists Stefan Pichler and Nicolas Ziebarth have just published an unreviewed working paper in the National Bureau of Economic Research, that sought to estimate the impact of paid sick leave on labor supply and avoiding presenteeism as manifest through reduced influenza cases. They were able to take advantage of the staggered implementation of several city (San Francisco, Washington DC, Seattle, Philly, Portland and New York City) and state (CT, CA, MA, OR) sick leave mandates in order to determine the reduction in population-level influenza-like illness (ILI) cases using Google Flu data (2003-2015). Other cities, and states not covered by the mandates served as controls. 

When they examined the impact of the mandates using city-level data and difference-in-differences models, they found that gaining access to paid sick leave resulted in a 5.5% reduction in ILI per 100,000 doctors visits. When analyzed using state-level data, they found a 2.5% reduction in ILI after adoption of mandatory sick pay. Importantly, the authors state that "infections rates may further decrease in the medium to long-run when employees have accrued larger amounts of paid sick days." The paper then takes a deep dive into the underlying behavioral mechanisms of these programs (contagious presenteeism and noncontagious absenteeism) and the positive and negative aspects of mandatory sick leave using US and German data, which makes for interesting reading if you have time. 

The findings of this paper suggest an important population-level benefit for mandatory sick leave policies, which suggests the US should consider passing a bill to make such a policy nationwide. In fact, President Obama said this during his 2015 State of the Union Address: “Send me a bill that gives every worker in America the opportunity to earn seven days of paid sick leave. It’s the right thing to do. It’s the right thing to do.” I'm thinking more mandatory sick leave and less influenza would make America a wee bit greater.

Image Source: nyc.gov

Root causes underlying the emergence of influenza vaccine mandates

Those that follow me on twitter or the blog have probably noticed my recent focus on trying to understand the emergence of compulsory influenza vaccination of healthcare workers. Before moving on from this topic, I wanted to share what I've learned in the process.

(1) There doesn't appear to be any estimate of the burden of nosocomial influenza in the US. We know healthcare-associated influenza does occur, but we don't have estimates for the proportion of influenza cases that occur in hospitals. Even if we did know the incidence, we don't have reliable estimates for what proportion is acquired from healthcare workers vs. visitors or family members. It seems like we'd need those numbers before pushing for a mandate.

(2) There is no data supporting the benefits of healthcare worker vaccination in acute care hospital settings. If we look at the CDC systematic review everyone quotes, there were only 4 randomized trials and all 4 were from long-term care settings. If we generously include the observational studies, 3 were from long-term care and only one was from a hospital setting. We are basing acute-care hospital policy on one observational study.

(3) Again from the CDC systematic review, "HCP vaccination rates ranged from 48% to 70% in the intervention arms and 5% to 32% in the control arms." Thus, there is no evidence that raising vaccination above 48% or 70% is beneficial in long-term care settings. Thus, if we have vaccination rates near 50%, do we need a mandate?

The last two things I learned are that none of the above matters. Science is not what is driving the push for mandates, unless you consider the studies showing mandates raise influenza vaccination rates among healthcare workers. Probably didn't need a study to show that.

(4) Yesterday, I wrote a post trying to bias the respondents of a twitter survey in favor of being cared for by a masked unvaccinated healthcare worker over a vaccinated one. As you can see by the results (below), despite my efforts, the large majority want their healthcare worker to be vaccinated. This is critical - despite the science, we just want people to be vaccinated. A huge driver behind influenza vaccine mandates must be this desire. Additionally, it's likely that masks are viewed negatively by patients. Vaccine mandates make folks feel safe and masks don't - very patient centered.

(5) The finally bit that occurred to me is that the reason vaccine mandates exist is because CMS and other governing bodies require hospitals to collect and report influenza vaccine coverage among their workers. There is also a target of 90% coverage that must be met. Thus, we have a QI target that exists despite minimal scientific evidence that it protects patients but we have to meet the target. And the only way to meet such an arbitrary target is through mandates. QED

The Rise of the MIC: Microbiological Industrial Complex

Mike "Alexander" Edmond

Note: This is the post I wanted to write regarding the NIH Clinical Center KPC outbreak last week until I noticed the posts and comments blaming the front line infection prevention staff.

"...we must guard against the acquisition of unwarranted influence, whether sought or unsought, by the military-industrial complex (MIC). The potential for the disastrous rise of misplaced power exists and will persist....As we peer into society's future, we-you and I, and our government-must avoid the impulse to live only for today, plundering, for our own ease and convenience, the precious resources of tomorrow. We cannot mortgage the material assets of our grandchildren without risking the loss also of their political and spiritual heritage." - President Eisenhower's Farewell Address January 17, 1961

In microbiology and clinical medicine, the MIC is the "lowest concentration of an antimicrobial that will inhibit the visible growth of a microorganism after overnight incubation."  I think it's time to recognize a new definition for MIC: the Microbiological Industrial Complex. The MIC encompasses the industry, associated lobbying efforts and government agencies that most benefit from the adoption of expensive and unproven testing and treatment. The MIC has had a tremendous impact on infection prevention practice through economic forces pushing for MRSA active surveillance mandates and perhaps mandatory flu vaccinations of health care workers. This MIC leads to the utilization of expensive (and largely unproven) interventions at great cost both economically and to the well-being of patients.  The more we spend on expensive sequencing, the less we can spend on actual prevention. Hand hygiene might not be sexy, but it does more to prevent the spread of resistant infections than any PCR test.

The latest evidence of the insidious rise of the MIC is the initial discussion surrounding the NIH Clinical Center KPC outbreak. So far, the only paper describing the outbreak covered the miracle of whole-genome sequencing and how it helped halt the outbreak, which it most certainly did not. The outbreak was halted using a grab bag of unproven and expensive interventions including the hiring of 9 hand hygiene "police" that monitored infection control practice 24-7.  Even NIH's Henry Masur speaking today on the Diane Rehm show said that sequencing "didn't conclusively prove" (what caused the outbreak).  Both he and Jule Segre suggested they only stepped up their infection control efforts because of the whole genome sequencing evidence, which is almost certainly not true. They would have used infection control escalation even without expensive testing. (listen to the Diane Rehm show segment here)

To understand the power of the MIC, you don't have to look further than a recent MSNBC report, which noted that the NIH sequencing cost $40,000 and suggested that this technique could spawn a $1 billion industry in the US alone. In discussing the whole genome technique, Dr. Segre was noted to say "When you have patients in your ICU who just paid $100,000 for an organ transplant,"...spending a few thousand dollars to protect them from an outbreak of deadly bacterial infections "doesn't seem like too much to ask."

It seems to me that since there is no evidence that whole genome identified the source of transmission here or elsewhere and even if it did it wouldn't have altered the course of the outbreak, we might better spend our infection control research and clinical dollars elsewhere.  Unfortunately, the MIC has more money and more NIH backing. The NIH has a National Human Genome Research Institute but it doesn't have a "National Infection Prevention Institute", for example.

Almost a year ago, Mike peered through his crystal ball and accurately predicted the future of KPC prevention in the US.  The NIH outbreak and report starts the countdown, and much like MRSA before it, the prevention efforts will be focused on expensive DNA surveillance efforts backed by large industry lobbying efforts and not investments in the research and expansion of basic and simple infection control efforts. It is easy to blame the healthcare worker for not washing their hands and look for a quick scientific panacea (DNA). Sadly, given that there have been only four high-quality hand hygiene improvement studies since 1980, we haven't provided clinicians with the proven tools to improve hand hygiene. If we continue to bow to the pressure of the MIC and avoid the harder tasks of infection prevention, we will be squandering our precious resources of tomorrow (antibiotics), as Eisenhower warned 50 years ago.

Further Reading:
(1) Maryn McKenna: The ‘NIH Superbug’: This Is Happening Every Day
(2) Ed Yong:  Genome detectives unravel spread of stealthy bacteria in a hospital
(3) Dr. Judy Stone: The NIH Superbug Story-A Missing Piece
(4) Mike the Mad Biologist: Some thoughts on the CRE Superbugs

Image source: wikimedia commons

Misguided mandates

Photo: New York Times

The Bangor Daily News is reporting that a bill being considered by the Maine legislature would mandate MRSA screening of high risk patients admitted to hospitals. I won't rehash all the writing we have done on active surveillance for MRSA, but you can see those posts here. So MRSA exceptionalism lives on, and more money will pour into the coffers of companies that produce MRSA test kits.

Take me home, country roads, to the place….

…where hospitals can decide for themselves how to vaccinate their workers against the flu. Sorry, those lyrics don’t really work. Still, West Virginia has decided against a statewide mandate for healthcare worker influenza vaccination.

I’m not sure this is big news, as I am not aware of how many states have decided to step into this issue—I had assumed that most would leave it up to each hospital. Maybe someone can enlighten.

I am blogging about this for two reasons. First, and most importantly, because West Virginia is Mike Edmond’s home state (or at least he went to college and med school in West Virginia). Here is a photo of Mike as a child, before he decided to pursue a career in medicine and was more interested in the banjo. Second, I wanted to report that our hospital is now at a 93% influenza vaccination rate without a mandate. If you recall, last year our shiny new mandate went down in flames after SEIU filed an injunction. So I’m especially proud of the fact that our healthcare workers are stepping up to be vaccinated without being forced to do so by threat of termination.

Ctrl+F

…will bring up the “Find and Replace” tool in Microsoft Word. Illinois legislators, you may want to use this tool today, so you can get yourselves a shiny new law to deal with the latest emerging antimicrobial resistance threat.

Hint: you’ll want to find “MRSA”, and replace with “KPC”. Make sure to go to “search options” and click on “Match Case”, or you’ll mess it all up.

That should do it.

Links:
Illinois screening legislation
IDSA abstract on emergence of KPC-producers in Illinois
Chicago Tribune story

What happens in Vegas...

...stays in Vegas? Always a good catchphrase, although not necessarily a good movie. I do often wonder if we should adopt a slogan in infection prevention such as "What happens in the ICU, stays in the ICU," where the happens part is MRSA or MDR-Acinetobacter acquisition.

It looks like there is a move afoot in Nevada to mandate universal MRSA screening. Sheila Leslie, D-Reno, whose otherwise healthy cousin recently died from MRSA infection at a California hospital, is considering drafting a bill on the subject. A recently posted Las Vegas Sun piece focused on the MRSA surveillance mandate in the VA system and quoted from the usual and also unusual subjects in the never ending debate around MRSA control.

Some of the more interesting quotes from the article:

“You get paid and paid and paid for doing the wrong thing in medicine. You don’t get paid for keeping people well,” said Phillip Longman, author of “Best Care Anywhere: Why VA Health Care Is Better Than Yours.”

“I know the bundled approach works, but I don’t know which component of the bundle is the most essential component of success.” Dr. Rajiv Jain, who leads the VA’s MRSA prevention program.

It appears that Nevada will look to the VA's data and experience in MRSA control. What isn't clear is if states or hospitals or even local health systems should try to apply data from a federal system with >150 acute-care hospitals. I do welcome the VA publishing their results and hope we see some nice VA-based interrupted time series data analyzed soon. When it comes to the results of the VA's MRSA mandate, what happens in the VA shouldn't stay in the VA.

Las Vegas Sun Article

Being Deb Yokoe - Atul Gawande's APIC keynote

I just sat through Atul Gawande's APIC keynote speech. I was expecting to be inspired and amazed. What I wasn't expecting was Deb Yokoe. Dr Gawande spent the first 30 minutes pacing the stage and discussing the day he followed Dr. Yokoe and her team around the Brigham. He described her as a smiling, occasionally giggling with dimples and tennis shoe wearing hospital epidemiologist. It must have been a very busy day as I think he mentioned 7-8 outbreaks she was dealing with at the time including tularemia and a frozen strawberry foodborne outbreak. Having worked with Dr Yokoe as a fellow in a research capacity, I have always appreciated her work and felt she never got enough credit for her contributions. It is very nice to see that changing.

A particularly touching part of the hospital tour with Dr Yokoe was when Gawande described the staff flowing in and out of the ICU rooms and realizing that this was far different than the controlled OR environment and then he realized one of the MRSA isolated patients was his own and that he might have given MRSA to the patient. He got his biggest applause when he quoted Deb as saying "I don't want to be an infection control cop.". Gawande agreed with her and spent some time discussing the importance of team work and getting infection control out of the police business. Well, he has every right to be hopeful. An important effort made at the Brigham that others should consider copying is that infection control is now a rotation for medical students. I suspect it isn't mandatory, but it is a start.

My favorite part of the talk, apart from his discussions of the surgical checklists that were highlighted in his latest book, was when he was asked a question about mandates. He described that in any advance there are early adopters, medium adopters and late adopters. He then added in a class of people called exceptionalists, people who say they are different or their hospitals are too different to do things like clabsi checklists. His point was that there is a time when mandates are needed but the key variable is the timing. One must wait to mandate until after the early adopters are on board and worked out all of the implementation details. Any mandate that comes too early, alienates these early adopters and hurts implementation. Food for thought.

Speaking of mandates.....

....Eli alerted me to a new screening law in Maine that went into effect last week. According to this news account, the law requires MRSA screening of all new admissions with recent hospital or long term care facility exposure, those on dialysis, and prisoners. Interestingly, it appears the law only mandates screening for six months, and then allows hospitals to decide themselves how to proceed with MRSA prevention….if true, it is a curious wrinkle. I was especially interested in this law, given that I practiced infectious diseases in Maine for 3 years before returning to Iowa to get further training in microbiology.

Take note of this quote from Dr. John Jernigan at CDC:

"There are some studies that suggest that programs that have employed active screening have been very effective at reducing MRSA infection rates. On the other hand there are those that have used the strategy and not been so effective. Also there are studies that suggest that MRSA infection rates can be reduced effectively without the use of active screening. It may have to be tailored to local situations and circumstances."

I’m scheduled to present a “pro-con” debate with John next month….since I agree entirely with this quote, I’m worried that our “debate” will be very boring.

Addendum: I found the legislation on the interwebs. This is what was submitted, and this is what passed (can you detect the subtle changes?). So the details above must be what the Maine Quality Forum cooked up.

New MRSA bill in Congress

The MRSA Infection Prevention and Patient Protection Act was recently introduced into both the US House of Representatives and Senate. The bill mandates testing for MRSA on admission and discharge from the ICU and other high risk units. We can only hope that CDC is able to influence the lawmakers by bringing MRSA into perspective. If not, the winners will be the companies who make the MRSA testing products. The losers will be hospital infection control programs, which are already overworked and underresourced, and the patients these programs serve.