When prevention success stagnates? Treat the patient!

Prevention is paramount – I do believe this, and I know that is so much of what healthcare epidemiologist strive for; however we often become very myopic and focus exclusively on “modifiable risk factors.” It is refreshing to read a nicely done epidemiologic study to illustrate what an impact the infectious disease community can have by improving the way we approach patient treatment. When reading the recent article in JAMA IM by Michihiko Goto and colleagues I was expecting a nice ecologic study showing an impressionistic picture of how improved treatment processes correlate with improved MRSA bacteremia mortality - but our VA colleagues working with big data have painted more of a realistic 

Le Déjeuner sur l’herbe Painting 

by Édouard Manet, 

1863 Musée d’Orsay, Paris

Google Arts & Culture

than an impressionistic picture. Using the VA database, capturing deaths occurring during both the inpatient stay and the post-discharge period, they quantify improved survival at the patient level is driven by improved processes of care for S. aureus bloodstream infection (Association of Evidence-Based Care Processes With Mortality in Staphylococcus aureus Bacteremia at Veterans Health Administration Hospitals, 2003-2014 JAMA IM).

I have been pushing for transitioning efforts to prevent S. aureus (more specifically MRSA) bacteremia to the post-discharge setting, worried that the recent reductions observed among hospital-onset MRSA BSI are not being realized in the post-acute care setting. We’re getting stuck; in fact we have been stuck for a while at preventing community-associated MRSA BSI (See figure). 

Goto and colleagues provide some clarity to preventing deaths related to MRSA BSI (and S. aureus BSI overall) even during periods of prevention stagnation such as we may be in currently. Of note, the incidences of healthcare-associated and hospital-onset MRSA BSI decreased in VHA hospitals between 2003 and 2014, whereas the incidence of CA bacteremia was stable. This is identical to trends illustrated nationally using CDC's EIP data, suggesting the VA analysis may be reflective of what is going on nationally.

Goto utilized the national Veterans Health Administration (VHA) health care system to first determine how to best risk adjust mortality; and then determine the independent effect of each of three pillars of guideline directed processes of care for managing S. aureus bacteremia (SAB): (1) appropriate antibiotic therapy, (2) echocardiography, and (3) consultation with ID specialists. They report lower risk-adjusted mortality among patients with SAB when they received (1), (2), or (3), and there was a nice dose-response relationship between the number of care processes and mortality. They estimate “57.3% of the decrease in risk-adjusted mortality among patients with SAB between 2003 and 2014 could be attributed to increased use of these evidence-based care processes.”

Their paper also sheds some light on the importance of capturing post-discharge data when quantifying mortality related to processes or infections related to the hospital setting!. Risk-adjusted mortality decreased from 23.5% in 2003 to 18.2% in 2014, regardless of MRSA, MSSA, and place of acquisition (Figure).

From Supplemental Figures, Goto et. al.

I was caught by the discrepancy between these mortality rates and the mortality reported by the CDCs Emerging Infections Program invasive MRSA Surveillance, reported around 12%. The latter is limited to in-hospital or 30 day mortality, whichever comes first. Recent data at IDWeek by one Emerging Infection Program site identified another 30% of deaths among patients with MRSA BSI occurred post-discharge. The Goto paper captures these deaths, making their conclusions more realistic. In addition, their risk adjustment for mortality included over 13 comorbidities, timing of infection, and susceptibility. They did an outstanding job of trying to evaluate the relative importance of each care process while accounting for changes/absence/presence of these underlying predictors of mortality (both inpatient and post-discharge). They even did a sensitivity analysis to account for early deaths, before these care processes could occur. Furthermore, the impact of receiving the care processes was similar regardless if the SAB was hospital-onset or community-onset!

The bottom line – following evidence base care processes does save lives. Their analysis support their conclusion that “there is a need for continued implementation of quality improvement initiatives to increase the adoption of these evidence-based care processes for patients with SAB.” Let’s be bold and call these what they are: performance measures! Here we have a potential metric (proportion of SAB receiving said care process), that are closely linked to improved survival, and can be captured electronically in an objective manner (at least in the VA!) improving the reliability of these metrics across facilities. I was glad to find through a google search that IDSA recently (December 2016) urged CMS to move in this direction -- although the details of the status of this proposed measure (#407) are not clear to me.

While we anticipate novel therapeutics for MRSA BSI including immunotherapy adjuvants or vaccines to become available in the not to distant future; quality improvement efforts in the evidence-based processes of care for SAB now will likely improve our patient’s outcome. I hope we can turn our attention here soon -- and have hospitals rewarded for doing so.

HOP, SCIP: JUNK!

There is an interesting paper in the most recent Clinical Infectious Diseases on the Surgical Care Improvement Project (SCIP) and the Hospital Outpatient Measures Project (HOP). These are national QI projects intended to improve surgical care. The article lays out numerous problems with these projects, yet despite that, results from these projects will now begin to impact hospital reimbursement.

The authors note: "Measures are rolled out before their full impact is assessed, using live hospitals as the testing ground and relying on individuals trying to comply with these measures to troubleshoot. When issues do arise that require the measures to be changed, response times are invariably at least 6 months; meanwhile patients may be at risk, and measures are consistently failed."

It would be very interesting to know what these projects have cost hospitals. At my hospital, we have 1.5 nurse FTE just to abstract the data. Beyond that are thousands of hours of physician and nurse time spent trying to improve compliance with the metrics. And yet, there remains no compelling published data that outcomes have been improved.

Dale Bratzler, the brainchild of the projects, writes a response in the same issue of CID. At best, his response is tepid, and sheds little light on why these projects should be continued. His commentary ends like this: "The specific issues with SCIP performance metrics highlighted by Weston and colleagues are clearly a source of frustration for providers. However, the authors do not provide any evidence that harm has occurred because of implementation of SCIP." Ok, so we wasted millions of dollars, frustrated clinicians, and are about to punish hospitals financially, but we don't think any patients were harmed. Now that's exactly why much of QI is viewed as a joke!

An Ounce of Evidence

Ashish Jha, a physician and health policy researcher at Harvard, has a new blog, An Ounce of Evidence. His first posting is an interesting and informative piece on hospital rankings and includes guest posts by staffers at Consumer Reports and Leap Frog. It's clearly worth taking a look.

Bundle fumble?

This week's JAMA has an excellent review (free full text here) on the prevention of ventilator-associated pneumonia (VAP). Specifically, the authors offer a critical assessment of the widely utilized IHI VAP bundle. They offer two important conclusions:

• "The ability of the bundle to prevent VAP has not been definitively established with high quality studies."
• "No large randomized study has demonstrated that reducing VAP using any strategy, including the IHI bundle, is associated with improvements in clinical outcomes."

Here's another example where an intervention has been touted as gospel, pushed hard and implemented broadly without the evidence necessary for a hardline approach.

Photo: OregonLive

Healthcare quality: Running from science

We've blogged before about our hang-ups with QI. So I was pleased to see a thoughtful perspective in this month's Health Affairs by Peter Pronovost and Richard Lilford entitled "A Road Map for Improving the Performance of Performance Measures." As you might guess, this piece focuses on the validity of quality metrics, and Pronovost deserves credit for trying to push the hospital quality community to clean up its act.

The conclusion of the essay is also the money quote: "For the past decade, health care quality has largely sought quick fixes and run from science; the results are evident. Let us hope that efforts in the next decade embrace science instead."

A surgical site infection enhancement bundle?

A new paper in the Archives of Surgery evaluated the implementation of a surgical site infection (SSI) prevention bundle for colon surgery (see free text of the paper here). The bundle contained the following components:

• Omission of mechanical bowel preparation
• Preoperative and intraoperative patient warming
• Increased concentration of inspired oxygen during and immediately after the surgical procedure
• Limiting intraoperative intravenous fluid volumes
• Use of wound barriers to protect the surgical wound from contamination during the procedure 

Each component of the bundle was supported by 1 or more randomized trials demonstrating reduction in SSIs. About 200 patients were randomized to receive either the bundle or standard care. SSIs were determined by IPs using CDC case definitions. The study was terminated after a planned interim analysis revealed a <1% chance of showing a positive effect of the bundle were the study to continue to the accrual goal.

The overall rate of infection was 35%. In the control group, 24% of patients developed an SSI vs. 45% in the intervention group (p .003). In multivariate analysis, the bundle was shown to an independent predictor for SSI (RR 2.49, CI95 1.36-4.56).

I was struck by the very high rate of infection in this study, so I looked at CDC's most recent surveillance report, which shows that the mean pooled infection rates for colon surgery depending on risk category ranges from 3.99% to 9.47%. The authors postulate that their case ascertainment may be greater since the study was performed at a VA hospital where outpatient follow-up of patients is much easier to track. However, even in light of that, the rate still seems quite high. But on the other hand, unless there is something very unique about these patients or the care provided at this hospital, you might think that an effective prevention bundle would have even more impact in a setting with exceedingly high infection rates. This raises more concern that the bundle was not just ineffective but actually increased the risk of post-operative infection.

This paper is another example of how immature implementation science remains. I think the authors of this paper are correct to conclude that bundles of evidence-based interventions need to be formally tested before there is wide spread implementation.

Don't pull that trigger!

The headline on the front page of the New York Times this week read "Study Finds No Progress in Safety at Hospitals." This article (graphic shown) reported on a paper in this week's New England Journal of Medicine (free text here). In this study, 240 charts from each of 10 hospitals in North Carolina were reviewed using the Institute for Healthcare Improvement's (IHI) Global Trigger Tool. The admissions reviewed spanned the years 2002 to 2007.

Now I didn't know much about the Trigger Tool and the methods section of the paper doesn't give much description, so I looked up the guide, which you can review here. The triggers are 53 different indicators that when observed in the medical record should prompt further review to assess for an adverse event. For example, administration of benadryl is a trigger to look for a drug allergy, which according to IHI is an adverse event. Adverse events are further classified by severity and whether they were preventable. Per the IHI guide, no more than 20 minutes can be spent on the review of any chart (that rule was also observed for the published study).

Per the IHI methodology, healthcare-associated infections are both a trigger and an adverse event. Here is what the guide states (p.17):

Any infection occurring after admission to the hospital is likely an adverse event, especially those related to procedures or devices. Infections that cause admission to the hospital should be reviewed to determine whether they are related to medical care (e.g., prior procedure, urinary catheter at home or in long-term care) versus naturally occurring disease (e.g., community-acquired pneumonia).

Note that HAIs are never defined. Unlike the CDC's National Healthcare Safety Network (NHSN), which defines infections using multiple data points, IHI methodology doesn't guide the reviewer as to case ascertainment. I did a PubMed search this morning and found no studies where the Trigger Tool was compared to NHSN methodology to assess its validity.

So here are some concerns I have about this paper and the Trigger Tool:

• By design the Trigger Tool is not true surveillance. There is no attempt to detect all instances of harm. Imagine looking at the medical record of a patient who stayed in the hospital for 8 months with a 20-minute time limit. While I can understand how the Trigger Tool might uncover problems in any given hospital using a case-based approach for quality improvement, to look for trends over time using these data doesn't make any sense since there is no attempt to capture all the cases of harm. Of what value is trending incomplete data? I think this harkens back to the philosophical differences between quality improvement and healthcare epidemiology that I've talked about before. 
• I have serious concerns regarding the validity of this approach for HAIs. We know how problematic surveillance can be even when using well-delineated case definitions and how poorly administrative claims data perform for HAIs. The IHI approach seems much more analogous to the administrative data approach.
• In the New England Journal paper the secular trends were shown only for all harms and preventable harms, but not for any of the component harms, such as HAIs. It would be interesting to see the trended data for HAIs. Recall that AHRQ, using administrative claims data, recently published a paper claiming that HAIs are increasing in the US, while CDC, using much more rigorous surveillance methodology, published the opposite conclusion.
• Generalizability seems to be problematic. In this paper 2,400 hospital records were reviewed from 10 hospitals in a single state. Over the same time period, there were approximately 220 million hospital admissions in the US. That means that about 1 in 100,000 hospital admissions were reviewed (and only partially given the 20-minute rule). While the published paper never attempts to generalize the study findings to the universe of US hospitals, the media certainly did, and the lead author of the study states in the New York Times, “It is unlikely that other regions of the country have fared better.”
• Some of the instances of "harm" are not preventable and I'm not sure how they are related to quality of care. For example, consider the case of a patient with no known drug allergy who is treated with an antibiotic and develops a rash. This would be classified as a harm, and it is indeed a harm to the patient, but it's not predictable and not preventable. How does it help us to trend such data? And how would we attempt to reduce this harm? It is preventable harm that needs our attention. 
• With regards to HAIs, even if these data were valid, I don't believe these data reflect the current state of affairs in US hospitals given that much improvement in infection rates has occurred since 2007.

So here we have another paper that beats us up some more. If truth be told, I bet that the quality of care in US hospitals is significantly better today than it was in 2002. It sure would be nice to see that in print, but it probably wouldn't hit the front page. 

P.S. It's amazing that IHI claimed to have saved 123,000 lives in the US due to its safety program for US hospitals, but now claims that during the same time frame there was little evidence of improvement in patient safety. Something doesn't compute......  

Why are quality and infection prevention programs like oil & water?

This week, I analyzed our most recent performance in the HOP project. HOP is an acronym for the Hospital Outpatient Quality Data Reporting Program (HOP QDRP), a CMS program that is publicly reported at Hospital Compare. HOP is the outpatient analogue of SCIP (the Surgical Care Improvement Project), though HOP has only 2 metrics--pre-procedure antibiotic selection and appropriate timing of the antibiotic dose. We slice the data by procedure, service and surgeon to look for areas where performance is suboptimal. I received an email from one of our senior surgeons who complained that he had several cases where he was deemed noncompliant because the pre-procedure anitbiotic was not given within the window period 60 minutes prior to incision. He pointed out that the reason for his "noncompliance" was that these were dialysis patients who had received a dose of vancomycin at dialysis the day before. And he was practicing good medicine because the patients would still have a therapeutic level of vancomycin at the time of the procedures (all vascular access procedures). So we posted a query as to why this situation would be considered noncompliant (i.e., could the rules be changed to allow for this situation?). We received prompt responses from the physician in charge of the national project, but he avoided answering the question. After multiple emails back and forth, he finally stated that the surgeon did the right thing, but it would still be deemed noncompliant. He went on to say that hospitals should not use the data in this way (i.e., drill it down to the provider level) and the project leadership could not possibly think of all the exceptions for when an antibiotic should not be given within 60 minutes of incision. Now I have some problems with his thinking--if you are going to publicly report our performance then I think you need to be flexible enough to allow for exceptions that actually reflect good practice, and in an environment of 24/7 communication it shouldn't be hard to have a panel of experts make decisions on requests for exceptions to the rules. With SCIP we've actually been dinged when a pre-op antibiotic was not given before incision for a patient who entered the OR in cardiac arrest! I've blogged before about how these types of problems really turn physicians off not just to these specific projects but to quality improvement projects in general.


All of this made me think some more about the differences in quality improvement and healthcare epidemiology. The table below is modified from a plenary talk I gave at SHEA a few years ago. These differences really become sources of friction when QI and hospital epi folks are pulled into common projects like SCIP and HOP.

Characteristic	Healthcare Epidemiology	Quality Improvement
Philosophic orientation	Modern	Post-modern
Primary influences	Science & medicine	Business
Analytic orientation	Population based	Often case based
Focus	Exploration & analysis	Modification
Primary audience	Internal stakeholders	External stakeholders
Primary task	Define problems, elucidate risk factors	Design & implement interventions
Content expertise	Almost always	Usually not
Strength	Rigorous methodology & validity	Process design
Approach	Structured, relatively uniform	Innovative
Delivery style	Instructive	Collaborative
Solutions	Targeted	Empiric
Tactics	Data oriented, relatively dull	Flashy campaigns, catchy slogans
Perspective	Long term	Short term, evolving
Tempo	Relatively slow	Relatively fast

I don't have any solutions for how to make the groups work together more effectively. But perhaps starting with a recognition that our approaches to problems are different is a start.

This healthcare quality report is excellent....for us to poop on!

So SHEA, APIC and IDSA have released a joint statement that echoes (and expands upon) Mike's concerns about the recent, highly publicized AHRQ report. The statement and accompanying set of talking points speak for themselves. I will quote one small section, and let you read the rest.

"We are concerned that any report coming from a government agency based solely on the use of administrative data, commonly referred to as billing/coding data, paints an inaccurate picture of healthcare-associated infections for the public. In contrast, another Department of Health and Human Services agency -- the Centers for Disease Control and Prevention (CDC) – is preparing to release epidemiologically sound, surveillance data based on the National Healthcare Safety Network (NHSN). Multiple studies have concluded that administrative coding data appears to be a poor tool for accurately identifying infections. This may create greater confusion among consumers."

Quality improvement: 2 steps forward, 1 step back

I don't know how many hospital infection prevention programs are tasked with tracking the performance metrics for the Surgical Care Improvement Project (SCIP). It's a quality improvement program designed to reduce post-operative complications. At many hospitals it belongs to the QI folks, but since it started with the tracking of infection prevention metrics (perioperative antimicrobial prophylaxis), at my hospital Infection Control houses the program. The byzantine rules for data abstraction are mind-numbing, and I'm really thankful to have a wonderful nurse abstractor. But at times these projects lose the forest for the trees. Here's a case I dealt with last week. A patient was admitted with a lawn mower blade injury resulting in a dirty, open wound that required vascular reconstruction. Because the surgeon continued the patient's antibiotic beyond 24 hours post-op, he was judged noncompliant because the rule says antibiotics have to stop within 24 hours unless an infection is present pre-operatively. In my opinion, as an infectious diseases specialist I agree with the surgeon, so if I am run over by a lawn mower, give me the antibiotics! As Peter Pronovost notes in his book, measuring quality is often an endeavor of dubious quality. In the big picture (at the hospital level), it probably doesn't matter that we have some misclassified cases like this one. However, quality shoots itself in the foot with these seemingly stupid issues that turn physicians off. And this is exactly why it's difficult to get buy-in from doctors on QI projects.

Intermountain Healthcare

Here's a preview of this Sunday's NY Times magazine piece on the Intermountain Healthcare approach to improving health care quality, which relies upon standardization and protocol-driven approaches to care.

Posted without comment, both because I haven't finished the article myself, and because I'm on the ID consult service right now--keeping busy seeing patients who for the most part do not fit easily into any diagnostic or treatment protocol.

Hospital quality indicators

A group of Australian researchers have published an inventory of hospital care quality indicators that are reported to external agencies, including those related to healthcare associated infections. They have cataloged and classified nearly 400 different metrics. Click here for a spreadsheet of their work.