Will Antimicrobial Stewardship be the Next Target for De-implementation?

First, an honest confession, Mike's tweet had nothing to do with antimicrobial stewardship, but rather contact precautions. But his point is just as valid when discussing antimicrobial stewardship and there will come a time when forces will align to question the benefits and costs of stewardship programs since now and in the future they will lack the "necessary" cluster-randomized trial evidence supporting their existence.

There is a longer discussion to be had here sometime in the future, when I'm not writing a Center grant renewal, but the key question is what we consider "high-level" evidence. For most de-implementation supporters and indeed most infection control and stewardship guideline authors, high-level evidence is synonymous with individual or cluster-randomized trials. They simply cannot accept non-randomized, quasi-experimental designs as evidence. It is gotten to the point that the recent CDI Guidelines completely excluded quasi-expermintal designs from their level of evidence figure, despite the fact that one of the original Grade Criteria papers lists QE studies in its table and allows them to be ranked higher than RCTs, if certain criteria are met.

OK.  So why am I rambling on about level of evidence and misapplying a tweet from 2 weeks ago? There was a new systematic review just published in AJIC by Leandro Bertollo and colleagues that asked the question: "Are antimicrobial stewardship programs effective strategies for preventing antibiotic resistance?" To answer this question they reviewed all studies published between from January 2012 to January 2017 and followed the standard PRISMA statement recommendations for reporting their findings.

Results: They identified and extracted data from 26 studies, of which 22 were single-center and four were multicenter studies. Study designs are listed in Table 2, below, with the special note that none of the before/after studies included a contemporaneous, unexposed control group. A major concern that the authors identified was that in 7 of the 26 studies (30%), there was evidence that infection control interventions were implemented at the same time as the stewardship intervention and that the majority (57%) of the stewardship studies that reported positive results were confounded by simultaneous implementation of new infection control practices. High fives for hand hygiene.

Their conclusion: "There is no solid evidence that ASPs are effective in reducing antibiotic resistance in hospital settings. There are still few studies analyzing this matter, most of them with inappropriate study designs. We uphold the need for more studies with appropriate study designs and standardized ASP interventions targeting common microorganism-antibiotic pairs."

The need for more studies. Sounds like the siren call for de-implementation to me. Sure, we can wait around a decade or four for some magical $20 million cluster-randomized study that swabs all patients on admission/discharge, completes a full microbiome analysis and tracks patients for a year post discharge for resistant infections. Or, we can expand our ideas around what "high-level" evidence means and fund well-designed and controlled quasi-experimental studies and also consider strong epidemiological evidence, such as exposure to antibiotics leads to colonization with resistant pathogens. We can be logical. Yeah, not gonna happen. But at least you were warned.

De-implementation and Noninferiority in Infection Control Studies

De-implementation or "stopping practices that lack supporting evidence" is a popular topic in infection control circles. In fact, just yesterday I read a discussion where the authors suggested we no longer need to practice hand hygiene after removing gloves when caring for patients with CDI. I guess there aren't randomized trials - you can't be serious!

Which brings me to a recent review in the NEJM by Laura Mauri and Ralph D'Agostino titled "Challenges in the Design and Interpretation of Noninferiority Trials."  This review is very well written - perhaps required reading for epidemiology students well written. In infection control, it is important to recognize that most de-implementation studies are really non-inferiority trials. For example, when we discontinue contact precautions, we are really suggesting that "stopping contact precautions" is non-inferior to continuing contact precautions in preventing MDRO transmission - of course ignoring that compliance with contact precautions is probably so poor that they are basically the same intervention!

In the contact precautions example, we would be testing whether stopping contact precautions "is not worse than the control (continuing contact precautions) by an acceptably small amount, with a given degree of confidence." The null hypothesis would be that discontinuing contact precautions leads to higher transmission of MDRO (i.e. is worse) and rejection of the null hypothesis is used to support the claim that discontinuing CP is noninferior. Here I suggest you stare at Figure 1 for a bit (probably easier to read in the paper with the description of each condition, but I have included it below anyway)

Further discussion about the design and analysis of these trials is way beyond the scope of a humble blog post; however, the authors include nice descriptions of methods for deriving noninferiority margins, the "constancy assumption" and statistical analysis approaches. But their 6th and 7th components of noninferiority trials are worth mentioning from an infection control standpoint:

6) Adequate ascertainment of outcomes: The authors write that "incomplete or inaccurate ascertainment of outcomes, as a result of loss to follow-up, treatment crossover or nonadherence, or outcomes that are difficult to measure or subjective, may cause the treatments being compared to falsely appear similar."  I would suggest that studies that seek to de-implement contact precautions that do not include admission/discharge surveillance cultures seeking to detect transmission events fail this criteria.

7) Issues with "Intention-to-Treat" in noninferiority designs: In a superiority studies (typical RCTs), intention-to-treat analysis, where anyone who receives the treatment is included even if they get one dose, is the gold standard. The authors write: "In a noninferiority study, however, if some patients did not receive the full course of the assigned treatment, an intention-to-treat analysis may produce a bias toward a false positive conclusion of noninferiority by narrowing the difference between the treatments. In some instances, a per-protocol analysis, which excludes patients who did not meet the inclusion criteria or did not receive the randomized, per-protocol assignment, may be preferable in a noninferiority trial. However, a per-protocol analysis may include fewer participants and introduce postrandomization bias. In general, both the intention-to-treat and per-protocol data sets are important. We suggest analyzing both sets and examining the results for consistency."

Just some things to think about as we read the coming wave of de-implementation studies in infection control including diagnostic stewardship.