Lack of Financial Incentives for Preventive Interventions

One of the barriers to infection control and antimicrobial stewardship is the lack of financial support from third-party payers, such as insurance companies and medicare. For example, wouldn't it be great if we could bill for hand hygiene every time it was practiced correctly? The end result of this lack of support is that society is willing to pay thousands for a CABG surgery, for example, but not willing to spend a penny on preventing a surgical site infection.

There is a new perspective in the NEJM by two physician behavioral economists, Katherine Pryor and Kevin Volpp, that examines barriers to putting preventive services on equal financial footing with treatments. While their comments largely focus on patient-targeted (vs population targeted) preventive services like lifestyle-modification for diabetes prevention or smoking cessation, they highlight many barriers that could apply equally to population-level interventions like infection control. I've highlighted a few:

"...treatments determined by the Food and Drug Administration (FDA) to be safe and effective are usually covered by insurers regardless of their cost, but preventive services have been held to a higher standard: they are often assessed on the basis of whether they generate a positive return on investment and save money in the short term. This disparity leads to overprovision of treatments and underprovision of preventive services"

"Even Medicare — which typically covers beneficiaries for life — holds preventive services to a higher standard, applying cost-effectiveness analyses when making coverage decisions about preventive services but not treatments. This double standard has resulted in coverage of cost-ineffective therapies with prices of up to hundreds of thousands of dollars per quality-adjusted life-year, including treatments of questionable benefit"

The perspetive is worth a careful read. Would also be nice to see a similar commentary describing these barriers from an infection control perspective too.

De-implementation and Noninferiority in Infection Control Studies

De-implementation or "stopping practices that lack supporting evidence" is a popular topic in infection control circles. In fact, just yesterday I read a discussion where the authors suggested we no longer need to practice hand hygiene after removing gloves when caring for patients with CDI. I guess there aren't randomized trials - you can't be serious!

Which brings me to a recent review in the NEJM by Laura Mauri and Ralph D'Agostino titled "Challenges in the Design and Interpretation of Noninferiority Trials."  This review is very well written - perhaps required reading for epidemiology students well written. In infection control, it is important to recognize that most de-implementation studies are really non-inferiority trials. For example, when we discontinue contact precautions, we are really suggesting that "stopping contact precautions" is non-inferior to continuing contact precautions in preventing MDRO transmission - of course ignoring that compliance with contact precautions is probably so poor that they are basically the same intervention!

In the contact precautions example, we would be testing whether stopping contact precautions "is not worse than the control (continuing contact precautions) by an acceptably small amount, with a given degree of confidence." The null hypothesis would be that discontinuing contact precautions leads to higher transmission of MDRO (i.e. is worse) and rejection of the null hypothesis is used to support the claim that discontinuing CP is noninferior. Here I suggest you stare at Figure 1 for a bit (probably easier to read in the paper with the description of each condition, but I have included it below anyway)

Further discussion about the design and analysis of these trials is way beyond the scope of a humble blog post; however, the authors include nice descriptions of methods for deriving noninferiority margins, the "constancy assumption" and statistical analysis approaches. But their 6th and 7th components of noninferiority trials are worth mentioning from an infection control standpoint:

6) Adequate ascertainment of outcomes: The authors write that "incomplete or inaccurate ascertainment of outcomes, as a result of loss to follow-up, treatment crossover or nonadherence, or outcomes that are difficult to measure or subjective, may cause the treatments being compared to falsely appear similar."  I would suggest that studies that seek to de-implement contact precautions that do not include admission/discharge surveillance cultures seeking to detect transmission events fail this criteria.

7) Issues with "Intention-to-Treat" in noninferiority designs: In a superiority studies (typical RCTs), intention-to-treat analysis, where anyone who receives the treatment is included even if they get one dose, is the gold standard. The authors write: "In a noninferiority study, however, if some patients did not receive the full course of the assigned treatment, an intention-to-treat analysis may produce a bias toward a false positive conclusion of noninferiority by narrowing the difference between the treatments. In some instances, a per-protocol analysis, which excludes patients who did not meet the inclusion criteria or did not receive the randomized, per-protocol assignment, may be preferable in a noninferiority trial. However, a per-protocol analysis may include fewer participants and introduce postrandomization bias. In general, both the intention-to-treat and per-protocol data sets are important. We suggest analyzing both sets and examining the results for consistency."

Just some things to think about as we read the coming wave of de-implementation studies in infection control including diagnostic stewardship.

Preventing Catheter-Associated Urinary Tract Infection in Acute Care

Dr. Mohamad Fakih

This guest post is by Mohamad Fakih, MD, MPH, Senior Medical Director of the Ascension Center of Excellence for Antimicrobial Stewardship and Infection Prevention; Sarah Krein, PhD, RN, Research Scientist and Research Professor at VA Ann Arbor Healthcare System and University of Michigan; and Sanjay Saint, MD, MPH, Chief of Medicine, VA Ann Arbor Healthcare System and University of Michigan Medical School

Catheter-associated urinary tract infection (CAUTI) – sometimes jokingly referred to as the “Rodney Dangerfield” of healthcare-associated infections – finally got respect in 2008 when the Centers for Medicare & Medicaid Services stopped reimbursing for these events if associated with a hospitalization. Since then, hospitals have attempted to decrease their CAUTI rates, with this work taking on more urgency now that CAUTI is publicly reported and associated with a reduction in compensation through value-based purchasing.

CAUTI, however, presents not only a financial risk to hospitals, but a substantial safety risk to patients, especially with prolonged – and often unnecessary -- use of the catheter. The “On the CUSP Stop CAUTI” initiative attempted to implement best practices to reduce CAUTI in hospitals across the United States. The program was modeled after a successful statewide initiative in Michigan that found a 25% reduction in catheter use and CAUTI, and included a qualitative assessment to better understand the socio-adaptive aspects of CAUTI prevention. The national program focused on sharing best practices to reduce urinary catheter risk, and helping healthcare workers change their behavior to adopt measures that reduce CAUTI. This effort, sponsored by the Agency for Healthcare Research and Quality (AHRQ), included a collaboration of professional societies, academic researchers, government agencies -- including the CDC -- and state hospital associations. The program was led by the Health Research and Educational Trust with the support of faculty from the University of Michigan, St. John Hospital and Medical Center, the MHA Keystone Center, and Johns Hopkins Medicine Armstrong Institute for Patient Safety and Quality.

The newly published NEJM article presents the results of the first 4 (of 9) cohorts that participated in the initiative, and encompasses 926 units in 603 hospitals across 32 states, the District of Columbia, and Puerto Rico.  These results showed a 32% reduction in CAUTI in the non-intensive care units (ICUs), but no change in ICUs. In addition, a 7% reduction in catheter use was seen in non-ICUs. Early in the work, we evaluated the indications for catheter use and found marked differences between the ICU and non-ICU. Catheter use was reported to be indicated ~ 94% of the time in the ICUs compared to ~65% for the non-ICUs. In addition, the primary indication for using the catheter in the ICU was for fluid monitoring in the critically ill. The newly released Ann Arbor criteria for urinary catheter use may help better clarify the appropriate indications in critically ill patients.

The main elements of the initiative that helped to produce these results were to: 
1) integrate evaluation for catheter use as part of healthcare worker’s daily routine, particularly nurses
2) avoid catheter use by considering alternative urine collection methods
3) ensure that aseptic practices are used when inserting and maintaining the catheter such as hand hygiene.

Additional elements included regular feedback on performance related to catheter use and CAUTI events. Identified gaps were addressed and mitigated. A key component was ensuring adoption of best practices and buy-in from different stakeholders. This was achieved by addressing  catheter harm for specific stakeholders, ensuring leadership support of the essential disciplines, underscoring the collaborative nature of CAUTI prevention, and identifying champions within the organization to lead the effort.

The “On the CUSP Stop CAUTI” initiative demonstrates that scaling up complex interventions --with technical and socio-adaptive components -- to a national level is achievable at least in non-ICUs. CAUTI is certainly not a “sexy” healthcare-associated condition but how a hospital addresses CAUTI likely says much about how such a facility attacks other endemic and mundane harms such as falls, delirium and pressure sores. We are currently extending our CAUTI prevention program to long-term care and to those hospital units – both ICU and non-ICU – that have persistently elevated CAUTI rates. We hope to share those results soon.

Will you trade me a CLABSI for a pneumothorax?

There was an interesting new study published today in the NEJM by Jean-Jacques Parienti and colleagues and funded by the French Ministry of Health. The study aimed to compare the rates of catheter-related bloodstream infection and symptomatic deep-vein thrombosis (DVT) in 2532 adult ICU patients randomized to subclavian (N=843), jugular (N=845) or femoral vein (844) sites for nontunneled central venous catheter insertion. Baseline characteristics were very similar across the three groups.

The results are not very surprising with higher rates of infection and symptomatic DVT when catheters were inserted using the jugular and femoral veins compared to the subclavian veins but these were almost cancelled out by a higher risk of pneumothorax when using the subclavian approach. (see figure below).

Here are my thoughts on the paper:

1. This is a very nice study that confirms the results of prior smaller studies in the field. 
2. The overall complication rate is low with approximately 97% of catheters in each group inserted safely without a pneumothorax and without infection or DVT. 
3. When all complications are included, it doesn’t appear there is a preferred site for catheter insertion. However, since hospitals are penalized for infections and not other complications**, there has been a preference for chosing the subclavian site, since it is associated with fewer infections. Unfortunately, the subclavian site had a much higher rate of pneumothorax. 
4. One important comment that the authors make is that how long the catheter remains in place can greatly increase the rate of infections and blood clots. So, if the doctor expects the catheter to remain in for a short period of time, she might chose a femoral or jugular vein approach to limit the pneumothorax risk, with very little infection or clotting risk since the catheter will be removed before the complication can occur. However, if the catheter is to remain in place for many days, it is probably worth the higher risk of pneumothorax associated with the subclavian site, which only occurs when the catheter is inserted, in order to reduce the long term infection and DVT risks. 
5. The study sites did not use daily chlorhexidine bathing and did not place chlorhexidine-impregnated dressings at the catheter insertion site. Both of these interventions have been shown to reduce catheter-related bloodstream infections. Thus, these results might not be generalizable to hospitals that use chlorhexidine bathing and/or chlorhexidine dressings. It's possible that either of these interventions or both could mitigate the infection risk rendering femoral or jugular vein approaches safer than they appear in this study.
6. Overall, it looks like fears of using the femoral vein are exaggerated, especially when you consider the rates of other complications like pneumothorax. It may be that in our efforts to get to zero CLABSI, we're putting patients at higher risk for other complications. Perhaps a more nuanced target of "any complications per catheter inserted" could replace CLABSI as a quality metric?

**Edit: Mike kindly informed me that hospitals are also penalized for iatrogenic pneumothorax through an AHRQ patient safety indicator, which is included in the CMS Hospital-Acquired Condition Reduction Program. Thus, selecting a subclavian site could lower your CLABSI penalty but raise your pneumothorax penalty. Of course!

Perry Mason and REDUCE MRSA: Another "Case of the Positive Negative"

This guest post is by Marc-Oliver Wright, MT(ASCP), MS, CIC
 from NorthShore University HealthSystem
 in Evanston, IL 

"We need Perry Mason. Someone to put you in place." – Ozzy Osbourne

This week’s articles in the NEJM (Huang et al. “Targeted versus Universal Decolonization to Prevent ICU Infection”) and accompanying editorial (Edmond and Wenzel “Screening Inpatients for MRSA-Case Closed”) left me reminiscing of classic TV courtroom dramas where the well intentioned and sophisticated district attorney has finished presenting his/her case but Perry Mason has just cleared his throat and Matlock is still shining his ankle boots.


For those of you who missed it, Huang and colleagues randomized 74 intensive care units (ICUs) at 43 Hospital Corporation of America hospitals to one of 3 interventions among 2 cohorts.

1. Cohort one: States with mandatory MRSA screening (n=5)
     a. MRSA screening and isolation for ICU admissions (3)
     b. MRSA screening, isolation and decolonization for ICU admissions (2)
2. Cohort two: States without mandatory MRSA screening (38)
     a. MRSA screening and isolation for ICU admissions (13)
     b. MRSA screening, isolation and decolonization for ICU admissions (12)
     c. No screening, decolonization for all ICU admissions (13)

Decolonization included 5 days of intranasal mupirocin therapy and daily baths with chlorhexidine gluconate (CHG) for the duration of their stay in the ICU.

Adjusted hazard ratios for clinical cultures of MRSA with both decolonization strategies netted confidence intervals that were less than 1.0. Ditto for bloodstream infections with any pathogen (not just MRSA). When the authors selected out bloodstream infections caused by MRSA the findings were not significant. In the pairwise analysis universal decolonization attained statistical significance over screening/isolation/decolonization in the unadjusted analysis for bloodstream infection due to any pathogen. The authors conclude that universal decolonization is superior to either targeted decolonization or screening and isolation without decolonization.

This was a large, rather well designed, certainly well coordinated evaluation of multiple ICUs across the country. But the case is far from closed.

Defense exhibit one: Follow the blood. This article further affirms an already well conducted study by some of the same authors that demonstrated CHG bathing of ICU patients reduced bloodstream infections in ICU patients (Arch Intern Med. 2007;167:2073-9, ICHE 2009;30:959-63). There’s ample biological plausibility for this and in general, the horizontal (as opposed to vertical one MDRO at a time) approach is more logical. However, this effect in all likelihood has nothing to do with mupirocin, which has never been independently associated with a reduction in bloodstream infections. CHG reduces all-pathogen bloodstream infections in ICU patients: no objection your honor.

Defense exhibit two: An eyewitness-I see you. MRSA infections are not restricted by the boundaries of the unit. An APCHE score is not a pre-requisite to acquiring an MDRO. According to the evidence, ICU patients are at higher risk for MRSA infection, but many of prior studies were conducted in large academic medical centers where and when the case mix was different from today. Arguably, it’s operationally easier to limit intervention and research studies by geography.

Illinois was the first state to succumb to MRSA screening legislation; the legislation which reads that such screening will be performed on patients in “all intensive care units, and other at-risk patients identified by the hospital.” We, in the Land of Lincoln, have all, by design, implemented the former, but the latter is a hodge podge ranging from denial (only ICU patients are high-risk) to the assumed, evidentiary but not all-encompassing (patients from LTC are high-risk) to the near complete (70+ variable model built decision support module embedded in the EMR). The VA as well as my own organization deployed MRSA prevention strategies in ICU as well as non-ICU settings.

This brings me to: Defense exhibit three: A character witness. My organization’s strategy for the past 9 years has been a strategy most akin to Group 2 (screening, isolation and decolonization). Though we started in the ICU, we failed to see the kind of reduction we wanted (see defense exhibit 2) until we later expanded to universal screening. We used this experience to develop a robust multi-variable prediction model that was built as decision support system into our electronic medical record. This tool automatically calculates a MRSA risk score as patient data is entered and evolves within the record during the patient’s admission. When the risk score reaches a threshold the user is instructed to screen the patient. We used this to transition from universal screening to targeted screening in January 2012.

Ok. I am by no means an expert on MRSA. I am good with numbers though.

In the NEJM article, the best intervention for reducing MRSA clinical isolates is a rate of infection of 2.1 per 1,000 patient days. At NorthShore (and for comparison purposes, these measures are in our ICUs only) in the past 38 months, our unadjusted rate of MRSA infection is 0.3 per 1,000 patient days. Their best intervention netted an infection rate ratio versus my own ICUs of 7.0 (95%CI: 4.3-11.5, p=5.9 * 10-23).

Similarly, the NEJM article reported an overall, unadjusted rate of bloodstream infection from any pathogen of 3.6 per 1,000 patient days in the universal decolonization intervention arm. At NorthShore (and for comparison purposes, this is in our ICUs only) in the past 38 months, our unadjusted rate of bloodstream infection is 1.08 per 1,000 patient days. Their best intervention netted an infection rate ratio versus my own ICUs of 3.33 (95%CI: 2.54 -4.38, p=1.3 * 10-22).

I’m afraid I’ve only had 1 BSI due to MRSA in the past 38 months. My crude rate comes out as 0.018 per 1,000 patient days. If I understand Table 3 of the NEJM article correctly, the universal decolonization group experienced 48 BSIs with MRSA during the intervention and attributable ICU patient days of 101.603 for a rate of 0.47 per 1,000 patient days. This yields an infection rate ratio versus my own ICUs of 26.11 (95%CI: 4.53-150.45, p=3.1 * 10-8).

I am all too happy when a group of dedicated and brilliant people in hospital epidemiology find a new and improved way to prevent infection. And for this group, they did discover a new strategy for reducing their MRSA and bloodstream infections. I was not surprised to see that they have decided to implement the universal decolonization strategy across all HCA hospitals. But as long as their best rates are between 3 and 26 times higher than mine, I’ll be sticking with my own plan.

Defense exhibit four: If the antibiotic doesn’t fit you must acquit. Arguably, the single greatest concerning factor of this study that wasn’t addressed is the rise of mupirocin resistance. This exhibit, like the preceding, is rooted in personal experience. Until this year, our organization decolonized all MRSA-screen positive patients. During that 8 year endeavor our proportion of mupirocin resistant MRSA increased from about 3% to almost 12%. Nasal decolonization has been most convincingly demonstrated to reduce infection risk in the pre-surgical patient population and arguably only with select procedures. Our rise in resistant strains and the overwhelming desire to reserve this ammunition for the people that really need it (pre-surgical patients) gave rise to our decision to suspend mupirocin decolonization starting in 2013. We hope to see our resistance pattern return to baseline, though such a decline will likely be slow.

Justice may be blind, but we shouldn’t be to known potential adverse outcomes like antibiotic resistance.

Closing argument:  If none of the above convinced you that this case is far from closed at least consider this final argument. Living in Illinois, I am frightened awake at night with the sounds of what I imagine to be fingers typing on keyboards. The well-intentioned hands of a well-meaning lawmaker in our state or federal capital pounding out some new legislation in light of the latest and greatest publication on preventing some iteration of our nation’s bacterial scourges. Convinced that if they don’t do something those conniving hospital epidemiologists and infection preventionists won’t do anything to protect our nation’s sickest and frailest citizens. That all we’re here for is the big money check we get every week. By heaven, we must mandate! *shudder*

This paper, like all well designed and well-conducted studies deserves critical attention and consideration. Consider the methods they used and those they didn’t, the intervention processes they measured (CHG/mupirocin supply) and those they did not (isolation compliance), the settings in which the study was conducted (ICUs) and where it was not (everywhere else) and the outcomes they measured (MRSA and BSIs) and those they did not (antibiotic resistance). You must also compare their outcomes to your own performance. With such a wide variety of organizations the results may be generalizable despite HCA’s additional vertical strategies for MRSA that may have overlapped the study periods. How do your rates in your methods group compare those published here (1, 2, or 3?) versus the alternatives? Use this evidence as a tool for your ongoing risk assessment and make an educated decision before someone makes an uneducated one for you.

The defense rests.

(Updated!) What you missed in Infection Prevention: December 6, 2012

1) Today's NEJM has lot's of interesting stuff.  First up, they have a review of the first cases of fungal infections associated with contaminated methylprednisolone injections in Tennessee. The report covers 66 case patients with 21 having confirmed Exserohilum rostratum infection and 1 having confirmed Aspergillus fumigatus infection.

2) Next up, NEJM has a perspective piece from the FDA that highlights infections secondary to contaminated antiseptic products including iodophors, alcohol products, CHG and quaternary-ammonium compounds.

3) And finally from the NEJM, Thomas Sandora and Donald Goldmann have a perspective piece outlining their suggestions for preventing hospital outbreaks of antibiotic-resistant bacteria. Most of what they offer is standard infection control dogma from the  - it's the healthcare worker's fault for not washing his/her hands diatribe - to the suggestion that a "parsimonious set of interventions aimed at reducing exposure to antibiotics may have the greatest effect on resistance." They even included a table of the suggested parsimonious stewardship interventions (below). I don't believe that any are backed by more than uncontrolled quasi-experimental studies or expert opinion. (Please correct me if I'm wrong!) Might they have recommended funding studies of these interventions instead?  It's hard enough to be a hospital epidemiologist in 2012 marketing the few evidence-based interventions at our disposal without laying the burden of making us defend interventions based on pure speculation on our backs.  Oh well.

4) There is a report in today's Ottawa Citizen by Helen Branswell that nicely outlines two NDM-1 outbreaks that occurred (October 2011 and January 2012) in Toronto. The article covers an study published in ICHE and another in CID. The latter described the transmission of the NDM-1 between E. coli and Klebsiella species in the same patient.

5) Last-but-not-least, check out this story in NPR that highlighted an innovative research study out of Michigan State University. Researchers modeled the spread of murders in Newark, NJ as an infectious disease and discovered that murder appears to be transmissible like an infectious pathogen. They are now figuring out why some neighborhoods are more resistant to homicide and how they might "vaccinate" populations to reduce murder. Pretty cool and Go Sparty!

Does pay for performance in HAI pay off?

The data is piling up suggesting that the CMS policy for nonpayment for HAIs have had little impact on reducing preventable HAIs in acute-care settings. The straw that might break the camel's back is a study in this week's NEJM by Grace Lee and colleagues.  Using a quasi-experimental design and time-series analysis, this AHRQ-funded study looked to see if there was a change in HAI rates after (vs before) the October 2008 nonpayment policy went into effect. 398 NHSN hospital provided data. The results are pretty conclusive: there were no changes in CLABSI, CAUTI or VAP after the implementation of the policy.  In the figures below you can see that things are getting better, just not due to nonpayment.

Of course this is not surprising. Peter McNair published a very nice study in Health Affairs (2009) that estimated that the total financial impact across the entire US would be about $1.1 million annually for six avoidable conditions. When you divide that amount by the number of US hospitals you get...about nothing per hospital. I think CMS might need a bigger stick.

Source: Lee GM et al. NEJM 2012; 367: 1428-37

Intrigued by the plot of Kent Sepkowitz’ perspective piece on antibiotic stewardship in the NEJM

This is a guest-authored piece by Dr. Tom Gottlieb (Senior Specialist in Microbiology and Infectious Diseases, Concord Hospital, New South Wales, Australia), wherein he discusses Kent Sepkowitz's latest commentary in the NEJM.

This is writ in the genre of a thriller in which a new generation of ID physician villains (the previous heroic ID generation’s illegitimate offspring, it seems ) abound as ‘fervent’ ‘chastisers of antibiotic overuse and abuse’. These ‘self-pitying’, ‘inept’, ‘feckless’ ‘prohibitionists’, ‘gnawed with regret’, indeed ‘deranged’, threaten to undermine and disenfranchise the orderly world of antibiotic prescribing, by seeking to banish miracle antibiotics forever from the world. Indeed their quest, (in cahoots with the loonies of the anti-vaccination lobby), is so dangerous, it is spelt out in the more sophisticated French; “an idée fixe”. These antibiotic nihilists cannot appreciate the true contribution antibiotic chemotherapy has made to individual patient care and also to the global well being of humanity.

Now I too enjoy debunking zealotry, and would not deny that there is plenty about antibiotic stewardship that is worth challenging in a balanced discourse. And stewardship, (or is it shepherd-ship?) can be clearly problematic. As ID physicians, we are frequently left in a schizoid situation where, as on one hand we attempt to control unnecessary use, on the other, in individual care we often contribute to broad-spectrum prescribing. Moreover some clever contrarianism never goes astray. But Sepkowitz’ pendulum swings beyond healthy scepticism, past contrarianism, to something akin to denialism.

Damn it, I too need to reach for the French dictionary. Un agent provocateur? Peut-être, saboteur? Because there is significant damage caused when worthwhile attempts, not as suggested by Sepkowitz to deny antibiotics for patients, but to preserve antibiotics into an uncertain future, are derided by ID doyens. But without him providing us with any cogent examples of groups or policies that would support the existence of these villanous ‘antibiotic prohibitionists’. And the very simplistic, polemical, good versus evil like vision he paints, creates a disservice to the antibiotic debate. (Or is it that for the sake of a good argument, as Oscar Wilde wrote, Sepkowitz feels that “in matters of great importance, style, not sincerity is the vital thing”)

I agree with Sepkowitz that “just in case” prescribing, often saves the day in settings of uncertainty. But one of the banes of our clinical existence is ‘just in case’ prolonged orthopedic prophylaxis, ‘just in case’ ceftriaxone in heart failure, ‘just in case’ stat gentamicin dose pre catheter removal, just in case treatment in case confusion is caused by a urine infection, or a few more days of treatment ‘just in case’, etc..; situations in which antibiotics are used to treat the prescribers’ anxieties rather than the patient’s condition.

I find Sepkowitz’s piece in the NEJM more a diatribe than a commentary. But it is a very useful piece too. It is a wake-up call that if we fail to convince other ID physicians regarding the merits of prudent antibiotic use, we have a long way to go to change attitudes amongst medical peers or in the community.

Image: Nicolas Poussin's Adoration of the Golden Calf. "The Golden calf of Stewardship" paraded in front of idolatrous ID physicians?

Manuscript Provenance: Why NEJM has a high retraction rate (and it's not a reason mentioned in the NYT)

A recent high-profile story in the NYT by Carl Zimmer discussed a surge in article retractions since 2000. Over the period 2000 to 2009, retractions have risen by 3300%. (see calcs below) The Time's article draws much of its data and discussion from a pair of editorials published by Ferric Fang and Arturo Casadevall covering "structural reforms" and "methodological and cultural reforms" and a recent news feature in Nature. I highly recommend the two calls for reform from Fang and Casadevall, which cover topics ranging from increased administrative burden to problems with grant review. However, it is hard to make a link from any of these problems and scientific error or misconduct without further research.

For example, much of the discussion falls into the category of "survival of the fittest" with scientists doing everything possible to publish in high-profile journals (e.g. NEJM), which then is supposed to lead to increased grant funding and wealth.  I'm not denying that this could be a factor in scientific fraud, but it's hard to imagine this would increase errors, directly. If you look at the NYT graphic (above), fraud is associated with a minority of all retractions. To provide evidence that a scientists desire for fame is driving the retraction epidemic, Fang and Casadevall, published another editorial that included an analysis linking impact factor of the journal to a retraction index. They found that the higher the impact factor, the higher rate of retraction with NEJM at the top.

Reasons given for the association include higher risk-taking by authors in papers submitted to high-ranking journals and that publications in high-impact journals may just be subject to greater scrutiny (e.g. via social media). These reasons are appealing, although I'm not sure there's a testable hypothesis among them. One testable "systemic aspect of the scientific publication process" that is likely to be associated with both journal rank and retraction (and more likely to be causal) is a manuscript's provenance.

Manuscript Provenance
In the art world, a painting's provenance refers to the chronology of ownership of a specific painting or work.  When I speak of a manuscript's provenance, I mean where it's been submitted and received peer review prior to publication.  As anyone who has submitted a manuscript knows, you almost always submit the paper to a higher-impact journal first and then if not reviewed or accepted, you aim a bit lower.  In fact, there is only one paper we've submitted out of >100, that's gone initially to a lower-ranked journal and then when rejected, was submitted and accepted by a higher-ranked journal.

Thus, manuscripts submitted and accepted at higher-ranked journals are more likely to have been reviewed only once, or received one round of peer review. Yes, I know that some papers go to JAMA then NEJM and some papers are submitted directly to a specialty journal and are reviewed once, but in general, papers published in higher-ranked journals have been through fewer rounds of peer review.  Unfortunately, these data are not easily available, but it's likely this hypothesis can be tested. Nevertheless, I think quantity of peer review is an important predictor of quality of the final product.  It is unlikely that more peer review could detect outright fraud, but most retractions aren't fraud related.

So before we blame the entire system for a few bad apples, there need to be more epidemiological studies as to why this is occurring.  My first suggestion is that we should consider manuscript provenance as a factor and my second suggestion is that journals should pay for peer review, so that it becomes a valued exercise.  Peer review service should also be considered for promotion. Sure the system could be improved in many ways, but it's a stretch from there to finding a causal pathway between a journal's impact factor and a higher retraction index. Oh, and this is all probably because of twitter anyway.


***Retraction Rate Calculation: Using rates of 3/year in 2000 to 180/year in 2009 from the NYT article and given that there's been a 64% increase in PubMed articles  (529,000 in 2000 to 866,000 articles in 2010), this represents an increase from 0.0006% to 0.02%, or a 3300% increase.

Best news ever! A Norovirus vaccine that works!

As a parent or two young kids, I dread the slightest hint of nausea. Now, perhaps, I might have one less pathogen to fear and perhaps in old age, I can even take a cruise.

In the recent NEJM there is a report of an RCT assessing the safety, immunogenicity, and efficacy of an intranasal norovirus viruslike particle (VLP) vaccine. The study was conducted in adults aged 18 to 50. They received two doses of either vaccine or placebo and were subsequently inoculated with Norwalk virus during an inpatient challenge with 10 times the infectious dose and monitored for infection and gastroenteritis symptoms for a minimum of 4 days. (No way I'd ever volunteer for this study)

The study included 98 patients with 90 completing both injections. A Norwalk virus–specific IgA seroresponse was found in 70% of vaccine recipients. This level of response is  similar to my old Maryland colleague Samer El Kamary's 2010 JID paper that showed a 79% response rate. Vaccination significantly reduced the frequencies of Norwalk virus gastroenteritis (occurring in 69% of placebo and 37% of vaccine recipients, P=0.006) and Norwalk virus infection (82% of placebo and 61% of vaccine recipients, P=0.05). Not perfect, but still a great result.

Note: These data were also presented at earlier IDSA (2010) and ICAAC (2011) meetings.

Source: RL Atmar et al. NEJM Dec 8, 2011

Update: This news is pretty good too. (Massive drop in in-hospital deaths...in Canada)

STAR*ICU study published: Barrier precautions not effective

It's only one study. Everybody take a deep breath. OK, exhale.

You might have already heard about this study and you might even know the results.  Someday, someone might discuss how this study was designed, and why the investigators decided to ship all of the microbiology specimens to NIH for processing resulting in a 5-day test turn-around time.  Someday, someone might explain why this study took 4 years to publish and the saga behind its eventual publication in the NEJM.  Someday, someone might even discuss how the difficulties completing this study might be hindering NIAID from funding other infection prevention clinical studies.  Someday, someone.

What can I say about the study?  Barrier precautions (ie. gloves or gowns/gloves) are ineffective in halting the transmission of MRSA and VRE in ICU settings.

Methods: The cluster-randomized trial (ie a largish quasi-experimental study but with a cool fancy name - see my "Random note" below) was completed in 2006 with the intervention lasting 6 months from March to August 2006. There were 10 intervention ICUs and 8 control ICUs.

Random note: There were 18 ICUs in this study, so it's somewhat like an 18-person RCT.  With such small numbers you can't expect that all measured and unmeasured confounders to be randomly distributed between the intervention and control ICUs. Thus, this is more like a large QE study than a standard RCT and needs to be analyzed as a QE study using multivariable regression controlling for known sources of confounding. Don't believe me? Check out Table 2 to see how different the intervention and control arms were in regards to topical and systemic antimicrobial exposure. You would not typically expect these "significant" differences in a large RCT (or large cluster-RCT).

Microbiology: Nasal swabs for MRSA surveillance cultures and stool or perianal swabs for VRE surveillance cultures were obtained from all patients within 2 days after their admission to the ICU, weekly thereafter, and within 2 days before or after their discharge from the ICU. Swabs were shipped overnight, 6 days a week, to the NIH. The mean number of days from obtaining surveillance cultures to reporting of results was 5.2 days.  I would have liked to see this reported in median days and I would also have liked this number to be reported from time of admission and not time from obtaining the culture since 2 days could pass between admission and obtaining the culture.

Planned Intervention: Known colonized or infected patients were placed on contact precautions. All other patients were placed on universal gloving from the time of admission until their discharge or until the results of surveillance cultures results returned. If surveillance cultures were positive, patients were upgraded to contact precautions (gowns/gloves) and if they were negative, they were downgraded to standard precautions.

Actual intervention as implemented: In the intervention ICUs, 92% of the ICU-days were spent under barrier precautions (51% contact precautions and 43% universal gloving) while in the control ICUs, 38% of ICU-days were spent under contact precautions.  Thus, indepedent of what anyone says, this study is about whether increasing barrier precautions from 38% to 92% reduces transmission. Also, 4 times as many patients in the intervention group were exposed to a topical antimicrobial (e.g. mupirocin), 12% vs 3.2%. Now, some will say that there wasn't 100% compliance with these interventions. I agree, this is not an efficacy trial.  As Ebb and I said in our JAMA commentary yesterday, cluster-randomized trials are real-world effectiveness trials in the domain of infection prevention.

Compliance: Overall, 47% of contacts in the intervention arm occurred with clean gloves and exit hand hygiene compliance vs 25% in the control ICUs. Compliance with contact precautions was relatively good in the intervention ICUs: gloves 82% and gowns 77%. Hand-hygiene compliance was also higher in the intervention vs. control ICUs (69% vs 59%)

MRSA or VRE colonization or infection: The mean incidence of MRSA or VRE per 1000 patient days at risk was actually higher in the intervention arm than the control arm (40.4 vs 35.6, p=0.35) but this was not statistically significant.

My thoughts:  How can this study not find a benefit when so many others have? Since most of the previous studies were uncontrolled quasi-experimental studies and we know that uncontrolled QE studies can over-estimate the measure of effect, it is possible that barrier precautions don't work.  It is also possible that other factors need to be included in any MDRO prevention program including attention to environmental cleaning and far higher compliance with the hand hygiene and contact precautions. However, the compliance rates reported in this study are not abnormally low (at least at the mean/median). Finally, perhaps decolonization is needed to achieve the results (at least for MRSA) that we've seen in other studies.

Another criticism that we've heard and will hear again is that the turn-around time for the microbiology was too-long.  This is mostly a red herring.  Since we have little evidence that gowns add much to gloves, and 92% of contacts in the intervention arm occurred with gloves, this study had FAR better glove compliance than we would expect with any typical ADI program in the real world.  The use of universal gloves in the pre-result period in the intervention arm really saved this study and, thus, it provides VERY useful information and should not be discounted. A quicker test turn-around would not have magically led to reduced transmission. Sorry.

Again, this is one study and it shouldn't be the last.  AHRQ is funding some very important MRSA (and VRE) prevention trials that many of us are involved with and I hope the publication of this paper won't discourage AHRQ (or NIH or CDC or VA) from funding these large and important studies.

What this study really tells us is that we can't fall back on legislative mandates in MDRO prevention and we must continue to search for the right combination of interventions along with developing better implementation strategies. Don't stop with the STAR*ICU study. Let this be the beginning.


Huskins W.C. et al. NEJM April 14, 2011