Essential reading on Candida auris

During my intern rotation on the University of Virginia bone marrow transplant unit, I convinced myself that a Candida krusei epidemic was brewing. One of my patients was infected, and the bug was (and is inherently) resistant to fluconazole, a drug that had only recently been introduced (yes, I’m old—the year was 1990). This never really came to pass—despite 30 years of widespread fluconazole use, C. krusei still accounts for < 5% of invasive candidiasis, and outbreaks are rare. 

Now, a Candida species that wasn’t even described a decade ago is emerging as a major problem in ICUs around the world. The Candida auris story is fascinating, puzzling, and concerning. For reasons nobody understands, the species emerged (or began to recognized) almost simultaneously on three different continents. Although risk factors for invasive C. auris are similar to those for other causes of invasive candidiasis (ICU stay, antibiotic exposure, device use), it also features high rates of antifungal resistance, persistence on environmental surfaces, resistance to commonly used disinfectants, frequent transmission in ICU environments, and has thus caused several large, difficult-to-control outbreaks.

If you want to catch up on this emerging pathogen without spending hours on a literature review, there’s an excellent summary publication now out in Clinical Microbiology Reviews from Anna Jeffery-Smith and colleagues. See Table 4 for a summary of infection prevention recommendations from UK, US, EU and South Africa.

De-implementation and Noninferiority in Infection Control Studies

De-implementation or "stopping practices that lack supporting evidence" is a popular topic in infection control circles. In fact, just yesterday I read a discussion where the authors suggested we no longer need to practice hand hygiene after removing gloves when caring for patients with CDI. I guess there aren't randomized trials - you can't be serious!

Which brings me to a recent review in the NEJM by Laura Mauri and Ralph D'Agostino titled "Challenges in the Design and Interpretation of Noninferiority Trials."  This review is very well written - perhaps required reading for epidemiology students well written. In infection control, it is important to recognize that most de-implementation studies are really non-inferiority trials. For example, when we discontinue contact precautions, we are really suggesting that "stopping contact precautions" is non-inferior to continuing contact precautions in preventing MDRO transmission - of course ignoring that compliance with contact precautions is probably so poor that they are basically the same intervention!

In the contact precautions example, we would be testing whether stopping contact precautions "is not worse than the control (continuing contact precautions) by an acceptably small amount, with a given degree of confidence." The null hypothesis would be that discontinuing contact precautions leads to higher transmission of MDRO (i.e. is worse) and rejection of the null hypothesis is used to support the claim that discontinuing CP is noninferior. Here I suggest you stare at Figure 1 for a bit (probably easier to read in the paper with the description of each condition, but I have included it below anyway)

Further discussion about the design and analysis of these trials is way beyond the scope of a humble blog post; however, the authors include nice descriptions of methods for deriving noninferiority margins, the "constancy assumption" and statistical analysis approaches. But their 6th and 7th components of noninferiority trials are worth mentioning from an infection control standpoint:

6) Adequate ascertainment of outcomes: The authors write that "incomplete or inaccurate ascertainment of outcomes, as a result of loss to follow-up, treatment crossover or nonadherence, or outcomes that are difficult to measure or subjective, may cause the treatments being compared to falsely appear similar."  I would suggest that studies that seek to de-implement contact precautions that do not include admission/discharge surveillance cultures seeking to detect transmission events fail this criteria.

7) Issues with "Intention-to-Treat" in noninferiority designs: In a superiority studies (typical RCTs), intention-to-treat analysis, where anyone who receives the treatment is included even if they get one dose, is the gold standard. The authors write: "In a noninferiority study, however, if some patients did not receive the full course of the assigned treatment, an intention-to-treat analysis may produce a bias toward a false positive conclusion of noninferiority by narrowing the difference between the treatments. In some instances, a per-protocol analysis, which excludes patients who did not meet the inclusion criteria or did not receive the randomized, per-protocol assignment, may be preferable in a noninferiority trial. However, a per-protocol analysis may include fewer participants and introduce postrandomization bias. In general, both the intention-to-treat and per-protocol data sets are important. We suggest analyzing both sets and examining the results for consistency."

Just some things to think about as we read the coming wave of de-implementation studies in infection control including diagnostic stewardship.

Construct Validity and Infection Control Nihilism

As describe by Cook and Campbell (1979) there are four components of validity: internal validity, external validity, statistical conclusion validity and construct validity. Internal validity relates to whether there is a causal relationship between an exposure (e.g. contact precautions) and an outcome (MRSA infections) that is free of bias. Randomization is thought to improve internal validity through reduction in confounding associated with unmeasured factors. External validity describes the generalizability of the findings to other populations (quasi-experimental studies typically have higher generalizability vs RCTs). Statistical validity is concerned with covariation between exposure and outcome and strength of the association (e.g. Type 1 and Type II error). All types of studies can have high or low statistical validity – it is independent of study design.

Finally, construct validity describes whether a test measures what it claims to be measuring. For example, if you claim a person is ESBL negative, is she actually free of ESBL colonization or infection and will not develop an ESBL infection in the future. As you can see, without high construct validity, all the other components of validity are unimportant. If your study design or microbiological method is unable to detect ESBL properly, it is irrelevant if you’ve completed a cluster-RCT or whether your p-value is significant. Thus, validity theory defines construct validity as the primary concern, subsuming all other types of validity evidence.

Which brings me to studies claiming contact precautions don’t prevent MRSA, ESBL or VRE. Let’s think about MRSA. If an MRSA negative patient is admitted to a hospital with a 4 day length of stay. On average (normal distribution) that patient would be expected to acquire MRSA at the end of day 2. Thus, they would have to go from acquisition to infection over the next two days prior to discharge for most studies to prove she didn’t acquire MRSA. Would two days even be long enough for her to have a positive nasal swab? So, how sensitive are surveillance cultures or clinical cultures at detecting this event. I’d suggest not sensitive at all. Thus, to have strong construct validity in any study looking at the benefits of eliminating contact precautions, the study would have to track patients for a prolonged period of time (months) and in particular look at infections that manifest during subsequent admissions including those to long term care facilities.

So, before we can make claims about the benefits or lack of benefits of infection control interventions we need to design studies with high construct validity. I would suggest that our ability to respond to current MDR-bacterial pandemics will foremost depend on us designing studies with strong construct validity. Pathogens will continue to harm our patients until we identify methods to halt their spread. The current trend towards infection control nihilism that is manifesting with those eliminating contact precautions based on studies with poor construct validity and typically very poor statistical validity (underpowered) is harming our patients – often after they are discharged from our facilities.

When will the p-value finally die?

Over the past 7 years!, I've written about my concerns regarding the misuse of p-values, which I've dubbed "death by p-value." First and foremost, no one really understands what a p-value means and perhaps more importantly, a singular focus on a p-value threshold detracts from important concepts such as effect measures (e.g. hazard ratios) or the continuous nature of confidence intervals.

Recently, I've seen an increasing number of calls to hasten the death of the p-value from folks such as Ken Rothman (you might have read one of his books?) and Miguel Hernan. Professor Hernan has a request on twitter saying "statistical significance must go" where he is asking for examples of outrageous misuses of p-values. He's also suggested that "if your journal still uses 'statistical significance' in 2017, retire your statistical consultant."

Below, I've posted a letter to the editor by Professor Rothman and colleagues that he recently circulated. It has a nice brief example of how p-values are frequently misused. The key sentence, I believe, is this one: "given the expected bias toward a null result that comes from non-adherence coupled with an intent-to-treat analysis, the interpretation of the authors and editorialists is perplexing." I'll ask, are there situations such as studies of contact precautions or hand hygiene interventions where they would be analyzed using an intent-to-treat analysis and non-adherence levels might be high?

I'll leave you with this video on p-values from Carl Bergstrom at University of Washington. He and Jevin West have a course and planned book titled "Calling Bullshit: in the age of big data," which has been well received. And to connect this to MDRO, Carl is the author of a 2004 PNAS paper "Ecological theory suggests that antimicrobial cycling will not reduce antimicrobial resistance in hospitals", which is a nice example of how math models can improve our understanding of antimicrobial stewardship interventions. It's still worth reading.

Part 2: Influenza Vaccine Effectiveness - The Test-Negative Control Design

Before going further, I want to highlight the purpose of yesterday's and today's post. Occasionally, it is important to question standard practice in epidemiology when something doesn't make sense to you. I've experienced this first hand 3-4 times in infectious disease epidemiology and consider it an important exercise and critically important to our field. You can read this post if you want to read more about how we've questioned the ID epidemiology dogma around control-group selection, quasi-experimental design methodology and inappropriately controlling for intermediates in the causal-pathway of outcomes studies. Of course, this doesn't prove anything in this instance but rather explains my motive.

Yesterday, I posted my initial thoughts on how the CDC calculated the preliminary influenza vaccine effectiveness for the 2012-2013. I've now had several informal discussions on the phone and in person with several vaccine experts and am starting to understand how/why they conducted the study the way they did. I'm even more convinced now that design considerations (e.g. which patients are included in the cohort) are influencing the statistical analysis, which should be separate considerations. I will explain this further below.

It appears that the current standard study design in vaccine effectiveness (VE) is called the "test-negative" control design. This design is a modified case-control design where controls are required to have been tested for influenza but have been found "test negative". This appears to be a useful design when the specificity of the diagnostic test for influenza is low (Orenstein EW 2007), which is not the case in the current MMWR report. Another reason mentioned for selecting this design is that it is thought to reduce the bias, since vaccinated individuals may be more like to seek medical care vs. unvaccinated individuals. However, this benefit would apply to any patient seen in the outpatient clinic and not necessarily require diagnostic testing for influenza. 

In favor of cohort studies over case-control or test-negative designs, is the fact that "if specificity was 100%, the cohort method VE estimate was equivalent to the true VE, regardless of test sensitivity or ARs of influenza- and non-influenza-ILIs." (Orenstein EW 2007) The Orenstein paper's stated purpose was to study the impact of sensitivity and specificity of influenza testing on VE estimates across these study designs. Any benefits for the test-negative approach appear to evaporate when using a great diagnostic test, like the rt-PCR used in the MMWR report.

From what I can tell, the Orenstein paper is frequently cited to justify the test-negative design, but given current conditions, the design doesn't seem to be as useful as it once was. I also remain unconvinced that a cohort of patients already being seen in a doctor's office can tell us anything about risk factors for "medically attended" influenza. More importantly, I'm still very concerned with how the cohort was analyzed. Even if there are very good reasons to enroll a cohort of patients who presented to a clinic in order to avoid bias from vaccinated patients presenting differentially to clinics and even if all were tested for influenza, it doesn't follow that you need to use an odds-ratio approach to measure vaccine effectiveness when the relative-risk approach is more accurate. It appears that correcting one wrong (differential medical care seeking in vaccinated vs unvaccinated) is leading to a countervailing wrong when the analysis is done incorrectly. If they had just called this a cohort of tested patients or a "tested cohort", perhaps this wouldn't have happened.

Oh, and I'm still not sure why they are excluding influenza B positive patients from their VE calculation  for influenza A. Shouldn't they have to look at each A strain separately then? Well, that's another post for another day.

Don't let authors control for factors in the causal pathway!

A downside of being an epidemiologist who thinks about antibiotic resistance 24-7 is that eventually it's hard to read the literature for fear of seeing another paper where authors make a critical mistake.  It's unfortunate, because the review process should catch these common errors and at least make the authors mention limitations in their discussion sections.  A case in point is a recent paper by Chen et al. in CID that sought to assess the impact of MRSA SCCmec type and vancomycin MIC on treatment failure. In this case, treatment failure was defined as all-cause 30-day mortality, persistent bacteremia, or recurrent bacteremia. They provide nice graphs showing higher mortality for hospital associated-MRSA strains (SCCmec type I, II, III) and for strains with vancomycin MICs > 2. So far, so good. But then, I looked at their multivariable model.

As expected, they controlled for septic shock or severe sepsis in their model. They should not have done this. If it were a comparative effectiveness study assessing various treatment outcomes, it would be appropriate to collect severity of illness including evidence of sepsis before treatment initiation and control for it in the model. However, Chen's study was only looking at outcomes based solely on strain differences. Thus, they should never have controlled for shock or sepsis. How else would a bacteremic patient die if not through sepsis? For a longer description of why controlling for factors in the causal pathway is suboptimal, see my post from 3/19/2012. I have pasted the key sections below. Note: We have been writing letters and review articles in CID pointing this issue out since the year 2000.

____
March 19, 2012: My first ever publication, and in some ways still my favorite, was a letter to the editor of CID that I wrote in 2000 pointing out a common flaw in outcome studies of infectious diseases. In the letter, I discussed a paper that looked at the outcomes (death) associated with methicillin-resistance in patients with S. aureus bacteremia. In the analysis, the authors controlled for septic shock in their regression model. I pointed out that shock is in the causal pathway between infection and death and, therefore, should not be controlled for in regression in models. This would be like controlling for car accidents when looking at the association between cell phone use and death. In infectious diseases, if you remove shock from the causal pathway, it is hard to see how you might otherwise die.

The error of controlling for intermediates is frequently repeated in ID outcome studies when, for example, authors control for illness severity using the APACHE score. If the APACHE is measured after the infection manifests, this variable would be in the causal pathway and should not be controlled for in the regression model. The APACHE should be measured before the infection manifests, as we did here. Jessina McGregor and JJ Furuno (both now at Oregon State) published a nice systematic review on optimal methods for ID outcome studies in CID back in 2007. Wouter Rottier (with Marc Bonten) just published a meta-analysis looking at the impact of confounders and intermediates (factors in the causal pathway) on ESBL-bacteremia outcomes. (JAC, March 5, 2012) I highly recommend that you read these studies prior to undertaking an ID outcome study.

The mystery of the Staphylococcus in retreat

Evidence keeps accumulating that Staphylococcus aureus disease, for whatever reason(s), is declining in incidence. The latest massive study to illustrate this trend is courtesy of the U.S. Military Health System and is published this week in JAMA. Using their integrated electronic medical record, military epidemiologists demonstrate that both community- and hospital-onset S. aureus bloodstream infections (MRSA and MSSA) declined in incidence between 2005-2010, and that the proportion of S. aureus skin/soft tissue infections due to MRSA declined beginning in 2007. While it might not be generalizable, the Military Health System is very large, encompassing active duty military, retirees, reservists and their families, and this study included over 56 million person-years of observation.

One strength of this study is that it shows not only healthcare-associated but also community S. aureus disease to be declining, a decline that is difficult to attribute to hospital-focused interventions. The reports now from NHSN, the Active Bacterial Core Surveillance program, the United Kingdom, the EARSS system, and now the military all speak to our ignorance of the complex interplay between S. aureus and its human reservoir. Successive long cycles of waxing and waning epidemic spread have been, and will be, the rule. 

For a nice example, check out this study from Oxfordshire hospitals demonstrating how their MRSA incidence began to drop well in advance of intensified infection control interventions, and how this drop coincided with the rise and fall of two competing strain types.

Image from Planet Science

Not reading the blog on Independence Day? You're missing this...

From Jennifer Gardy - an IDWeek Speaker and Microbiologist

Epidemiological surveillance testing is a waste

Only last month, we posted on the Washington state pertussis outbreak. Back then, there were 640 reported cases through March.  A month later there are 1284 total cases, up from 128 the prior year.  To me, these seem like important data. For one, we've used them to sound the alarm over low vaccination rates.  Now an article in today's NY Times highlights the impact that state budget woes have had on the public health infrastructure and how this has blunted the epidemic response.

Skagit County's (pop: 117,000) top medical officer, ER doc Dr. Howard Leibrand, has some choice words for pertussis testing, which I've pasted below:

If the signs are there, he said — especially a persistent, deep cough and indication of contact with a confirmed victim — doctors should simply treat patients with antibiotics. The pertussis test can cost up to $400 and delay treatment by days. About 14.6 percent of Skagit County residents have no health insurance, according to a state study conducted last year, up from 11.6 percent in 2008. 

“There has been half a million dollars spent on testing in this county,” Dr. Leibrand said late last week. “Do you know how much vaccination you can buy for half a million dollars?” And testing, he added, benefits only the epidemiologists, not the patients. “It’s an outrageous way to spend your health care dollar.” 

Since antibiotic overuse has no cost or downsides from a public health perspective and we don't need to understand the scope of the epidemic, this is probably cool.

Image source: www.healthheritageresearch.com

If you're criticizing methods, your methods better be strong

This morning I innocently set out to review a new study in ARIC that looks at methodological issues in C. difficile outcomes studies assessing the association between infection and excess hospital stay.  The authors make a very important point in the paper: "The studies did not collect data concerning the time of onset of CDI; therefore, it is not possible to exclude the possibility of reverse causality, in which longer lengths of hospitalisation may have increased the risk of CDI." They then go on to discuss time-dependent bias and recommend the use of multi-state models...we've made this point before.

What caused me the emotional anguish was their Table 1 that lists the 16 studies included in the review. In column two, three of the studies are listed as retrospective case-control studies...what??!!  How can you do a case-control study with an outcome being hospital length of stay?  I'm pretty sure you can't, at least not easily.

You see, a case-control study requires identifying an outcome and looking back for risk-factors associated with the outcome. For example, you could look at 50 people that died (and 100 that didn't die) and see how many had CDI, to determine if CDI was a risk-factor for death.  For length of stay, I can't even make up a good way to do a case-control study.  Would you find patients who stay >14 days and compare them to patients that stay <14 days? The three studies listed MUST have been cohort studies, so why did the authors of the ARIC paper seeking to teach us about proper methods for outcomes studies list them as case-control studies? Perhaps they pasted them from the individual studies' methods sections?

Just to confirm this, I looked at all three papers: (Ananthakrishnan (2008), Bajaj (2010), Pepin 2005).

Pepin: from the Methods  - "We compared mortality and total length of hospital stay among inpatients in whom nosocomial CDAD developed and among control subjects without CDAD."  This is a cohort study - exposed and unexposed to CDAD, looking forward to outcomes.  Just because they incorrectly use the word control, doesn't make this a case-control study. (Strike 1)

Ananthakrishnan: "Our primary case group (C difficile–IBD group) included patients who had a primary diagnosis of C difficile colitis and a secondary diagnosis of either Chrohn's Disease (CD) or Ulcerative Colitis (UC). Patients admitted to hospital with a primary diagnosis of CD or UC without a diagnosis of C. difficile colitis formed one of our comparison groups (IBD group)." Another cohort study - defined by the exposure to C. difficile and not by the outcome. (Strike 2)

Bajaj: "Among the cohort of hospitalized patients with any diagnosis of cirrhosis, co-existing diagnosis of C. difficile was associated with significantly greater in-hospital mortality."(Strike 3)

The error of incorrectly describing cohort studies as case-control studies is very common - enough to be a pandemic.  The journal CID is one of the worst offenders. One of my favorite examples is this recent VAP treatment cohort study described as a case-control study in the title!  When I've mentioned the problem to CID editors, they suggest I write a letter.  You can't write letters for each issue.

OK: Cohort - exposed/unexposed look forward in time to the outcome. Case-control - find outcomes and look back for exposures. Got it?  EOR

Optimal Epidemiological Methods for Infection Prevention Studies

I've spent the last 12+ years writing about epidemiological methods for conducting risk-factor, outcomes and intervention studies of hospital-acquired infections. Instead of always reviewing the latest and greatest studies, I thought it might be fun to look back at some of the epi-methods papers that many of my colleagues and I have published since 2000. I think we've made some important contributions to infection prevention research, so it's kinda fun to look back at these. Over the next month or so, I hope to review other epi-methods topics that I think are particularly relevant to the study of hospital-acquired infections.

A decade ago, Anthony Harris and Yehuda Carmeli (and other folks) outlined optimal control-group selection in risk-factor studies for antibiotic resistant infections. (see here, here and here) Prior to these important studies, authors would frequently use patients infected with the susceptible organism as the control group. For example, when looking at risks for MRSA they would select MSSA controls, which is incorrect.  Unfortunately, many authors still select the wrong control group and unknowingly publish conditional odds-ratios.  I will discuss this more in a later post.

My first ever publication, and in some ways still my favorite, was a letter to the editor of CID that I wrote in 2000 pointing out a common flaw in outcome studies of infectious diseases. In the letter, I discussed a paper that looked at the outcomes (death) associated with methicillin-resistance in patients with S. aureus bacteremia. In the analysis, the authors controlled for septic shock in their regression model. I pointed out that shock is in the causal pathway between infection and death and, therefore, should not be controlled for in regression in models. This would be like controlling for car accidents when looking at the association between cell phone use and death. In infectious diseases, if you remove shock from the causal pathway, it is hard to see how you might otherwise die.

The error of controlling for intermediates is frequently repeated in ID outcome studies when, for example, authors control for illness severity using the APACHE score. If the APACHE is measured after the infection manifests, this variable would be in the causal pathway and should not be controlled for in the regression model. The APACHE should be measured before the infection manifests, as we did here. Jessina McGregor and JJ Furuno (both now at Oregon State) published a nice systematic review on optimal methods for ID outcome studies in CID back in 2007. Wouter Rottier (with Marc Bonten) just published a meta-analysis looking at the impact of confounders and intermediates (factors in the causal pathway) on ESBL-bacteremia outcomes. (JAC, March 5, 2012) I highly recommend that you read these studies prior to undertaking an ID outcome study.

Anthony Harris and I have also written extensively on the appropriate use and analysis of quasi-experimental studies looking at interventions to prevent hospital-acquired infections. In a trilogy of CID review articles, we reviewed the optimal quasi-experimental designs (2004), the frequency of each design's use (2005) and appropriate statistical analysis of time-series data (2007). If you're planning on doing a non-randomized study of any infection prevention intervention, please look these papers over. Following the optimal methods outlined in these reviews will improve your studies and also increase the chances that your intervention study's results will make the grade and be included in future systematic reviews, such as Cochrane reviews.

Image Reference: DA Grimes, Lancet 2002;359:57-61

(More) Other Advice to Young Epidemiologists

1) Don't call anyone an idiot.
2) Question subjects carefully.



For "other advice" see ICHE 2006

Lying About Prognosis Might Not Be Lying

There is a lot of chatter (or here) about Lisa Iezzoni's study on physician openness and honesty, that was recently published in Health Affairs.  We've discussed the importance of disclosing medical errors numerous times, as well as the importance of disclosing financial conflicts of interest, so physicians who aren't honest in those domains, will get little sympathy here.  However, one aspect of the survey findings, I think, deserves more discussion, namely the disclosure of prognosis.

To quote from the article: "...more than half said that they had described a prognosis more positively than the facts warranted."

Is this really "lying"?  What is the importance of disclosing mean, median or mode survival?  Will patients or families even understand the difference?  How do you explain a normal or skewed survival curve?  If you can't describe the distribution or they don't understand it, is that dishonesty? Importantly, how does this all impact "hope"?

One article, written many years ago (1985) by Stephen J Gould, the Harvard evolutionary biologist, does a better job describing why "The Median Isn't the Message" than anything I could write. He tells how he reacted to his 1982 mesothelioma diagnosis. He lived until 2002. This essay has been highly influential to me throughout my medical career.

Conflicting Results in Clinical Trials: the APC Example

Dan posted last week on activated protein C (Xigris) being withdrawn from the market.  David Rind (Evidence in Medicine blog) has posted his thoughts on what might have been behind the initial positive study in 2001 and subsequent negative studies.  It is worth a read.

David Rind: APC and Conflicting Trials (10/29/2011)

Null and Void

xkcd.com