Construct Validity and Infection Control Nihilism
Finally, construct validity describes whether a test measures what it claims to be measuring. For example, if you claim a person is ESBL negative, is she actually free of ESBL colonization or infection and will not develop an ESBL infection in the future. As you can see, without high construct validity, all the other components of validity are unimportant. If your study design or microbiological method is unable to detect ESBL properly, it is irrelevant if you’ve completed a cluster-RCT or whether your p-value is significant. Thus, validity theory defines construct validity as the primary concern, subsuming all other types of validity evidence.
Which brings me to studies claiming contact precautions don’t prevent MRSA, ESBL or VRE. Let’s think about MRSA. If an MRSA negative patient is admitted to a hospital with a 4 day length of stay. On average (normal distribution) that patient would be expected to acquire MRSA at the end of day 2. Thus, they would have to go from acquisition to infection over the next two days prior to discharge for most studies to prove she didn’t acquire MRSA. Would two days even be long enough for her to have a positive nasal swab? So, how sensitive are surveillance cultures or clinical cultures at detecting this event. I’d suggest not sensitive at all. Thus, to have strong construct validity in any study looking at the benefits of eliminating contact precautions, the study would have to track patients for a prolonged period of time (months) and in particular look at infections that manifest during subsequent admissions including those to long term care facilities.
So, before we can make claims about the benefits or lack of benefits of infection control interventions we need to design studies with high construct validity. I would suggest that our ability to respond to current MDR-bacterial pandemics will foremost depend on us designing studies with strong construct validity. Pathogens will continue to harm our patients until we identify methods to halt their spread. The current trend towards infection control nihilism that is manifesting with those eliminating contact precautions based on studies with poor construct validity and typically very poor statistical validity (underpowered) is harming our patients – often after they are discharged from our facilities.
"No, Donny, these men are nihilists, there's nothing to be afraid of." - Walter, The Big Lebowski
ReplyDeleteI think it is very helpful to put this into the validity framework that you have described. There's clearly a void of quality valid evidence in IP. But from the "nihilist" point of view, do we have evidence with high construct validity that contact precautions DO prevent infection or colonization with MDROs compared to [insert horizontal intervention here] alone?
I'm a big fan of the blog - keep up the great work.
There are many studies that show that healthcare workers caring for colonized patients become contaminated with the resistant bacteria. There are also studies showing that this HCW can then spread bacteria to other patients and their environment. These studies have high construct validity. Thanks for reading!
DeleteInteresting--but I'm not sure you have answered the question Andrew asked. Does application of CP as many institutions currently apply them (to a subset of patients found to carry a specific MDRO) reduce clinically relevant adverse outcomes? If you have good studies with high construct validity that suggest they do, then you should cite them, and also compare them to controlled trials (like STAR*ICU, MOSAR, REDUCE MRSA) that suggest that expansion of the use of CP for specific MDROs doesn't seem to impact these outcomes compared to other interventions (e.g. enhanced HH (MOSAR), CHG bathing (REDUCE MRSA), etc.).
DeleteBecause what I've been hearing recently is: (1) anyone who wants to change the way CP are applied (specifically by reducing their use for MRSA/VRE carriers) is either an "eliminator" or a "nihilist", and (2) it is "unpossible" to design and fund a study large enough to demonstrate that CP are effective (because of power and construct validity issues), so perhaps we shouldn't even try (and furthermore, we should continue to expand the use of CP in the absence of compelling data that they work, even as we struggle with a whole of bunch of related issues (like bed availability and patient throughput, AEs not captured by the IHI trigger tool in ICU settings, etc.)).
Sustainable and effective approaches to prevent bacterial transmission events in healthcare settings are essential. If we could (and soon we may be able to) fully evaluate the microbiota of all patients (on admission and during their stay), we'd likely conclude that such approaches need to be applied to every patient and account for every pathogen (resistant or susceptible to antibiotic X). If this means universal application of some form of barrier precautions (CP), then let's do it. But the strange way we currently apply CP has little-to-no evidentiary foundation, and it is appropriate to question it and to experiment with new approaches.
Ok. I'll go with the Netherlands. That's my study.
DeleteBut I'd also suggest that we need to consider other levels of evidence. Sorry! Hospitals are still the engine of resistance - I still see prior healthcare exposure as a major risk for MDRO so I'm willing to consider other non RCT data in my decision making including mathematical models - probably the highest form of systematic review.
I'd also suggest that there are MANY things we can't study on the planet for cost and feasibility reasons. If we required RCTs to prove things we could just say that we can eliminate Medicaid since there is no RCT showing eliminating Medicaid harms people.
It's been interesting when talking with colleagues here at ICPIC including many from the US. I'd say that the overwhelming feeling is surprise that those on the eliminate contact precautions train were very critical of quasi studies showing benefits of contract precautions but love highlighting quasi studies now showing lack of benefit. We can go into the many issues with why you need far larger studies to show equivalence (the goal for eliminating CP) and they would have different thresholds for construct validity in a future post etc.
Anyway - I'm for keeping Medicaid without an RCT. You?
No RCT...keep Medicaid!
DeleteJust to make a small point regarding the irony in your statement--in this discussion about the quality and value of observational data--that we should not eliminate Medicaid "since there is no RCT showing eliminating Medicaid harms people."
DeleteWhile no RCT for Medicaid may be technically true, a very very good observational study with controlling by instrumental variable has provided a bounty of information (generally with good construct validity, it seems) on the impact of Medicaid:
http://www.nber.org/papers/w17190
http://www.nejm.org/doi/full/10.1056/NEJMsa1212321#t=article
https://www.aeaweb.org/articles?id=10.1257/aer.104.5.322
http://www.nejm.org/doi/full/10.1056/NEJMp1609533#t=article
The findings have been a mixed bag (depending on your perspective) of outcomes related to healthcare utilization, individual costs, and measures of health.
Thanks for the blog and discussion!
ReplyDeleteMore weight to engineered solutions in infection control will remove human error. If PPE is not helping, reduce the cost and apply known solutions like SmartFlo3 ozonated water hand hygiene sinks, self-disinfecting has to be better than once a day cleaning!
It's time to put away reliance on RCTs, 68% of which cannot be reproduced, and take a systematic, pragmatic, component modeling approach to understanding the role of the environment in pathogen exposure and transmission. One such promising approach - Dr Kelly Reynolds @ U of Arizona mathematical modeling - Quantitative Microbial Risk Assessment (QMRA).
ReplyDelete