Metrics, Decision Makers and Hospital Epidemiologists
This is a guest post by Silvia Munoz-Price, MD, PhD, Enterprise Epidemiologist and Professor of Medicine, Division of Infectious Diseases, Froedtert and the Medical College of Wisconsin.
During the past few weeks I have been mulling over two issues
related to Quality and Infection Control that have to do with metrics, their
use, and the impact that they have in our hospitals. Two separate topics but
related to a certain degree.
1. My hospital has a strong Quality Department and a
stronger culture of safety. For the most part we have very few serious
infections and even fewer MRSA, VRE, and almost no multidrug resistant Gram
negative rods. Our compliance with hand hygiene is 73% (as per >5000
observations collected over a couple of months by managers from units that are
not their own and concordant readings obtained by the Infection Control Department.
I am trying to tell you that 73% is probably close to accurate). From my
perspective, if indeed true, this is an acceptable rate. I think we should concentrate
on preserving this rate and maybe focus on areas/providers that need more
attention. However, my hospital wants this number to be close to 100%. So, here
are my questions: should we invest more time, effort, and money to push for higher
compliance with hand hygiene? What would be the return on investment of increasing
our hand hygiene compliance and how could we measure its impact? What type of
infections would we prevent by increasing hand hygiene compliance? Yes, our
CLABSIs and CAUTI rates could be better, but would they noticeably improve by
pushing hand hygiene above 80%? (I’m intentionally not discussing C. difficile infections as these have a
different etiology in my hospital). And most importantly what other
interventions would we be overlooking by focusing so much on hand hygiene? Who
decides where to allocate time, effort and money?
2. Over the past couple of decades I have observed a shift
in Infection Control throughout the various places I have worked at from
minimal interest/resources given to Infection Control, to a phase where more
visibility was given to Infection Control matters --probably due to metrics--to
the allocation of more interest/resources, to publicly reported metrics, to institutional
notoriety and reimbursements linked to metrics, to a tunneled vision about
metrics. On this topic, many years ago I observed a human behavior that since I
have observed multiple times: when we started placing fluorescent powder on surfaces
and giving feedback to providers on the degree of removal, cleaning rates
improved (https://www.ncbi.nlm.nih.gov/pubmed/21460514). Feedback was then
given weekly and in a public manner, with a substantial amount of pressure
placed on EVS to improve. The numbers got much better. We eventually figured out that EVS providers bought their own ultraviolet lamps to find the
fluorescent markers and spot clean them. What happened? The metric made sense at the
beginning, initially it probably achieved its intended outcome (more frequent
cleaning) but then a threshold was crossed, after which what mattered was the
metric (as a number) and no longer the intended outcome (more frequent
cleaning). I think the same is happening
to our hospitals. Metrics were beneficial initially (more resources for
Infection Control, more visibility, more engagement of staff, lower infections)
but I would argue that we are on the other phase of the metrics (unintended
consequences). All the pressure we are
placing for lower and lower metrics (get to zero CAUTIs!) might be backfiring.
We are deploying large teams and resources to decrease certain conditions which
is shifting our capability to address issues that might be equally or even more
important in our respective hospitals
(e.g. post craniotomy SSIs). So, who decides how to allocate our limited
resources? I would argue that Quality is now the main decision maker and that
these decisions are strongly guided by publicly reported metrics. Is this good
or bad for our patients? I am not sure. Is this good for the field of Hospital
Epidemiology? I don't think it is. Hospital Epidemiologists need to re-think
their involvement in Quality and become their hospitals’ Chief Quality
Officers. That is the only way I see that we will impart some sense to the
decisions of resource allocation. However, the opportunity for our specialty to
lead in this field might have already passed.
Excellent post as always Dr. Muñoz-Price. My two thoughts. First, my colleague pointed me to this interview of Mark Chassin (Joint Commission) with this quote that I think explains the "100% target: "It’s interesting that a number of the hospitals were misled by faulty data to believe that they were doing as well as, say, 85%, at baseline rather than 48%."
ReplyDeleteref http://www.jointcommissionjournal.com/article/S1553-7250(10)36070-3/abstract
It was this study that he described that I believe led JC to remove a hand hygiene target
You last comment reminds me of Goodhart's law which has been paraphrased as saying: "When a measure becomes a target, it ceases to be a good measure."
So I am not sure QI can really get around this problem unless we get rid of public reporting and use audits to only improve local control efforts - like the old NNIS days. Not going to happen sadly.
Just opened the site and found it very timely, thank you Sylvia. Sad reality - despite my best efforts at teaching hospital epidemiology to Masters students, literally last week not a single student picked Hand Hygiene compliance as a topic of interest in our pro/con debates about investing effort to improve quality and prevent infections! I may have failed, but I think getting from 75% compliance to closer to 100% will take patient/family pressure/engagement (perhaps through satisfaction surveys linked to payment). Although the pathway from hosptial epidemiology to CQO is a logical pathway - payment drivers will trump (is there another word I can use...) efforts that "should be done" I think our task may be to translate (or create evidence) that what should be done is cost effective at moving the metric (rather than the shortcuts).
ReplyDeleteThank you for this blog, Dr. Munoz-Price. My fellow Dr. Madiha Tahir has a poster at SHEA Spring 2018 on our investigation of bacteremia from peripheral IVs. MSSA bacteremia from a PIV and MRSA bacteremia from a central line both harm the patient, but only one will be publicly reported and result in a lower Hospital Compare quality racing and lower CMS reimbursement for the hospital. I acknowledge that reporting does lead to accountability, but we need non-gameable outcomes measures that are important to patients and better, patient-level risk adjustment.
ReplyDeleteThanks for the feedback Amy, Scott and Eli.
ReplyDeleteSo many issues here:
1. Amy: I think there is no way around to having negative repercusions of metrics. Reading your post reminded me of the "balloon effect" that we see with bacteria when we switch antibiotic pressure from one class to another. Think about it...even bacteria try to circunvent an intervention!
2. Scott: The percentage of hospital epidemiologists that actually push the field forward is relatively small within an already small group. We need to do something with our specialty before it dies. Also, even with science backing us up, if Joint Commission decides that A should be done then all administrators will push for A even though science might not back that up. You are probably the right person to ask this: Do Hospital Epidemiologists advice Joint Commission and CMS on what they should observe? Sometimes the feedback seems random and rather individual-based.
Regarding your students, I know that people are not going to like this statement but here it is: even though very important, hand hygiene is not sexy, so I can understand why the decided to pursue other choices. This is one more reason why we should mix up our field with the super hot topic of the microbiome, transcriptomics, resistome, virome, etc. Lets put some spice to what we do and lets make it a tad more difficult for non trained individuals to think they can do our jobs.
Thanks for the thought-provoking article Dr. Muñoz-Price! Your question of "Who decides where to allocate time, effort and money?" is a critical one, and I agree that scientists/epidemiologists need be a part of that conversation. Everything has a cost (even hand-washing) and the cost/benefit should be compared to make a decision; not just chasing 100% compliance or a (potentially arbitrary) target score that may not have a clinically-meaningful impact. And, more importantly, implemented practices should be based on scientific data in concert with best practices for a comprehensive approach and not just bundling everything together. It really takes a team effort to get there because no single person can synthesize all of the data out there.
ReplyDeleteHow have you been able to elevate this point to win over support for pursuing new approaches?