Metrics, Decision Makers and Hospital Epidemiologists

This is a guest post by Silvia Munoz-Price, MD, PhD, Enterprise Epidemiologist and Professor of Medicine, Division of Infectious Diseases, Froedtert and the Medical College of Wisconsin.


During the past few weeks I have been mulling over two issues related to Quality and Infection Control that have to do with metrics, their use, and the impact that they have in our hospitals. Two separate topics but related to a certain degree.

1. My hospital has a strong Quality Department and a stronger culture of safety. For the most part we have very few serious infections and even fewer MRSA, VRE, and almost no multidrug resistant Gram negative rods. Our compliance with hand hygiene is 73% (as per >5000 observations collected over a couple of months by managers from units that are not their own and concordant readings obtained by the Infection Control Department. I am trying to tell you that 73% is probably close to accurate). From my perspective, if indeed true, this is an acceptable rate. I think we should concentrate on preserving this rate and maybe focus on areas/providers that need more attention. However, my hospital wants this number to be close to 100%. So, here are my questions: should we invest more time, effort, and money to push for higher compliance with hand hygiene? What would be the return on investment of increasing our hand hygiene compliance and how could we measure its impact? What type of infections would we prevent by increasing hand hygiene compliance? Yes, our CLABSIs and CAUTI rates could be better, but would they noticeably improve by pushing hand hygiene above 80%? (I’m intentionally not discussing C. difficile infections as these have a different etiology in my hospital). And most importantly what other interventions would we be overlooking by focusing so much on hand hygiene? Who decides where to allocate time, effort and money?

2. Over the past couple of decades I have observed a shift in Infection Control throughout the various places I have worked at from minimal interest/resources given to Infection Control, to a phase where more visibility was given to Infection Control matters --probably due to metrics--to the allocation of more interest/resources, to publicly reported metrics, to institutional notoriety and reimbursements linked to metrics, to a tunneled vision about metrics. On this topic, many years ago I observed a human behavior that since I have observed multiple times: when we started placing fluorescent powder on surfaces and giving feedback to providers on the degree of removal, cleaning rates improved (https://www.ncbi.nlm.nih.gov/pubmed/21460514). Feedback was then given weekly and in a public manner, with a substantial amount of pressure placed on EVS to improve. The numbers got much better. We eventually figured out that EVS providers bought their own ultraviolet lamps to find the fluorescent markers and spot clean them. What happened? The metric made sense at the beginning, initially it probably achieved its intended outcome (more frequent cleaning) but then a threshold was crossed, after which what mattered was the metric (as a number) and no longer the intended outcome (more frequent cleaning).  I think the same is happening to our hospitals. Metrics were beneficial initially (more resources for Infection Control, more visibility, more engagement of staff, lower infections) but I would argue that we are on the other phase of the metrics (unintended consequences).  All the pressure we are placing for lower and lower metrics (get to zero CAUTIs!) might be backfiring. We are deploying large teams and resources to decrease certain conditions which is shifting our capability to address issues that might be equally or even more important in our respective hospitals   (e.g. post craniotomy  SSIs).  So, who decides how to allocate our limited resources? I would argue that Quality is now the main decision maker and that these decisions are strongly guided by publicly reported metrics. Is this good or bad for our patients? I am not sure. Is this good for the field of Hospital Epidemiology? I don't think it is. Hospital Epidemiologists need to re-think their involvement in Quality and become their hospitals’ Chief Quality Officers. That is the only way I see that we will impart some sense to the decisions of resource allocation. However, the opportunity for our specialty to lead in this field might have already passed.     

Comments

  1. Excellent post as always Dr. Muñoz-Price. My two thoughts. First, my colleague pointed me to this interview of Mark Chassin (Joint Commission) with this quote that I think explains the "100% target: "It’s interesting that a number of the hospitals were misled by faulty data to believe that they were doing as well as, say, 85%, at baseline rather than 48%."

    ref http://www.jointcommissionjournal.com/article/S1553-7250(10)36070-3/abstract

    It was this study that he described that I believe led JC to remove a hand hygiene target

    You last comment reminds me of Goodhart's law which has been paraphrased as saying: "When a measure becomes a target, it ceases to be a good measure."

    So I am not sure QI can really get around this problem unless we get rid of public reporting and use audits to only improve local control efforts - like the old NNIS days. Not going to happen sadly.

    ReplyDelete
  2. Just opened the site and found it very timely, thank you Sylvia. Sad reality - despite my best efforts at teaching hospital epidemiology to Masters students, literally last week not a single student picked Hand Hygiene compliance as a topic of interest in our pro/con debates about investing effort to improve quality and prevent infections! I may have failed, but I think getting from 75% compliance to closer to 100% will take patient/family pressure/engagement (perhaps through satisfaction surveys linked to payment). Although the pathway from hosptial epidemiology to CQO is a logical pathway - payment drivers will trump (is there another word I can use...) efforts that "should be done" I think our task may be to translate (or create evidence) that what should be done is cost effective at moving the metric (rather than the shortcuts).

    ReplyDelete
  3. Thank you for this blog, Dr. Munoz-Price. My fellow Dr. Madiha Tahir has a poster at SHEA Spring 2018 on our investigation of bacteremia from peripheral IVs. MSSA bacteremia from a PIV and MRSA bacteremia from a central line both harm the patient, but only one will be publicly reported and result in a lower Hospital Compare quality racing and lower CMS reimbursement for the hospital. I acknowledge that reporting does lead to accountability, but we need non-gameable outcomes measures that are important to patients and better, patient-level risk adjustment.

    ReplyDelete
  4. Thanks for the feedback Amy, Scott and Eli.
    So many issues here:
    1. Amy: I think there is no way around to having negative repercusions of metrics. Reading your post reminded me of the "balloon effect" that we see with bacteria when we switch antibiotic pressure from one class to another. Think about it...even bacteria try to circunvent an intervention!
    2. Scott: The percentage of hospital epidemiologists that actually push the field forward is relatively small within an already small group. We need to do something with our specialty before it dies. Also, even with science backing us up, if Joint Commission decides that A should be done then all administrators will push for A even though science might not back that up. You are probably the right person to ask this: Do Hospital Epidemiologists advice Joint Commission and CMS on what they should observe? Sometimes the feedback seems random and rather individual-based.
    Regarding your students, I know that people are not going to like this statement but here it is: even though very important, hand hygiene is not sexy, so I can understand why the decided to pursue other choices. This is one more reason why we should mix up our field with the super hot topic of the microbiome, transcriptomics, resistome, virome, etc. Lets put some spice to what we do and lets make it a tad more difficult for non trained individuals to think they can do our jobs.

    ReplyDelete
  5. Thanks for the thought-provoking article Dr. Muñoz-Price! Your question of "Who decides where to allocate time, effort and money?" is a critical one, and I agree that scientists/epidemiologists need be a part of that conversation. Everything has a cost (even hand-washing) and the cost/benefit should be compared to make a decision; not just chasing 100% compliance or a (potentially arbitrary) target score that may not have a clinically-meaningful impact. And, more importantly, implemented practices should be based on scientific data in concert with best practices for a comprehensive approach and not just bundling everything together. It really takes a team effort to get there because no single person can synthesize all of the data out there.

    How have you been able to elevate this point to win over support for pursuing new approaches?

    ReplyDelete

Post a Comment

Thanks for submitting your comment to the Controversies blog. To reduce spam, all comments will be reviewed by the blog moderator prior to publishing. However, all legitimate comments will be published, whether they agree with or oppose the content of the post.

Most Read Posts (Last 30 Days)