Surveillance under pressure

There’s a great success story now published online in ICHE. The CDC, using National Nosocomial Infection Surveillance (NNIS) and National Healthcare Safety Network (NHSN) data, estimates that 100-200K central line associated bloodstream infections (CLABSIs) have been prevented since 1990 through implementation of evidence-based prevention practices. This accomplishment should be celebrated as a demonstration of the real progress that has been made in hospital infection prevention. As Mary Dixon-Woods and our fellow blogger Eli point out in an accompanying editorial, however, these results are also a time to reflect on how much surveillance has changed since 1990.

The CLABSI surveillance that we once performed exclusively to guide local prevention efforts is now used for much different purposes, with rates reported publicly and soon to have a real impact on each hospital’s bottom line. The pressure to bring CLABSI rates to “zero”, by any means necessary, gets passed along from hospital administrators to unit directors and infection prevention programs, turning CLABSI rates into what Mary and Eli correctly describe as a “reactive measure.” To quote their editorial,

“the more that organizations are incentivized by the prospect of shaming or financial penalties to decrease sensitivity—and thus not to find cases—the less certain it is that they are reporting a valid assessment of their infection rate”

It is instructive to examine what happens in other professions when intense pressure is brought to bear on a metric. Five minutes on The Google is enough to inform about what happens when law enforcement is under pressure to lower crime rates, or when teachers are under pressure to improve student test scores. Are police officers and teachers more inclined to “cheat” than are those tasked with counting infections in hospitals? Do officers who misclassify a burglary as a theft after receiving a call from a commander really have nothing in common with the IP program that misclassifies a primary CLABSI as secondary after a call from a unit director or hospital administrator?

This increased pressure is also felt at the CDC and NHSN, as a metric that was initially designed for one purpose is now appropriated for very different purposes. We recently performed an email survey of over 50 prominent hospital epidemiologists to gather their opinions about the direction of surveillance over the next decade. The results can be found here. Some of the highlights:

• Over 75% of those surveyed thought it likely or extremely likely that their local surveillance efforts will erode to focus only on those linked to payment policies or state/federal requirements.
• All thought that HAI surveillance metrics linked to payment policies and state/federal mandates would continue to grow to include more outcome and process measures.
• Respondents felt that pay-for-performance metrics were most likely to drive practice change (moreso than public disclosure of data, use of data by practitioners, or release of national summary statistics).
• Fewer than half thought it was likely that fully automated metrics from existing data elements would replace manual review of records for HAI determination.
• About half of respondents thought that the increased attention to HAI prevention from payment policies, mandates and public reporting has made patients safer (13% thought it hadn’t, and 40% thought the jury was still out).
• Almost 80% of respondents thought that infection prevention experts and clinical providers should have a much larger role in developing and modifying state/federal reporting requirements.

In other words, there is real concern in the HAI prevention community that the increased attention to HAIs, the drive to “zero”, the link to payment policies and public reporting requirements, is a double-edged sword. It has resulted in some tangible successes (CLABSI reductions being a prime example), but threatens to undermine our ability to respond nimbly to emerging local priorities by consuming all of our time and energy, and by producing data that no longer accurately reflect the true rate of adverse outcomes. To quote again from Mary and Eli:

"Undermining our surveillance system to serve ill-designed demands for accountability means that it may no longer be useful for monitoring and driving patient-safety improvements. That would truly be a shame."