Even with the Hawthorne Effect, Hand Hygiene Compliance Still Hideous

One way to deal with low hand hygiene compliance

A recurring theme has emerged in hand hygiene science - when you really look, compliance is very very low. A study 5 years ago reported that compliance was below 10% when hidden video monitoring was utilized. Our group has quantified the impact of the Hawthorne Effect, improved behavior when subjects know they're being observed, on hand hygiene compliance. In a multicenter study, we found that both measured exit and entry compliance increased the longer direct observers remained on the unit.

There is a report of a new APIC abstract in ABC-News that further quantifies that impact of the Hawthorne Effect and highlights the lack of investment in hand hygiene programs. Investigators from Santa Clara, California compared compliance measured by well-recognized Infection Prevention nurses, to observations collected by unknown high-school and college-aged volunteers who were trained to use the same surveillance methods. Here are their findings:

So, what do I make of these findings? First, even 57% is too low. Second, hospitals and healthcare systems continue to throw hand hygiene programs under the rug. We are much happier to report compliance rates of 100% collected by nurse managers on the floor (or compliance of 57% by recognized IPs) and ignore the problem than spend time and money detecting compliance rates of 22%, which would then require additional investments in proven hand hygiene interventions.

Any administrator, who thinks compliance in their hospital is higher than 70% or 90%, won't invest in hand hygiene programs. Since hospitals are happier to report compliance of 90% to the Joint Commission, we also won't invest in technological and socio-adaptive interventions that will finally improve the safety of our hospitals. We must work to create a safety culture where it is better to report hand hygiene compliance of 20% than falsely high compliance rates of 90%.

image source: montonara.com

Give Directly-Observed Hand Hygiene Compliance Monitoring a Chance

As I've said before, "It's amazing how little evidence is required before infection prevention interventions are adopted." This phenomenon is particularly evident in the setting of automated hand hygiene monitoring systems. Few trials have been completed that have assessed the efficacy, effectiveness or cost-effectiveness of these systems, at least as of our recent systematic review.  However, it appears that many facilities are purchasing these systems anyway. Why is this?

It appears that there are two potential limitations of directly-observed hand hygiene monitoring driving the purchase of automated systems: (1) the fear that the Hawthorne effect renders all direct observations invalid and (2) the idea that all hand hygiene opportunities must be counted, i.e. it's unacceptable to sample opportunities. There is a third potential reason for adopting these systems - anecdotal reports that automated systems improve compliance. However, there is little published evidence that compliance increases are sustained and some evidence that automated systems actually decrease compliance.

With the potential limitations of directly observed compliance monitoring in mind, our research group just completed a study published in ICHE that sought to determine if/when the Hawthorne effect appears in hand hygiene observation data. The idea being, if it takes some period of time for HCW to know they are being observed, shorter periods of hand hygiene monitoring could be less susceptible to bias from the Hawthorne effect. After 3,432 hours of observations and 11,444 witnessed opportunities in a multi-center study, we calculated that room-entry compliance increases after 38 minutes and room exit compliance increases after 14 minutes and then again after 50 minutes (Figure below). Thus, it appears that limiting direct observation periods to less than 15 minutes limits the impact of the Hawthorne effect.

The other aim of our study was to determine the number of hand hygiene opportunities that must be observed to have an adequate sample size for comparison. For example, if hand hygiene compliance in your ICU is 80% this month and you would like to get it to 90% next month, you would need to observe 108 opportunities in each month to have enough power to detect a difference. We provide a table (below) so that anyone can quickly determine the number of observations in a month/quarter/year needed to compare time-periods or units/wards or facilities. We hope you find this information useful and that you'll still give directly observed hand hygiene compliance monitoring a chance.

Shouldn't evidence guide our selection of hand-hygiene surveillance systems?

It's amazing how little evidence is required before infection prevention interventions are adopted. A current example of this is the installation of automated hand-hygiene surveillance systems that track healthcare worker room entry and hand hygiene compliance. Hospitals are committing significant resources, both financial and person-time, to implement these systems with minimal evidence that they sustainably improve compliance or are accurate and cost-effective. To channel Jerry Maguire, before hospitals "show them the money", shouldn't they ask companies to "show me the meta-analysis"?

With that in mind, I really enjoyed reading the study in February's ICHE by Luke Chen and colleagues at Duke describing the implementation of an electronically-assisted, directly-observed hand hygiene surveillance system. The investigators recognized that directly observed compliance remains the gold standard, but also realized that economic and time costs, along with potential biases such as the Hawthorne effect, limit its utility. Thus, they set out to improve on the gold standard by modifying it to address potential biases and reduce costs.

Beginning in 2009, non-secret (they wore ID badges) auditors began monitoring compliance in 40 wards/clinics. They observed two moments of compliance, before/after room entry. All data was entered into wireless-PDAs that were linked to a centralized server allowing instantly updated 30-day tracking reports.

Overall, the compliance rate reported was 88% after 100,000 observations. That's pretty good. What's more, they reported compliance by the order of observation. For example, they calculated the average compliance for the first opportunity observed, the second opportunity and so on. The reason they did this, is they hypothesized that the Hawthorne effect wouldn't kick-in until the observer was seen by the healthcare worker and that this would become more likely the longer the direct observer remained on the ward. What did they find? Look for yourself:

What do you see? They found compliance for the first five observations was less than 86% while the average over the 35th to 43rd observations was 95%. We see an actual Hawthorne effect. Excellent. After seeing this, and for other reasons as well, they created standard operating procedures that observers used. The most important change to their procedures was they limited observations to 10 minutes or 10 total opportunities before moving to the next ward. Following these changes, compliance was reported to be 86%.

Of course there are limitations to this study in that it was a single-center study and lacked economic data to help guide its broader adoption. The authors acknowledged these and are addressing them already in a future study. My sense is that this method will be more effective and cost-effective than fully-automated systems and also keep infection preventionists visible on the wards to identify and address other important issues, which they can't do from behind their computer screens. But unlike many pushing for a new hand hygiene monitoring system, I'm going to start collecting the data and let the evidence guide my decision making.

Image: Luke Chen, MBBS MPH