More on CLABSI validation

A new study in the American Journal of Infection Control looks at validation of publicly reported CLABSI (central line associated bloodstream infection) data in New York. To my knowledge, New York has done the most work on validation, so it's worth taking a look at this paper. The authors analyzed CLABSI data from 2007 through 2010. By year, the sensitivity of reported data varied from 68% to 74%, and for the entire time period was 71%. Of concern, there appeared to be no improvement over time, and for 2010 sensitivity for medical ICUs was only 46%. Specificity ranged from 90 to 99%. So, about 30% of CLABSIs are not reported in a state that has focused on validity. It makes you wonder how bad validity may be in states that have done little work on this. It's interesting that consumer advocacy groups don't seem to be too concerned about the validity issue, and to some degree neither do payers. A consumer advocate once told me, "Just give us all the data. We don't care if it's correct." Caveat emptor!

Graphic: WhyWeSuffer.com

HICPAC lays down the letter of the law

This week's Annals of Internal Medicine has a paper entitled "Public Reporting of Health Care–Associated Surveillance Data: Recommendations From the Healthcare Infection Control Practices Advisory Committee." It's a somewhat misleading title, as the paper isn't about public reporting, it's about what happens in hospitals before HAI data are publicly reported. The primary message is this: if a case meets criteria for HAI sensu stricto, it's an HAI, dammit! Don't go asking a doctor for her clinical opinion on whether it's an infection and then erasing that case from your line list if she says, "doesn't look like an infection to me."

I have mixed feelings about this. On the one hand, in order to allow for valid inter-facility comparisons of infection rates, everyone has to play by the same rules. I think we all get that. But it's incredibly frustrating to review a case that is clearly not an HAI, yet be forced to label it as such. And it's not an uncommon occurrence. Last week, my IPs brought me a case of a patient admitted with pneumonia and because the patient's condition worsened after admission, we were forced to label the case a possible VAP. This morning, an IP from another state emailed me a case of a patient who was admitted with an infected wound, went to the OR, and subsequently developed a surgical site infection, which had to be categorized as an HAI. The IP notes, "the same patient, without the surgery, would have a community acquired wound infection and would not be counted as an HAI."

Even more commonly, hospitals with large oncology populations see many cases of bloodstream infections with enteric flora in patients who just so happen to have a central line. While I give CDC credit for now allowing us to classify cases as mucosal barrier injury related bloodstream infections, it's of little value, as these infections are still publicly reported as CLABSIs. And to add insult to injury, those poor IPs in Pennsylvania have to send patients a letter telling them they suffered an HAI that wasn't an HAI.

All of these problems with post-ascertainment veto and adjudication are occurring because the stakes are high. Most of the time, this is done in good faith, I believe. There is a big push in hospitals to hold staff accountable for adverse events, and it really stings to have the finger pointed at you for an event that was not preventable, or maybe not even an event. It undermines the credibility of IPs and hospital epidemiologists with clinicians when you call a single positive VRE blood culture in a neutropenic leukemic patient a CLABSI. To mitigate that, I find myself appearing at committee meetings to explain that while this case technically meets the criteria for CLABSI, all evidence tells us this is an infection not related to the central line, and is in fact, not preventable. Some hospitals keep two sets of books--the official publicly reported set, and the internally "correct" set.

So while I agree with HICPAC in spirit, this paper only addresses a part of the problem. It ignores the fact that we need definitions with more specificity, and those definitions are needed now. The mucosal barrier injury infection definition has been validated, so let's use it for public reporting. Task IPs to send in descriptions of cases where the definitions are not working, catalog them, categorize them, and start fixing the definitions in a timely manner. Allow for relatively rapid tweaking of definitions instead of acting as if definitions are carved in stone and represent some absolute truth. While we will never be able to have perfect case definitions in the murky world of medicine, fixing the underlying problem to the degree that it can be fixed would decrease the drive for post-ascertainment veto and adjudication. Or how about embracing adjudication but have it occur at a central level? In a world of electronic communication it wouldn't be that difficult.

And one last thing: I think that any official statement from HICPAC should be in the public domain, not behind a journal paywall. Perhaps this paper is also posted somewhere on CDC's website, but I was not able to locate it, if it is indeed there.

Why validation is important

A new study in the American Journal of Infection Control describes the validation of publicly reported central line associated bloodstream infection (CLABSI) data in Colorado. The study encompassed a review of data from ICUs at 35 acute care hospitals and 8 LTACHs for the first quarter of 2010. Charts of 519 patients with positive blood cultures were reviewed. Results of this study, as well as 3 other statewide CLABSI validation projects are summarized below.

	Sensitivity	Specificity
Connecticut	48%	99%
New York	74%	95%
Oregon	72%	99%
Colorado	83%	99%

These 4 studies show that 17-53% of CLABSI cases were misclassified as non-CLABSIs, primarily due to labeling CLABSIs as secondary bacteremias. Interestingly, the Colorado study attempted to delineate the underlying causes of misclassification. Two reasons were found to be significant: lack of an electronic medical record or data mining software, and review of potential cases by an infectious diseases physician (i.e., post-surveillance certification).

Given all the attention paid to publicly reported HAI data and the important implications with regards to reimbursement, it's imperative that hospitals produce valid data. However, validation projects remain infrequent due to their cost. We have a long way to go...