Ebola Screening: Hey CDC! Is it an AND or an OR?

Like many of you, we've all been assisting with Ebola planning for the past several months. Just today, Mike and I were in another planning meeting and the subject of our screening algorithm came up. Mike had noticed that the September 4th CDC Definition of Person Under Investigation (PUI) that we've been using: "fever of greater than 38.6 degrees Celsius or 101.5 degrees Fahrenheit, AND additional symptoms such as severe headache, muscle pain, vomiting, diarrhea, abdominal pain, or unexplained hemorrhage" is now different in the PDF algorithm that the CDC posted on October 2nd. It now says "FEVER OR EVD symptoms." I've included the relevant portion of the PDF algorithm above.

This may seem like a minor change, but for many facilities it will be a big deal. To go from a relatively sensitive "fever AND" to a very sensitive "fever OR" is not a small change. This may require EDs and clinics to screen additional patients and result in a large increase in laboratory tests being sent for Ebola diagnosis. I think that CDC meant to suggest that fever is enough to prompt a travel history, but now anyone with fever, headache, weakness, muscle pain, vomiting, diarrhea, abdominal pain or hemorrhage will also need to be asked a travel history. In practice this will mean that every patient that walks into an ED or clinic should be asked a travel history first and then asked about symptoms. I'm not convinced this should happen and feel much more comfortable with having symptoms guide travel history. But since almost any general complaint is now included, we may be left with no other choice but to ask travel history first. And of course, we don't know what CDC wants us to do since both documents are live and offer conflicting advice.

So, which is it CDC?  Is it an AND or an OR?

Guest Post: A "Hybrid" Surveillance approach for SSI

Dr. Connie Savor Price has been co-project director and PI on a 2009 AHRQ ACTION contract charged with improving the measurement of surgical site infection risk stratification and outcome detection. The final report has just been published online and she was kind enough to stop by our humble blog to briefly describe the project.

With pressure on infection control programs to expand surveillance for public reporting and pay-for -performance, increasing surveillance efficiency without sacrificing data validity creates a conundrum. Purely automated systems for surgical site infection (SSI) surveillance have been developed and utilized, but are not validated or generalizable for these purposes.

In this report, we describe development of "hybrid" surveillance approach using highly sensitive electronic algorithms for detection of SSI targeted for subsequent human-adjudication. Electronic algorithms to detect deep and organ-space SSI after coronary artery bypass grafting, total hip and knee arthroplasties, and herniorrhaphies were created using a sample of nationwide Veterans Affairs Surgical Quality Improvement Program data (VASQIP). The algorithms were tested against VASQIP data, and then assessed for generalization using data from hospitals from three different, external (non-VA) healthcare systems.

Although all algorithms performed reasonably well at identifying deep and organ-space SSI among the VASQIP test data set, performance was variable when tested against data from the outside systems. As one would expect, the observed variation was primarily due to differences in the data collected and stored electronically in each system. While not surprising, perhaps some are disappointed that this research did not produce a universally applicable surveillance tool for "out-of-the-box" utility for all infection control programs. But just wait. As meaningful use incentives help us move toward interoperable electronic health records throughout the United States, this surveillance strategy will hold promise as a reliable tool for detecting potential surgical site infection.

We can't predict HAI with ICD-9 codes and it's only going to get worse

I'm getting ready for a chat with a reporter concerned with issues surrounding HAI surveillance. During my preparation, I thought again through the issues of code-based algorithms (e.g. ICD-9) and I've come to the conclusion that they are useless for assessing the burden of HAIs and HAI trends and it's only going to get worse.

One area we (and many others) have looked at is the utility of ICD-9 code-based algorithms (ie administrative codes) for detecting HAIs efficiently. A key metric frequently reported by researchers is the sensitivity of a specific code or code algorithm, which is great if the purpose of the algorithm is to improve the efficiency of detection by manual methods. Thus, if the sensitivity is high-enough, you could use the code-based algorithm to reduce the number of charts that require an IP's review. If you are using codes in this way, great!  I have no problems with that.

However, many are now using code-based algorithms to track trends in specific HAIs and measure the burden of disease. My general feeling on these is that they should be completely avoided for several reasons:

1) No matter how sensitive the algorithm is, all we care about here (since we are not validating with manual review) is the positive predictive value (i.e. the proportion of all code-positive patients that actually have the HAI of interest)

2) The PPV is very low for almost all HAI algorithms

3) If we are doing our job and lowering the incidence of HAI per admission in our hospitals the PPV by definition will only get worse (given a fixed sensitivity and specificity)

To show you why I have these concerns I have constructed two 2x2 tables evaluating an excellent hypothetical code-based algorithm for UTI with a sensitivity and specificity set at 95%.  In this first 2x2, I have evaluated the performance of the algorithm when the HAI has a 5% incidence per admission (i.e., 5% of the admissions had a UTI). You can see that such a great algorithm with a high-prevalance of disease, has a poor PPV of 50% - like flipping a coin.

Now, assume we have done an amazing job and cut our HAI rate down to 1%.  Given the same hypothetical algorithm, our PPV is now a horrible 16%. Thus, as we get better at preventing HAIs, we get worse at detecting them using code-based algorithms. Are you comfortable saying UTIs are increasing or decreasing or are associated with a certain level of excess costs, when only 16% of the UTIs in your estimation are actual (true positive) UTIs?  Me neither.

Are you pro procalcitonin?

Clinicians over prescribe antibiotics. I spent several years exploring the risk-averse nature of physicians (specifically ID physicians) when it comes to avoiding treatment failure in diabetic foot ulcer, community-acquired pneumonia (CAP), and CVC bacteremia. Using binary choice contingent-valuation analysis our group determined that ID docs were very risk averse. For example, three of 34 ID physicians found a failure rate of 1% in CAP to be unacceptably high. So how do we acknowledge the risk-averse nature of clinicians, while at the same time safely improving antimicrobial stewardship through reducing over-prescription or shortening duration of therapy?

One possible and much heralded solution was going to be improved diagnostic microbiological tests, such as PNA FISH, which can rapidly detect clinically important (ie you need to alter therapy) pathogens, such as MRSA. These require waiting for positive blood cultures and haven't yet fully caught on, for whatever reason.  Importantly, these micro tests can't tell you if the patient is infected vs. colonized or what clinical syndrome they might have, such as pneumonia. Even a chest xray can't tell you if your patient has pneumonia.  That's why I was cautiously hopeful as I read a meta-analysis by Philipp Schuetz et al. of procalcitonin in directing antibiotic initiation and duration in acute respiratory infection, published online May 9th in CID.

Procalcitonin has gotten a lot of attention recently since it's been found that levels are high in severe bacterial infections but lower in viral or non-specific illnesses. Thus, algorithms that include procalcitonin cut-offs might help target initiation and withdrawal of antibiotic therapy without impacting clinical outcomes. To verify this claim, the authors reviewed 14 clinical trials with a total of 4221 patients to determine mortality, treatment failure and total antibiotic exposure in patients treated under procalcitonin-guided algorithms versus standard therapy. Mortality was similar in both groups (OR 0.94, 95% CI [0.71-1.23]) and treatment failure was slightly lower with procalcitonin (OR 0.82 [0.71-0.97]. Both of these were despite relatively poor-adherence in some of the studies. Importantly, total antibiotic exposure was decreased when using procalcitonin containing algorithms with a statistically significant reduction of 3.47 days. However, sample size was low when limited to ICU patients, so further studies are needed there.

These results do suggest that procalcitonin-containing antibiotic treatment algorithms may have an important role in respiratory tract infection therapy, particularly from a public-health, antibiotic resistance standpoint.  It remains to be seen if algorithms with limited clinical benefit for the individual patient but large potential benefits in terms of stewardship and public health will ever be widely adopted. Physicians are still risk averse, after all. Fingers crossed.

Source: Schuetz P. et al Clin Infect Dis 2012 (online May 9, 2012)

Image source: July 2009, Clinical Laboratory News