The Good, the Bad, and the Ugly

The good Dr. Diekema stopped by Dartmouth-Hitchcock last week to deliver medical grand rounds. He provided a wonderful update on infection prevention. You can click on the screen shot to the right or the link below to view a video of the presentation, which includes his full slide deck. Enjoy!

Source: Dartmouth-Hitchcock Grand Rounds 1/25/2013

HOP, SCIP: JUNK!

There is an interesting paper in the most recent Clinical Infectious Diseases on the Surgical Care Improvement Project (SCIP) and the Hospital Outpatient Measures Project (HOP). These are national QI projects intended to improve surgical care. The article lays out numerous problems with these projects, yet despite that, results from these projects will now begin to impact hospital reimbursement.

The authors note: "Measures are rolled out before their full impact is assessed, using live hospitals as the testing ground and relying on individuals trying to comply with these measures to troubleshoot. When issues do arise that require the measures to be changed, response times are invariably at least 6 months; meanwhile patients may be at risk, and measures are consistently failed."

It would be very interesting to know what these projects have cost hospitals. At my hospital, we have 1.5 nurse FTE just to abstract the data. Beyond that are thousands of hours of physician and nurse time spent trying to improve compliance with the metrics. And yet, there remains no compelling published data that outcomes have been improved.

Dale Bratzler, the brainchild of the projects, writes a response in the same issue of CID. At best, his response is tepid, and sheds little light on why these projects should be continued. His commentary ends like this: "The specific issues with SCIP performance metrics highlighted by Weston and colleagues are clearly a source of frustration for providers. However, the authors do not provide any evidence that harm has occurred because of implementation of SCIP." Ok, so we wasted millions of dollars, frustrated clinicians, and are about to punish hospitals financially, but we don't think any patients were harmed. Now that's exactly why much of QI is viewed as a joke!

Things is gettin' worser - Pediatric Undervaccination

The typical trajectory of public health over the past several centuries has pointed upwards towards improvements including longer life-expectency. However, as we've entered the post-scientific era, where our country receives its medical advice from celebrities, it's not at all surprising that things can move in the wrong direction. Of course, vaccines are a victim of their own successes since absence of a terrible diseases like measles convinces people they don't need to vaccinate their children and also more susceptible to false claims of vaccine side-effects, but I digress.

Authors Jason Glanz and colleagues recently published a matched-cohort study of healthcare utilization in vaccinated vs. under-vaccinated children less than 3 years old. Their primary findings were that undervaccinated children had lower rates of outpatient visits but higher rates of inpatient admissions.  There are also some interesting findings about how parental choice in vaccination is associated with healthcare utilization. What was more interesting to me is how things have changed gotten worse in the birth cohorts over the 5 years studied (2004 to 2008).  In the figure below, time to first vaccination has increased greatly over the 5 years (solid shapes), as have the rates of 'no vaccination'. So things are moving upward, but in this case, that's the wrong direction.

Reference: Glanz JM, Newcomer SR, Narwaney KJ, et al. A Population-Based Cohort Study of Undervaccination in 8 Managed Care Organizations Across the United States. JAMA Pediatr. 2013;():1-8. doi:10.1001/jamapediatrics.2013.502.

Influenza Vaccine Effectiveness Study Author Responds

Late last week, Dr. Jackson, one of the co-authors of the recent MMWR influenza vaccine effectiveness report, sent me an email response to my posts discussing how and why they measure influenza vaccine effectiveness the way they do. I thought that in the interest of fairness, I should post his full email, rather than pasting his response into a largely hidden comment section. I will likely post a follow-up to this at some point. Additionally, I want to thank Dr. Jackson publicly for responding in this fashion, both professionally and academically. The intent of my posts was academic, and I am pleased that he responded the way he did.

Dear Dr. Perencevich,

I am one of the co-authors on the MMWR article on influenza vaccine effectiveness, and I read your blog posts about that article. I believe I can clear up some of your questions. I tried posting this as a comment on the blog, but the website wouldn't let me, so my apologies for e-mailing you instead. Feel free to post this as comment on the website if you are able.

 (1) Regarding your first point, this study used what is known as a “test-negative” design. In the test-negative design, we enroll patients with a medically attended acute respiratory illness (MAARI). We then test these enrollees for influenza, and assess who was previously vaccinated and who was not. The test-negative design is based on an assumption that the rate of MAARI caused by pathogens other than influenza is the same in both vaccinated and unvaccinated persons. If this assumption is true, than our test-negative subjects are representative of the population from which the influenza-positive cases came, and our study does give an estimate of how well the vaccine reduces the risk of getting sick enough to visit the doctor.

 Although the paper refers to cases and controls, this is not a true case-control study, since a true case-control study requires that we know who is a case and who is not before we sample them. The design is closer to the “indirect cohort” method proposed by Claire Broome for studying pneumococcal vaccine effectiveness [NEJM 1980; 303:549-52]. The real advantage of the test-negative design is that it controls for differences in healthcare-seeking behavior. If we did the full cohort study you proposed, there would be variation among the cohort members in how often they seek healthcare, and these variations would be related both to their likelihood of being vaccinated and to their likelihood of going to the doctor if they got influenza. By only sampling people who come to the doctor, we control for those differences.

(2) Regarding your second point and third points, the relative risk (RR) is not an appropriate measure of association for this study design. The RR is (obviously) a measure of risk, which would be based on the cumulative incidence of disease in some defined cohort. In this study, we do not sample the full cohort; we simply use the influenza negative subjects to estimate the frequency of vaccination in the cohort. Using the RR in this setting would give a biased estimate of vaccine effectiveness, which is seen in the sample calculations you provided.

Sampling in a test-negative design is conceptually similar to incidence density sampling in a case-control study. When using incidence density sampling, the exposure odds ratio is a direct estimate of the incidence rate ratio and not an approximation to the RR. 

(3) Finally, regarding your final question about including the flu B cases in the estimate of VE against flu A: As mentioned above, the test-negative design assumes that the rate of non-flu MAARI is the same in vaccinated and unvaccinated persons. If we included flu B in the non-case group, we would be violating this assumption, because the vaccine does protect against B, and the rate of non-flu A MAARI would no longer be the same in vaccinated and unvaccinated persons.

 I hope this clears up your questions!
~Mike Jackson, Group Health Research Institute

Semmelweis was right!

We do a lot of blogging about hand hygiene, and Eli frequently points out the dearth of well-designed studies examining improvement approaches and/or the relationship between HH improvement and reduced infection rates.

In that context, I’d like to recommend this quasi-experimental study from Kathy Kirkland and her colleagues at Dartmouth, published last month in the BMJ Quality & Safety journal. Eli can inform us as to whether it would have “made the cut” for the 2011 Cochrane Review, but I really like this report. The interventions and setting are clearly described, more than one measure of HH is used (direct observation + product use), infection rates are reported over the entire time period, and the discussion section is thoughtful.

There are limitations to the work, but I encourage those who haven’t yet seen it to read it. Mike can add his experience here, as I think his group does more monthly observations, and has had a similar HH journey. I’ve also long been skeptical that rates of 90% can be achieved or sustained in settings where observations are truly clandestine (as I think in most hospitals the HH observers quickly become quite familiar to unit personnel). I’d love to be disabused of this skepticism, if it is indeed misplaced.


Figure from Kirkland KB, et al. BMJ Qual Saf 2012;21:1019-26.

Hand Sanitizer - Doing The Math(s)

link to this comic: http://xkcd.com/1161/

--> and if you love cartoons, you will love these influenza-specific New Yorker cartoons

Proof.

Finally, the purists out there who require demonstration of efficacy by a randomized clinical trial before attempting a novel therapy can now breathe a great sigh of relief. The New England Journal of Medicine has just published an RCT that demonstrates the clinical utility of fecal transplantation for C. difficile infection. In fact, fecal transplants worked so well that the trial was terminated early after an interim analysis.

Patients in the study all had C. difficile infection with at least one relapse. They were randomized to one of three study arms: (1) a 4-day course of oral vancomycin followed by bowel lavage then fecal transplant via nasoduodenal tube; (2) a 14-day course of oral vancomycin; or (3) oral vancomycin plus bowel lavage. In the transplant group, 13 of 16 patients were cured after 1 fecal infusion (2 of the remaining 3 were cured after a second infusion). In contrast only 4 of 13 in the vanco group, and 3 of 13 in the vanco plus lavage group were cured. Bottom line: fecal transplantation had an overall cure rate of 96%.

There remain two barriers for patients to access this highly effective therapy: (1) very few physicians perform the procedure, in part, I think, because there is no reimbursement despite the several person-hours required to prepare the fecal solution and administer it; and (2) insurance companies will not reimburse for donor testing, which costs approximately $1500.

So we've proven what we already knew. Now it's time to look at more interesting questions: does fecal transplantation work for irritable bowel syndrome and inflammatory bowel disease?

Graphic:  Andrea Levy, Cleveland Plain Dealer.