Shocking: Another review of the evidence finds flu vaccine has no miracle powers

The Science section of the New York Times this week had an article on a new report regarding influenza vaccination. This report (free full text here) from the University of Minnesota is the kingdaddy of all analyses on influenza vaccination--a 3-year project funded by the Alfred P. Sloan Foundation that reviewed 12,000 papers back to 1936 and involved interviewing 88 influenza experts. The report is 123 pages without appendices and includes over 500 references. 

Here's my summary of the chapter on vaccine performance of the two major vaccines (trivalent inactivated influenza vaccine [TIV] and live attenuated influenza vaccine [LAIV]):

Inconsistent evidence of protection
High level of protection (83%)
Healthy adults
Moderate protection (59%)
Lack of evidence of protection
Paucity of evidence for protection
Inconsistent evidence of protection

What about use of the vaccine in healthcare workers? This can be found on pages 57-58 of the report and I have pasted below the important discussion of the HICPAC recommendation regarding offering influenza vaccine to healthcare workers):

The 2006 statement on influenza vaccination of healthcare personnel (HCP) from the Healthcare Infection Control Practices Advisory Committee (HICPAC) and ACIP illustrates potential concerns with using a grading scale.[63] This recommendation used the HICPAC grading scale, which is similar to the GRADE criteria in that it provides a structure for ranking the evidence. All recommendations were approved by the HICPAC and the ACIP. This document has been used widely as evidence to support HCP vaccination policies, including mandating vaccination. It offers six recommendations, and one was deemed to have the highest possible evidence, category IA. Category IA recommendations are “strongly supported by well-designed experimental, clinical, or epidemiological studies.”[63] The recommendation in the HICPAC document that received a category IA rating states:  “Offer influenza vaccine annually to all eligible HCP to protect staff, patients, and family members and to decrease HCP absenteeism. Use of either available vaccine (inactivated and live, attenuated influenza vaccine [LAIV]) is recommended for eligible persons. During periods when inactivated vaccine is in short supply, use of LAIV is especially encouraged when feasible for eligible HCP.”[63]
This recommendation is supported in part by this key summary statement in the HICPAC document: “Vaccination of HCP reduces transmission of influenza in healthcare settings, staff illness and absenteeism, and influenza-related morbidity and mortality among persons at increased risk for severe influenza illness.[64-67]” In the first study cited, the authors did
not find a statistically significant reduction in patient mortality associated with HCP vaccination, after adjusting for covariates.[64] In the second study, the authors concluded that “we do not have any direct evidence that the reductions in rates of patient mortality and influenza-like illness that were associated with HCW vaccination were due to prevention of influenza.”[65] In the third study, vaccination did not reduce the episodes of self-reported respiratory infection or the number of days ill with a respiratory infection, but it did reduce the time employees were 58 unable to work because of a respiratory infection.[66] In the fourth study, the authors reported reductions in absenteeism and illness among HCP that were not statistically significant.[67] The authors did, however, report serologically confirmed vaccine effectiveness of 88% for H3N2 and 89% for influenza B across three influenza seasons.[67] Since only two of the four studies cited provide some support for the HICPAC statement and the others no support, it is unclear how the quality of evidence in these studies received a category IA evidence grade. Another review conducted in the same time frame by the Cochrane Collaboration noted that the two RCTs cited in this recommendation were at “moderate risk of bias.”[68] They concluded that “both elderly people in institutions and the healthcare workers who care for them could be vaccinated for their own protection, but an incremental benefit of vaccinating healthcare workers for elderly people has yet to be proven in well-controlled clinical trials.”[68]
So this report questions the evidence base for even recommending influenza vaccination to healthcare workers. Yet, SHEA's position is so over-reaching that it calls for mandating vaccination and firing noncompliant healthcare workers. This is now the fourth independent analysis that does not support the SHEA position statement (read about the others here, here and here).

I continue to be fascinated by the post-modern disdain for evidence. A marvelous example from this week is the shock and utter disbelief suffered by Mitt Romney and his staff on learning that Barack Obama won the presidential election, despite nearly every poll indicating that Romney would lose. I guess I naively thought that somehow epidemiologists were immune to such bias but SHEA's flu vaccine position suggests otherwise.

One of the recommendations in the Minnesota report is that "scientifically sound estimates of influenza vaccines’ efficacy and effectiveness must become the cornerstone of policy recommendations." Amen. And it's time for SHEA to retract its policy!


  1. I am still parsing through this report, but want to say for the sake of open dialogue: Let's please be careful in making the distinction between whether the current body of evidence proves a negative (i.e. proves that the current flu vaccine modalities are ineffective in protecting us and our patients) versus whether the current body of evidence is too weak to be definitive either way. "Has yet to be proven in well controlled clinical trials" (report p. 58) is categorically NOT the same as "has been definitively refuted."

    Challenge studies would be difficult to do at the scale and thoroughness that would really be needed... And I doubt we could get many healthcare workers to volunteer for such research. But I also would like to doubt that we are at a point where we would consider it appropriate to risk a de-facto experiment of dropping herd immunity to all time lows by removing any regulation or recommendation for taking the shot.

    This is an excellent opportunity for educating ourselves, each other, and the public. As we digest this report it will behoove us to think about what the source studies are actually asking and measuring, and what they are not. Is serological evidence of an immune response raised to a vaccine a sufficient scientific surrogate for protection upon exposure to virus? (No!) How would we measure actual protection against exposure in the real world setting? (Very hard/expensive to do!) Are the studies following these endpoints for long enough? Are the sample sizes large enough to account for variations in risk for exposure to flu virus and underlying variation in immunity? (Not everyone falls in the center of the bell curve!) Are the studies checking whether the people who get sick after being vaccinated are ill with a flu strain not included in the vaccine, or another URV altogether? (Maybe not!) And the list goes on...

    Sarah Timberlake, BS, BSN, RN

  2. I totally agree that absence of evidence is not evidence of absence of effect. My point is that expectations for compliance must be tightly correlated to the quality of evidence. So an expectation for 100% compliance (and firing for noncompliance) should require a very high level of evidence. In my estimation, for the case of flu vaccine for healthcare workers, we have enough evidence to recommend but not mandate.

  3. We have had a "mandatory response" program for a few years now, which requires either getting vaccinated or actively declining. There is no sanction for declining, and it is simple to do (it is done online, and the HCW is asked to indicate if the reason is personal, medical, or religious). I think the only sanction for failing to respond is the flurry of e-mails you get until you do....

    We achieve ~ 90% vaccination rate with this approach. I would be interested in the incremental cost-benefit of trying to push it higher by making it a condition of employment, terminating a small number of staff, fighting (again) with our union, etc., etc. Overall our current approach seems to be about right, given the available data.

  4. Is there an easy to understand reason why a "miracle" flu vaccine is so elusive, while "miracle" vaccines for other viruses seem commonplace? Is it that flu is so much more abundant when it's in season that people get a slew of exposures, but with others exposures are slight so the vaccine doesn't have to be as effective? Or is it just that the vaccine doesn't work so wel because they are guessing at the organism strain every year? Something else?

  5. I like the reference to postmodernism, suggesting in this instance that important infection prevention policy decisions are not fully driven by data. Rather, these are driven by belief or conviction. As epidemiologists and scientists, it is important for us to be skeptical and to recognize bias in all realms of our practice.

    We should only be mandating infection prevention policy changes based on a high level of evidence, particularly when there is potential punitive action.


Post a Comment

Thanks for submitting your comment to the Controversies blog. To reduce spam, all comments will be reviewed by the blog moderator prior to publishing. However, all legitimate comments will be published, whether they agree with or oppose the content of the post.

Most Read Posts (Last 30 Days)