That’s what I learned from today’s
New York Times editorial. In their review of the two NEJM studies we covered last week (
here and
here), they describe the VA study as “broader” (true, if by broader they mean larger) and “possibly more rigorous” (untrue). They also point out that “if other hospitals could replicate the effort, thousands of patients might be saved from needless infections”. The editorial board at the NY Times should know that hospitals across the country are
already saving thousands as MRSA HAI rates
drop nationwide! And many of these hospitals are saving lives without also
stimulating the
economy by doing universal MRSA screening. A bundle with multiple interventions, one of which is unproven and hugely resource intensive, doesn’t seem in keeping with the spirit of health care reform. And sadly, determining the proper role of the most expensive element in the VA's MRSA bundle requires studies that use something we epidermatologists like to call a “control group”.
That’s what I learned from today’s New York Times editorial. In their review of the two NEJM studies we covered last week (here and here), they describe the VA study as “broader” (true, if by broader they mean larger) and “possibly more rigorous” (untrue). They also point out that “if other hospitals could replicate the effort, thousands of patients might be saved from needless infections”. The editorial board at the NY Times should know that hospitals across the country are already saving thousands as MRSA HAI rates drop nationwide! And many of these hospitals are saving lives without also stimulating the economy by doing universal MRSA screening. A bundle with multiple interventions, one of which is unproven and hugely resource intensive, doesn’t seem in keeping with the spirit of health care reform. And sadly, determining the proper role of the most expensive element in the VA's MRSA bundle requires studies that use something we epidermatologists like to call a “control group”.
The VA study collected time-series data on monthly infection rates per 1000 patient-days. The study design was a before-after quasi-experimental design without a control group. To improve the internal validity of the study, it would have been very helpful to have a control group so that trends seen in the VA could be compared to see if they were greater or lesser than rates in other settings.
ReplyDeleteHowever, even without a control group, it would have been helpful if they analyzed the data using segmented regression techniques. Studies with time-series data, such as this VA study, can have very high internal (and statistical)validity even without a control group, if analyzed using the proper statistical techniques. Segmented Poisson regression could have allowed for the assessment of changes in intercept/slope before and after the intervention, which is far more powerful than the analyses completed in the study.
These regression methods would also allow for control of important unequally-distributed confounders which appear even in studies with control groups (see the Huskin's study and the strong stats methods he used).
A good place to read more about the proper analysis of time-series (quasi-experimental) data is Michele Shardell's CID review:
http://cid.oxfordjournals.org/content/45/7/901.full?sid=94c68990-d11e-42a4-8959-a9e0ed3b9d0c
I was working at VA Central Office at the time and I can tell you that the choice was either do the intervention Pittsburgh-style or don't do it at all. Sec Nicholson decided he wanted the program, as he saw it working, to be replicated. Now they/we have the luxury of trying to figure out if we can cut out one or two of the interventions, because we know the bundle with all the interventions worked. It also worked for C diff and VRE, so we know it can't just be the testing. The testing may have contributed simply by convincing people that the program was "serious" and not just another intervention that focused on trying harder. No knowing which part(s) of the bundle are necessary is a good problem to have. In 2005 we had a bad problem: MRSA and C diff were both going up (I don't remember the VRE data), and there seemed to be consensus that there wasn't much to be done about it. MRSA at a high rate seemed like a "new normal." Since then the toothpaste has been put back in the tube.
ReplyDeleteHi neldridge202, Thanks for pointing out what is lost in much of the discussion--the VA directive wasn't a "study", it was a system-wide QI project, and based upon the NEJM paper a tremendously successful one. It wasn't similar in any way to the STAR*ICU study, it was a different animal entirely, and I don't think it is useful to attempt to compare them. I also agree that the most important role that screening played in the study was to "shock the system" into taking the whole intervention seriously. Not the whole system, because many VAs had a handle on MRSA already--the original MRSA infection metric I saw (hospital-specific) showed a number of high outliers....just getting those hospitals to pay attention could have accounted for a large proportion of the observed impact of the intervention (it would be nice to see the data broken out by individual facility, as we've seen with data reported out of the Keystone project). Maybe that's in an appendix somewhere, I'll have to check.
ReplyDeleteThanks for continued and active interest in this blog.
Dan
...and yes, I realize I keep calling it a study anyway, by which I mean a retrospective analysis of the effectiveness of a system-wide QI project!
ReplyDeleteFrom my reading of the paper, the only components of the bundle that could have reduced VRE and C. diff were hand hygiene and culture change. So it begs the question of whether the culturing & isolation of patients had any effect. And there is little mention of the opportunity cost--what could the VA have done with the money it spent on cultures? All in all, I see this paper as a big step backward for infection prevention in the US because it further drives the false belief that active surveillance for MRSA is required to control it.
ReplyDeleteI think any use of contact precautions could reduce VRE and C. diff even if it wasn't targeted at them. Factors to consider are the underlying co-colonization rate and also the fact the isolating a colonization-negative patient could protect them. For example, isolating an MRS+ patient who is co-colonized for VRE could limit the transmission of VRE to other patients. At the same time, isolating that same MRSA+ patient could prevent it from acquiring C. diff. Thus, the findings of the VA investigation could be explained by increased use of contract precautions.
ReplyDeleteThanks for the positive feedback Dan. I don't know if we can ever figure some of these things out in a sort of permanent way. The automobile deaths per vehicle mile driven went down 75% over the 40 years from the mid 60s to the mid 00s, and no one knows why for sure. The big 5 reasons in my opinion are seat belt usage, less frequent drunk driving, better ERs for people in accidents, road improvements (like rumble strips), and miscellaneous car improvements (better tires, brakes, crumple zones, etc.). But this is, again, a good problem to have, i.e., figuring out exactly why fewer people are accidentally being killed. If you think MRSA testing is a waste of money, how about all those sprinklers in hospitals to prevent fires from spreading? They must have cost billions nationwide, and are almost never used.
ReplyDeleteI looked up the press release for the MRSA Prevention kickoff in 2007. It's on-line at: http://www.va.gov/opa/pressrel/pressrelease.cfm?id=1333 . Give these guys credit for one thing: they actually did what they said they were going to.
Thanks for the positive feedback Dan. I don't know if we can ever figure some of these things out in a sort of permanent way. The automobile deaths per vehicle mile driven went down 75% over the 40 years from the mid 60s to the mid 00s, and no one knows why for sure. The big 5 reasons in my opinion are seat belt usage, less frequent drunk driving, better ERs for people in accidents, road improvements (like rumble strips), and miscellaneous car improvements (better tires, brakes, crumple zones, etc.). But this is, again, a good problem to have, i.e., figuring out exactly why fewer people are accidentally being killed. If you think MRSA testing is a waste of money, how about all those sprinklers in hospitals to prevent fires from spreading? They must have cost billions nationwide, and are almost never used.
ReplyDeleteI looked up the press release for the MRSA Prevention kickoff in 2007. It's on-line at: http://www.va.gov/opa/pressrel/pressrelease.cfm?id=1333 . Give these guys credit for one thing: they actually did what they said they were going to.