Bloggers correct the New England Journal on ICU decontamination article

Blogger Peer Review Corrects NEJM Article's Error

In June 2013, PulmCCM unquestioningly reported the results of a major randomized trial in the New England Journal of Medicine by Susan S. Huang et al, showing that decontaminating patients upon arrival to the ICU with chlorhexidine baths and nasal mupirocin resulted in a dramatic drop in nosocomial infections (from MRSA, and all bloodstream infections). How dramatic? Their reported number needed to treat was only 54 to prevent a bloodstream infection, and 181 to prevent an MRSA infection. These numbers suggested highly favorable cost-benefit estimates, and a potential new lifesaving treatment. With study participation by the CDC and Agency for Healthcare Research and Quality (AHRQ), the convincing results supported immediate practice-changing deployment. The article made it through the world-renowned peer review process at NEJM. The academic community received the published results without a contrary peep. The AHRQ posted advice on its website to facilitate universal implementation. But then the bloggers got hold of it. In August, Oliver Flower started a discussion on at the excellent Intensive Care Network blog (prompted by researcher Sandra Ware, and Gordon Hoig, blogging at evidencebased.net) after their journal club couldn't get Dr. Huang et al's numbers to add up:

After lead author Dr Huang graciously responded and joined the discussion, it was determined that the calculation of a NNT of 54 to prevent a bloodstream infection was not correct. The number needed to treat was actually 99. No explanation was given; it seems to have been a simple calculation error. The always-cordial online conversation progressed, and ultimately, NEJM posted an erratum in December:

The error had clinical and economic implications, as it would seem to roughly double the cost per prevented infection, and this intervention is on deck as a candidate for (conceivably) worldwide implementation. New cost-benefit analyses are underway including the corrected data. Dr. Flower's estimate is that implementing decolonization would result in a bloodstream infection prevented every 388 patient-days with the intervention in place. Still probably worthwhile, but less resounding than first reported.  But the episode also illustrates the tendency toward error in all of us as individuals and groups, and emphasizes the need for multiple layers of error-checking, regardless of (or especially because of) the prestige of authors, institutions, and journals. This was not a complex statistical derivation -- it was a relatively simple calculation, and the error was discovered using techniques most of us learned in our evidence-based medicine classes in med school. Obviously, I did not notice it -- because I didn't double-check their numbers, either. On the bright side, we've been shown a great example of the potential of crowd-sourcing and digital media in providing a valuable, effective new layer to research verification. Kudos to these keen-eyed bloggers and social media sleuths -- and my apologies for missing this and reporting the erroneous information to you. Anyone interested in becoming PulmCCM's honorary statistician / peer-reviewer?