Compounding Error: The Afterlife of Bad Science

Judit Dobránszki

When errors are discovered in published papers, corrections need to be made quickly and completely. Failure to do this poses significant risks for authors, editors, journals, and publishers. Scientific or other errors in published papers decrease the confidence that readers and the peer community may have in overall scientific credibility. Erroneous literature may pose a risk to the wider academic pool and to the public. A manuscript that references another manuscript that contains errors in effect promulgates the error deeper into the literature, biasing the downstream literature. Thus, when errors are detected and reported to an editor, journal, or publisher they should be rapidly corrected, even in the absence of the approval of the author(s), provided that the claims are factual and validated.

The issue of the scale of the error also has to be taken into consideration. Journal editors who take reports of errors seriously and seek to correct the literature in a swift and thorough manner deserve praise for doing so. This constitutes editorial independence, i.e., the freedom to make choices that are in the best academic and scholarly interests of the scientific community. When authors are aware of errors but fail to correct them, it should be classified as misconduct.1

Currently published scientific papers, with or without errors, serve as the bulwark of higher education systems around the world for undergraduate, graduate, and postgraduate studies. Swift correction of errors in the scientific literature is therefore particularly important for students, who do not have the experience to detect what is correct and what might be erroneous, and who thus assume the validity of what is published.

Moreover, many biomedical studies are used to improve health care, agriculture, and society, and provide technical advances that affect the daily lives of the public, so we need to develop methods to correct faulty published literature, thereby ensuring its veracity and accuracy, even long after it has been published. Participation in journal clubs, which meet regularly to evaluate recent articles in the academic literature of a particular branch of science or philosophy, and whose members have wide-ranging backgrounds, qualifications, and expertise, provide one avenue to correcting the published literature through post-publication peer review. PPPR, the independent review of literature after it has been published, thus fortifies the reliability of published science.2

Responsible correction increases trust among scientists and society alike in scientific findings and the broader scientific literature, as well as in the competence of editors, editorial boards, and publishers. The converse is true when errors are not corrected. We believe that self-correction of the science literature is an over-hyped concept. As Ferric C. Fang, a professor of laboratory medicine at the University of Washington School of Medicine, and Arturo Casadevall, the chair in microbiology and immunology at Albert Einstein College of Medicine, aptly state: “This is the current unsatisfying state of affairs with regard to self-correction. Science can be self-correcting, but this requires the concerted efforts of scientists, journals, institutions and governments. The self-correction process lacks transparency and consistency, and many potential conflicts of interest may interfere along the way.”3 In response to such weaknesses, PPPR has rapidly emerged as a recognized form of correcting the science literature.4

In fact, PPPR is currently the only viable and effective way of retroactively correcting the published literature. PPPR identifies erroneous methodology, flawed analyses, manipulated figures, and inappropriate citations, and challenges hyperbolic claims or conclusions. PPPR also identifies problems or inconsistencies on the part of editors, journals, and publishers. It examines editorial and publication protocol and their ethical basis, and attempts to hold editors, journals, and publishers accountable for what they have published.

However, PPPR has not yet established a set of rules or norms. Although PubPeer and PubMed Commons are moderated, secrecy still exists about who moderates and what assessment criteria are used. Many academics justifiably fear retribution—from editors of the papers under review, from the scientists whose work they are critiquing, from their own colleagues or peers—and therefore shy away from getting involved in PPPR and in public criticism of the published literature or the scientific establishment. Those who don’t feel comfortable posting a critique by name, or even anonymously, could easily contact a colleague to represent their concerns to editors or to the public. However, such individuals should be held fully accountable for all claims made through a third party. Ultimately, the best way to participate in PPPR is to take responsibility for one’s opinions, including criticisms, by name, since the consequences of citing faulty literature and the continual citation even of retracted papers are vast.

Risks of Citing Retracted Literature

Sometimes retracted papers continue to be cited, even years after their actual retraction. As conservation biologists A. Mel Cosentino and Diogo Veríssimo state, “The continued citation of retracted papers is a major issue because it spreads misinformation throughout the scientific literature, providing a false premise for future research and thus seriously impacting the advancement of science.”5 Moreover, it can have devastating effects on society.6

An illustration of these risks is presented by a group of health scientists at the University of South Australia, who examined how frequently a retracted paper, “Effects of Omega-3 Polyunsaturated Fatty Acids in Inflammatory Markers in COPD,” by Wataru Matsuyama et al., was cited years after it was retracted for misconduct (data falsification).7 Ashley S. Fulton and her co-authors found that even after retraction in 2008, the paper had been cited fifty-two times, and that only two of these citations included the retraction notice. We recently examined the PDF of the retracted article accessible at the URL for Chest® (an Elsevier-published journal), the official publication of the American College of Chest Physicians), on the ACPP website. It did not contain “RETRACTION” red-stamped across every page—unlike a typical retraction from another Elsevier journal,8 as recommended by the Committee on Publication Ethics (COPE) guidelines for retractions: “Notices of retraction should be clearly identified as a retraction (i.e. distinct from other types of correction or comment).”9

Elsevier is a member of COPE, “a forum for editors and publishers of peer reviewed journals to discuss all aspects of publication ethics” that “advises editors on how to handle cases of research and publication misconduct.”10 Even though the Chest® website clearly displays the erratum and retraction notice for the piece by Matsuyama et al.,11 the Elsevier (i.e., Science Direct®) URL for the article is not marked as retracted, nor does it carry any notice of retraction, thus directly misleading the public into thinking that this paper is both valid and correct.12 Finally, this retracted paper is still being sold—via Science Direct®, Elsevier’s “leading information solution for researchers, teachers, students, health care professionals and information professionals”—with a pay-per-download price of $27.95, even eight years after the retraction.13

The PDF file that is downloaded from Science Direct® does not contain a “RETRACTION” red-stamped across every page, in direct contravention of COPE retraction guidelines, which undermines the meaning and importance of COPE. In a separate analysis of Matsuyama et al.’s paper, we found strong indications that the notices and/or policies pertaining to errata and retractions show as much as 61 percent deviation from COPE-established guidelines, as assessed in three ethical bodies and fifteen science, technology, and medicine publishers.14 It is therefore unsurprising that only two of fifty-two citations of the 2005 Matsuyama et al. paper since its 2008 retraction mention that retraction. And it is likely that researchers will continue to cite this retracted paper as the direct result of incorrect labeling on the Science Direct® website.

In a paper describing a similar situation, doctors at the Medical University of Graz Helmar Bornemann-Cimenti, Istvan S. Szilagyi, and Andreas Sandner-Kiesling found that twenty papers by prolific, influential pain management researcher Scott S. Reuben—a Tufts University professor of anesthesiology and pain medicine until he was sentenced to prison for health care fraud—all of which were retracted in 2009 due to data fabrication, continued to be cited between 2009 and 2014. Bornemann-Cimenti et al. found that by 2014, 45 percent of the retracted papers continued to be cited at least once, with only one quarter of them clearly indicating the retracted nature, either on the paper’s URL, on the PDF, or both.15

A recent spate of eight paper retractions, almost twenty corrections, two investigations, and the loss of an EMBO Gold Medal award for beleaguered plant biologist Olivier Voinnet, who was found to have manipulated data16—all within the span of one year—reveals the weaknesses within scientific publishing. That thousands of citations of this faulty and fraudulent work have accrued illustrates how retracted articles can continue to be cited unless proactive efforts are made to prevent it.

In “Scientific Misconduct: Cleaning Up the Paper Trail,” AAAS Science’s staff writer Jennifer Couzin and Katherine Unger listed other examples of retracted pieces with a high number of citations after retraction, sometimes much more than before retraction and even years after retraction.17 And the risks of citing a retracted paper may increase when the publisher’s official websites are not directly accessed to download a piece. The survival of retracted papers in personal libraries and on institutional or commercial websites in the form of copies of a publisher’s version or the final version of a manuscript, which do not contain the retraction note as it appears on the original publisher’s website, may be high. For example, independent researcher/consultant Philip M. Davis detected 321 public copies for 289 retracted papers on non-publishers’ websites.18

Citing retracted research papers and the false information that is spread because of them extends beyond science’s borders and can potentially cause severe damage to society.19 A harmful consequence—the misinformation of the general public and policy makers—was clearly shown by Cosentino and Veríssimo in “Ending the Citation of Retracted Papers.”20 One example includes misinforming the public about the effect of a vaccine on human health as advertised by a highly cited 1998 paper by Andrew J. Wakefield et al., who claimed that a side-effect of the MMR vaccination was behavioral disorders, including autism (in nine out of twelve children studied), because behavioral symptoms occurred in healthy children being vaccinated.21 The false research paper generated more than a thousand citations, even six years after its 2010 retraction,22 and spurred a movement against vaccinations, thereby exposing unvaccinated children to health dangers, as discussed by members of Health Protection Scotland.23 To present an example of spreading misinformation among policy makers, Cosentino and Veríssimo also discuss the development and practice of bird conservation strategies, before the falsified papers were retracted, based on data fabrication and false information on the relationship between the immunology of wild birds and the use of veterinary drugs in domesticated birds.24

The ultimate damage caused by citing retracted literature is to the trust that should prevail among scientists and between scientists and society.

Concerns, Proposals, and Conclusions

In addition to these repercussions, journals and publishers may unfairly benefit from erroneous literature that increases the journal’s “impact factor” (or other journal- or paper-based metrics based on number of citations), which serves as a proxy for a publication’s relative importance within its field, as well as its sales via subscriptions or pay-per-download PDF files. In the case of authors whose papers present problems, their work may continue to be cited, as if no errors exist, giving them unwarranted prominence.25

PPPR allows scientists and others, including anonymous commentators,26 to analyze published literature by reexamining factors that may have escaped the permeable traditional peer review system27 and identifying mistakes, problems, or other issues that may have existed since publication but were either undetected or unreported.28 The scientific integrity of a published paper lies in its validity and reflects the rigor of the peer review system and editorial board in place at the time that that manuscript was approved for publication. Within the broader framework of editorial responsibilities,29 current editors or editors-in-chief who are made aware of errors that exist in the published literature are responsible for correcting erroneous literature, even if they themselves were not responsible for overseeing the peer review or for approving the publication of the problematic paper. In an extraordinarily extreme case, the Netherlands will be spending €8 million to survey all of its researchers in a bid to assess research misconduct or “sloppy science.”30 A more common way of conducting PPPR involves the dedicated, and free, examination of pockets of literature (within a scientist’s realm of expertise, or not) to identify problems, as was done by Jaime A. Teixeira da Silva, a co-author of this article, with papers published from 2008–2016 by Paolo Macchiarini—a thoracic surgeon and former professor of regenerative medicine at Karolinska Institutet in Stockholm who is currently being investigated for research fraud.31 Authors, editors, journals, and publishers are responsible for ensuring that retracted papers do not continue to be cited, but this may not always be in their control, as, for example, when pirated papers posted on Sci-Hub remain in their unretracted state.32 Nevertheless, all efforts that can be made must be made.

  • Share
Most Commented

April 16, 2021

1.

Social Justice 101: Intro. to Cancel Culture

Understanding the illogical origin of cancel culture, we can more easily accept mistakes, flaws, and errors in history, and in ourselves, as part of our fallen nature....

April 19, 2021

2.

Critical Race Theory and the Will to Power

A review of "1620: A Critical Response to the 1619 Project" by NAS President Peter W. Wood....

May 30, 2018

3.

The Case for Colonialism

From the summer issue of Academic Questions, we reprint the controversial article, "The Case for Colonialism." ...

Most Read

August 23, 2021

1.

Testing the Tests for Racism

What is the veracity of "audit" studies, conducted primarily by sociologists, that appear to demonstrate that people of color confront intense bias at every level of society?...

May 30, 2018

2.

The Case for Colonialism

From the summer issue of Academic Questions, we reprint the controversial article, "The Case for Colonialism." ...

March 29, 2019

3.

Homogenous: The Political Affiliations of Elite Liberal Arts College Faculty

A study on the partisanship of liberal arts professors at America's top universities. ...