The Establishment Defends Irreproducibility

David Randall

A recent article by Robinson Meyer in The Atlantic continues the drumbeat of opposition by the scientific establishment to the EPA’s proposed rule Strengthening Transparency in Regulatory Science. (The NAS has provided a public comment in favor of that rule.) Meyer quotes various scientists as saying that forbidding the use of un-transparent data will cripple public health regulation. In making this argument, both Meyer and the scientists he quote fail to mention several previous points of contention. Readers interested in this issue should find these points worth reviewing.

Meyer does not mention the possibility of de-identifying previous research, rendering it anonymous, and thus preserving the anonymity of participants. There is legitimate debate as to how much such research would cost: the EPA claims that it would cost more than $250 million a year—although this may be an inflated estimate, provided to deter would-be reformers. There is also the question of how effective de-anonymizing techniques can be, in an age of massive computational power.

Americans should debate the cost of anonymization, its priority within EPA spending, and how best to de-anonymize data. The NAS favors a substantial shift of resources by the EPA to anonymize data, using the best available techniques, sufficient in scale to allow for the de-identification within 10 years of the majority of research used by the EPA to inform its regulations. We urge the scientific establishment to join us in this call, which will provide the EPA properly reproducible scientific data.

Meyer also does not mention the possibility that scientists can use publicly accessible data to study public health issues—as some have already begun to do.

“This is a very highly contentious political climate, and we are taking the extra step to be as transparent as we can be,” says biostatistician Francesca Dominici of the Harvard T. H. Chan School of Public Health in Boston, who led the studies. In one, published in The New England Journal of Medicine in June 2017, she and her colleagues used publicly accessible air pollution data and records compiled by the federal government’s Medicare health insurance program to show that even modest pollution reductions could save more than 10,000 lives per year. In another, published in JAMA last December, they linked short-term exposure to air pollution levels below current limits to premature death among the elderly.

Stricter transparency regulations may annoy scientists, and add marginally greater costs to their procedure—but the contention that science will be crippled depends on the presumption that scientific ingenuity cannot find workable ways to meet strict transparency standards. Dominici’s research already provides evidence that this presumption is incorrect.

The NAS urges the scientific establishment to take account of these issues and developments as it critiques the EPA’s proposed rule. And we repeat our recommendation to the EPA that it adopt strict transparency regulations—with redoubled confidence that American scientists can produce solid research that meets these regulatory standards.

  • Share

Most Commented

May 1, 2017


Binghamton University Responds to NAS’s Report

SUNY Binghamton offers critiques and corrections to Outsourced to China. ...

March 23, 2011


Looking for Answers? Ask a Scholar!

NAS is partnering with Intellectual Takeout to answer questions that call for scholarly judgment and can't be answered by Wikipedia. ...

September 5, 2014


Post Traumatic Slave Syndrome

"[PTSS] provides an evidence-proof explanation that lifts away moral responsibility from those engaged in self-destructive, anti-social, and criminal behavior."...

Most Read

May 15, 2015


Where Did We Get the Idea That Only White People Can Be Racist?

A look at the double standard that has arisen regarding racism, illustrated recently by the reaction to a black professor's biased comments on Twitter....

June 15, 2020


See Clearly, Decide Wisely, Act Justly

An Open Letter to the Converse College Community in response to its administrative decisions relating to diversity and inclusion on campus. ...

June 12, 2020


Defending Against the Jacobins

As college presidents shirk academic reform in favor of more activist programs, the question remains: Will boards of trustees walk with higher ed. leadership over the edge of the cliff?...