The Establishment Defends Irreproducibility

David Randall

A recent article by Robinson Meyer in The Atlantic continues the drumbeat of opposition by the scientific establishment to the EPA’s proposed rule Strengthening Transparency in Regulatory Science. (The NAS has provided a public comment in favor of that rule.) Meyer quotes various scientists as saying that forbidding the use of un-transparent data will cripple public health regulation. In making this argument, both Meyer and the scientists he quote fail to mention several previous points of contention. Readers interested in this issue should find these points worth reviewing.

Meyer does not mention the possibility of de-identifying previous research, rendering it anonymous, and thus preserving the anonymity of participants. There is legitimate debate as to how much such research would cost: the EPA claims that it would cost more than $250 million a year—although this may be an inflated estimate, provided to deter would-be reformers. There is also the question of how effective de-anonymizing techniques can be, in an age of massive computational power.

Americans should debate the cost of anonymization, its priority within EPA spending, and how best to de-anonymize data. The NAS favors a substantial shift of resources by the EPA to anonymize data, using the best available techniques, sufficient in scale to allow for the de-identification within 10 years of the majority of research used by the EPA to inform its regulations. We urge the scientific establishment to join us in this call, which will provide the EPA properly reproducible scientific data.

Meyer also does not mention the possibility that scientists can use publicly accessible data to study public health issues—as some have already begun to do.

“This is a very highly contentious political climate, and we are taking the extra step to be as transparent as we can be,” says biostatistician Francesca Dominici of the Harvard T. H. Chan School of Public Health in Boston, who led the studies. In one, published in The New England Journal of Medicine in June 2017, she and her colleagues used publicly accessible air pollution data and records compiled by the federal government’s Medicare health insurance program to show that even modest pollution reductions could save more than 10,000 lives per year. In another, published in JAMA last December, they linked short-term exposure to air pollution levels below current limits to premature death among the elderly.

Stricter transparency regulations may annoy scientists, and add marginally greater costs to their procedure—but the contention that science will be crippled depends on the presumption that scientific ingenuity cannot find workable ways to meet strict transparency standards. Dominici’s research already provides evidence that this presumption is incorrect.

The NAS urges the scientific establishment to take account of these issues and developments as it critiques the EPA’s proposed rule. And we repeat our recommendation to the EPA that it adopt strict transparency regulations—with redoubled confidence that American scientists can produce solid research that meets these regulatory standards.

  • Share

Most Commented

October 25, 2022

1.

NAS President Peter Wood Addresses the Pending Racial Preferences Cases

Read NAS president Peter Wood's remarks on the upcoming Supreme Court cases, which he presented at a meeting of "Oasis," an informal group of academics and intellectuals based in......

July 25, 2022

2.

Against Transgenderism

The ideology of transgenderism strives to slam shut any door that offers opposition to its attempts to acquire power and control. This statement explains our opposition to such an ideology i......

October 20, 2022

3.

NAS Statement on Nomination of Ben Sasse for University of Florida President

We believe that Senator Sasse would make an excellent president of the University of Florida, and we urge the Board of Trustees to follow the search committee’s recommendation....

Most Read

May 15, 2015

1.

Where Did We Get the Idea That Only White People Can Be Racist?

A look at the double standard that has arisen regarding racism, illustrated recently by the reaction to a black professor's biased comments on Twitter....

October 12, 2010

2.

Ask a Scholar: What is the True Definition of Latino?

What does it mean to be Latino? Are only Latin American people Latino, or does the term apply to anyone whose language derived from Latin?...

May 28, 2009

3.

"O Cosmic Birther!" The Lord's Prayer Meets the American College Textbook

A philosophy professor finds an Internet legend subbing for the Gospel truth and looks in vain for "critical reasoning."...