New York, NY, May 20, 2021 – The National Association of Scholars (NAS) has released a new report examining the consequences of irreproducible science on government policy and regulation. The first report of Shifting Sands: Unsound Science and Unsafe Regulations focuses on PM2.5 Regulation and irreproducible research in the field of environmental epidemiology, which informs the U.S. Environmental Protection Agency’s (EPA) policies and regulations. The report finds compelling circumstantial evidence that the environmental epidemiology literature on PM2.5—specifically for mortality, asthma, and heart attack claims—has been affected by statistical practices that have rendered the underlying research untrustworthy.
“The scientific world incentivizes the publishing of exciting research with significant association claims—but not reproducible research,” explained NAS Director of Research David Randall. “This encourages researchers, wittingly or negligently, to use a variety of statistical practices to produce positive, but likely false, claims.”
Shifting Sands argues that federal agencies would benefit from more rigorous and transparent statistical processes in laying the foundation for regulations. If these regulatory regimes are founded on faulty science, it could threaten to undermine public policy initiatives, and especially public trust in government-funded research. This is an outcome Americans can ill afford when trust in policymakers is already at an all-time low.
Randall continued: “Replication makes for sound science. Without it, our government is relying on potentially faulty research to create and enforce regulations that have real consequences on the quality of life for millions of Americans. I and my colleagues, Warren Kindzierski and Stanley Young, offer 11 recommendations to bring the EPA’s methodologies up to the level of best available science.” These include:
- Adopt resampling methods as part of the EPA’s standard battery of tests applied to environmental epidemiology research.
- Rely exclusively on meta-analyses that take account of endemic biases, including question research procedure, HARKing, and p-hacking.
- Require pre-registration and registered reports of all research that informs the EPA’s regulations.
- Require public access to all research data used to justify regulations.
- Place a greater weight on reproduced research for informing EPA regulations.
America’s government agencies must shore up the science that underlies its regulatory regime. Failing to do so will have economic consequences and further erode public confidence in the nation’s leading policy experts.
NAS is a network of scholars and citizens united by a commitment to academic freedom, disinterested scholarship, and excellence in American higher education. Membership in NAS is open to all who share a commitment to these broad principles. NAS publishes a journal and has state and regional affiliates. Visit NAS at www.nas.org.
###
If you would like more information about this issue, please contact David Randall at [email protected].
