Administrative discretion in scientific funding: Evidence from a prestigious postdoctoral training program

2020
The scientific community is engaged in an active debate on the value of its peer-review system. Does peer review actually serve the role we envision for it-that of helping government agencies predict what ideas have the best chance of contributing to scientific advancement? Many federal agencies use a two-step review process that includes programmatic discretion in selecting awards. This process allows us to determine whether success in a future independent scientific-research career is more accurately predicted by peer-review recommendations or discretion by program staff and institute leaders. Using data from a prestigious training program at the National Institute of Health (NIH), the Ruth L. Kirschstein National Research Service Award (NRSA), we provide evidence on the efficacy of peer review. We find that, despite all current claims to the contrary, the existing peer-review system works as intended. It more closely predicts high-quality science and future research independence than discretion. We discover also that regression discontinuity, the econometric method typically used to examine the effect of scientific funding, does not fit many scientific-funding models and should only be used with caution when studying federal awards for science.
RESEARCH POLICY
卷号:49|期号:4
ISSN:0048-7333
收录类型
SSCI
发表日期
2020
学科领域
循证经济学
国家
美国
语种
英语
DOI
10.1016/j.respol.2020.103953
其他关键词
GRANT APPLICATIONS; SCIENCE; IMPACT; REVIEWERS; QUALITY; BIAS
EISSN
1873-7625
资助机构
NRSA funding; NIH Division of Biomedical Research Workforce (DBRW)United States Department of Health & Human ServicesNational Institutes of Health (NIH) - USA [HHSN276201300089U]
资助信息
This paper reports the results of research and analysis undertaken by the National Institutes of Health (NIH) staff, consultants, and contractors. We acknowledge generous support from Lisa Evans, James Holt, Henry Khachaturian, Kay Lund, Sherry Mills, Tony Moore, Matthew Notowidigdo, Walter T. Schaffer, and Jennifer Sutton. This work would not have been possible without the amazing data skills and research assistance of Frances D. Carter-Johnson, Maria Isabel Larenas, Ryan Pfirrmann-Powell, and Ying Zeng within the NIH Division of Biomedical Research Workforce Programs, as well as participants from the 2018 Association for Public Policy Analysis and Management (APPAM) fall research conference and seminar participants at Iowa State University. We appreciate the time and candor given to us by various NIH institute program officers and training directors who so graciously agreed to be interviewed in an effort to help us truly understand the NRSA funding process. We also give very special thanks to Pierre Azoulay, Hyungjo Hur, Ming Lei, Jon Lorsch, Julie Mason, Michael Spittel, and Jonathan Wiest for taking the time to review and provide critical feedback. This analysis began while Misty L. Heggeness was employed at the National Institutes of Health. Donna Ginther acknowledges financial support from NIH Division of Biomedical Research Workforce (DBRW) (Contract No. HHSN276201300089U) to NETE Solutions. Any errors in this paper are the sole responsibility of the authors. The views presented here do not reflect any official position of the National Institutes of Health, U.S. Census Bureau, the Federal Reserve System, the Federal Reserve Bank of Minneapolis, or the University of Kansas.
被引频次(WOS)
4
被引更新日期
2022-01
来源机构
University of Kansas Federal Reserve System - USA
关键词
Peer review Scientific funding Biomedical workforce Innovation