Predicting Productivity Returns on Investment
Thirty Years of Peer Review, Grant Funding, and Publication of Highly Cited Papers at the National Heart, Lung, and Blood Institute
This article requires a subscription to view the full text. If you have a subscription you may use the login form below to view the article. Access to this article can also be purchased.
There are conflicting data about the ability of peer review percentile rankings to predict grant productivity, as measured through publications and citations. To understand the nature of these apparent conflicting findings, we analyzed bibliometric outcomes of 6873 de novo cardiovascular R01 grants funded by the National Heart, Lung, and Blood Institute (NHLBI) between 1980 and 2011. Our outcomes focus on top-10% articles, meaning articles that were cited more often than 90% of other articles on the same topic, of the same type (eg, article, editorial), and published in the same year. The 6873 grants yielded 62 468 articles, of which 13 507 (or 22%) were top-10% articles. There was a modest association between better grant percentile ranking and number of top-10% articles. However, discrimination was poor (area under receiver operating characteristic curve [ROC], 0.52; 95% confidence interval, 0.51–0.53). Furthermore, better percentile ranking was also associated with higher annual and total inflation-adjusted grant budgets. There was no association between grant percentile ranking and grant outcome as assessed by number of top-10% articles per $million spent. Hence, the seemingly conflicting findings on peer review percentile ranking of grants and subsequent productivity largely reflect differing questions and outcomes. Taken together, these findings raise questions about how best National Institutes of Health (NIH) should use peer review assessments to make complex funding decisions.
- National Institutes of Health (US)
- National Heart, Lung, and Blood Institute (US)
- peer review
- ROC curve
- Received May 8, 2015.
- Revision received June 11, 2015.
- Accepted June 18, 2015.
- © 2015 American Heart Association, Inc.