Matching Taxpayer Funding to Population Health Needs
Not so Simple
In their critically acclaimed book Decisive, Chip and Dan Heath (from Stanford and Duke Universities) focus on the 4 most pernicious villains of decision-making.1 They are (1) narrow framing, seeing a decision as either X or Y, but not X and Y; (2) confirmation bias, paying attention to only those data that are consistent with prior beliefs; (3) reliance on short-term emotion; and (4) sticking with decisions rather than revising or reversing them if events do not turn out as predicted. The Heath brothers propose that decision-makers adopt a “WRAP” model in which they Widen their options, Reality test their assumptions, Attain distance before deciding, and Prepare to be wrong.
The scientific community and the public are understandably interested in how government research sponsors make decisions. This interest has become increasingly acute as funds contract at the same time as research capacity continues to grow.2 The National Institutes of Health (NIH), as the world’s largest public sponsor of biomedical research, is under considerable scrutiny by many stakeholders,2–4 including scientists whose careers often depend on NIH monies, universities that factor NIH funding into their business models, and the public, which looks to the NIH to enable scientific discoveries that will relieve suffering from disease and disability. Many have voiced concern that the NIH should reconsider its business processes, from scientists who wonder about the value of NIH’s funding of big-science epidemiological studies, clinical trials, and research centers3; from policy makers who wonder whether the NIH’s approach to making funding decisions is outmoded2,4; and from critics like Michael Hanna who writes in this issue of Circulation Research that the NIH is failing to properly match funding to population health needs.5
Hanna argues that NIH funding allocations for disease-based research should be determined entirely by disease burden: if a disease causes XX% of the US population’s disability and premature death, NIH should allocate XX% of its disease-based research funds to study of that disease. He presents data on 29 specific disease conditions whose burdens are represented by the Global Burden of Disease Study’s Disability Adjusted Life Years measurements, comparing those measures to NIH funding levels. Hanna finds that these 29 conditions account for 54% of the US’s Disability Adjusted Life Years; NIH only allocates 44% of its disease-based funding to these 29 conditions. Furthermore, some conditions (eg, malaria, AIDS/HIV, tuberculosis, and breast cancer) are overfunded—that is the ratio of NIH funding to contribution to Disability Adjusted Life Years exceeds 100%—whereas other conditions (eg, migraine and chronic obstructive lung disease) are underfunded. Hanna calls on policy makers and NIH to “readjust appropriations … to be consistent with the current distribution of burden-of-illness in the American population.”
At first glance, Hanna’s arguments and proposals seem reasonable. What possible objection could there be for identifying the distribution of population disease burden and then, based on that measure alone, recalibrating disease-based research funding according to that distribution?
Using the Heath brother’s WRAP model,1 one might consider 4 possible concerns with a disease-burden-only approach:
Widen options: although the NIH should consider disease burden when allocating funds, it should consider other criteria.
Reality test assumptions: we should test the assumptions that NIH is failing to allocate funds according to disease burden and that more research funding on disease X automatically translates into less disease X in a straightforward, linear manner.
Attain emotional distance: health and disease are intensely personal because all of us are directly affected. Are parochial appeals leading to inefficient funding decisions?
Prepare to be wrong: the NIH and its constituents should evaluate, using scientifically rigorous approaches, whether its funding approaches are leading to desired outcomes.
We will consider each of these in turn.
First, the NIH might consider widening its options by not only focusing on disease burden but also considering other criteria. Indeed, the Institute of Medicine report that Hanna cites, did exactly that.6 The Institute of Medicine recommended that NIH not only focus on public health needs: it should consider scientific quality, potential for scientific progress, portfolio diversification, and adequate support of infrastructure. The Institute of Medicine cautioned NIH to broaden its approach to public health needs “beyond the medical model … to include the preservation and maintenance of health and function.”6 Furthermore, the Institute of Medicine noted, “The relationship between [burden of disease] data and allocation of research funding will not be simple because health problems are not equally ripe for research advances.”6
Thus, there are other criteria to consider. New technologies or databases may create opportunities likely to yield important findings. New epidemics and globalization present threats to health and opportunities for research; witness the still ongoing Ebola epidemic. For certain kinds of research, industry funding may play a bigger role; NIH funds only a minority of multicenter randomized cardiovascular trials.7
Rare diseases by their nature may offer insights that go beyond affected patients; deciphering the mechanisms of rare diseases may lead to the understanding of the pathophysiology of more common diseases. Indeed there are many examples of rare disease discoveries that led to important advances for patients with common disease: research on a hemorrhagic disease of cattle led to the discovery of warfarin,8 making it possible to reduce stroke risk in atrial fibrillation and to implant mechanical heart valves; research on genetically induced hypercholesterolemia led to the discovery of statins9 and PCSK9 inhibitors10; and research on progeria has offered unique insights into vascular pathology by showing that the disease-causing protein, progerin, is found in all cells and increases with aging.11 Abnormalities of transforming growth factor-beta activity, largely explored in several rare diseases, now seem to be linked to common allergic disorders.12 Technologies being developed for use in rare diseases, such as gene therapy vectors, may conceivably be used in more common diseases. A recent National Heart, Lung, and Blood Institute (NHLBI) working group noted that HIV disease offers a naturally occurring case of an accelerated but commonly occurring pathological process—such as inflammation—that can offer insights to a host of common diseases within the NHLBI mission.13 These stories remind us that science by its very nature is unpredictable, is heavy tailed with a small proportion of discoveries yielding the greatest effect, and nonlinear because complex networks of molecular pathways, disease processes, and socio-behavioral contexts all render meaningless simple maps for finding specific cures.14,15
Second, we should consider Hanna’s assumption that NIH is failing to allocate funds according to burden of disease. In perhaps the most sophisticated analysis done to date, Sampat and colleagues compared NIH funding levels addressing 107 diseases as classified by the NIH’s Research, Condition, and Disease Categorization to disease burden as measured by deaths and hospitalizations.16 The investigators found a strong and statistically significant relationship (r=0.73) and found that certain applied grant mechanisms function as safety values for assuring fine-tuning when needed. Furthermore, coronary artery disease was found to be slightly above the regression line, that is, NIH funding for coronary artery disease was slightly more than would be expected for the number of deaths and hospitalizations attributable to it, whereas hypertension was found to be slightly below the regression line. Thus, for these 2 common cardiovascular diseases, NIH funding seems to be in line with disease burden.
Even so, Sampat and colleagues did not find a perfect 1:1 association between disease burden and NIH funding, which leads us to consider a second assumption that more research funding on a particular disease automatically translates to less disease burden. Manton and colleagues formally assessed this assumption and reported mixed findings: for heart disease and stroke, there were associations between research funding levels and subsequent declines, whereas for cancer and diabetes mellitus, there were no clear associations.17 Even for heart disease and stroke, there appeared to be a distinctly nonlinear threshold pattern—once certain key discoveries were made, it was possible to gain substantially greater effect for any given level of funding.
Third, we should attain emotional distance when making policy decisions. Hanna suggests that “financial allocations … may be influenced by outside lobbying from special interest groups … [and that] the media surely also shapes [perceptions] about the importance of some illnesses.”5 Vociferous and politically active lobbying groups, as well as the media, might appeal to emotional sensibilities when trying to sway decision-makers. Carefully analyzed empirical data suggest that this commonly held notion is overstated. Hegde and Sampat recently reported on an analysis of the association between rare disease lobbying and NIH funding.18 They did find an association—more lobbying predicted more funding—but the effects were subtle and appeared to affect only a small proportion of funds. Levels of lobbying paralleled levels of new scientific opportunity and disease burden, suggesting that lobbying may have a useful information role.18 In any case, it seems that NIH’s multilevel decision-making processes, involving study sections, program officers, and advisory Councils, do not allow for simple appeals to emotion to lead to immediate or short-term increase in funding.
Finally, NIH and its advisors should prepare to be wrong by evaluating and reevaluating the effect of its decisions. Indeed, we at NHLBI have embraced the framework of Results-Based Accountability, as we described in these pages a few years ago.19 We have found, for example, that many of our clinical trials, particularly small surrogate-outcome trials, do not see their primary results published in a timely manner20 and are therefore carefully considering the suggestions of thought leaders who have called on NIH to reconsider its funding priorities for clinical trials.21 We have found that peer-review percentile scores do not predict grant citation effect, leading some NIH leaders to change their funding models.22 Beyond NHLBI, NIH leaders have undertaken reassessments of certain programs, often with the assistance of external stakeholders; just days ago, NIH heeded the advice of an external working group and decided to close the National Children’s Study.23
Hanna’s concerns reflect, appropriately, the extraordinary importance that NIH-funded research has had, and continues to have, on our nations’ goals of enhancing health and economic progress. Yet, Hanna’s disease-burden-only approach may well make us vulnerable to the Heath brothers’ 4 villains of decision-making.1 We at NHLBI welcome, indeed invite, input from all stakeholders to offer feedback and advice for how best to assess our effect and reconsider our business models. We do so with the conviction that by widening options, reality testing assumptions, attaining emotional distance, and preparing to be wrong, we will make the best possible decisions that address our common interest to see a healthier world.
The views expressed in this essay are those of the authors and do not necessarily reflect the official views of the National Heart, Lung, and Blood Institute, the National Institute of Health or the US government.
- Nonstandard Abbreviations and Acronyms
- National Heart, Lung, and Blood Institute
- National Institutes of Health
- © 2015 American Heart Association, Inc.
- Heath C,
- Heath D.
- Alberts B,
- Kirschner MW,
- Tilghman S,
- Varmus H.
- Rosbash M.
- Langer JS.
- Hanna M.
- 6.↵Priority-Setting Processes - Scientific Opportunities and Public Needs - NCBI Bookshelf [Internet]. [cited 2014 Dec 14]; http://www.ncbi.nlm.nih.gov/books/NBK45366/.
- Goldstein JL,
- Brown MS.
- Gordon LB,
- Cao K,
- Collins FS.
- Frischmeyer-Guerrerio PA,
- Guerrerio AL,
- Oswald G,
- Chichester K,
- Myers L,
- Halushka MK,
- Oliva-Hemker M,
- Wood RA,
- Dietz HC.
- Shah MR,
- Cook N,
- Wong R,
- Hsue P,
- Ridker P,
- Currier J,
- Shurin S.
- Lauer MS.
- Manton KG,
- Gu XL,
- Lowrimore G,
- Ullian A,
- Tolley HD.
- Lauer MS.
- Kaltman JR,
- Evans FJ,
- Danthi NS,
- Wu CO,
- DiMichele DM,
- Lauer MS.
- 23.↵NIH cancels massive U.S. children’s study | Science/AAAS | News [Internet]. [cited 2014 Dec 14]; http://news.sciencemag.org/funding/2014/12/nih-cancels-massive-u-s-children-s-study.