This week’s podcast is on using small experiments (many of which are failures) as a way to get better success as an end result, with a product or service.
Three economists, Pierre Azoulay, Gustavo Manso, and Joshua Graff Zivin, have rigorously evaluated the Hughes approach. They found that once scientists received HHMI funding, they produced more failures—but also far more highly cited “blockbuster” research papers. In other words: more risk, more reward. Yet HHMI research grants, big as they are, are substantially less than one-tenth of 1 percent of global R&D.
Specifically the abstract of this paper states (emphasis mine):
Specifically, we study the careers of investigators of the Howard Hughes Medical Institute (HHMI), which tolerates early failure, rewards long- term success, and gives its appointees great freedom to experiment; and grantees from the National Institute of Health, which are subject to short review cycles, pre-defined deliverables, and renewal policies unforgiving of failure. Using a combination of propensity-score weighting and difference-in-differences estimation strategies, we find that HHMI investigators produce high-impact papers at a much higher rate than two control groups of similarly-accomplished NIH-funded scientists. Moreover, the direction of their research changes in ways that suggest the program induces them to explore novel lines of inquiry.
The group of scientists that have more freedom to make mistakes early and greater freedom to experiment are the ones that are doing the most ground-breaking work.
(Hat tip to Shawn Graham for pointing this one out to me!)