By Rob Mitchum // December 9, 2015
In two recent studies, CI Senior Fellows James Evans and Andrey Rzhetsky built a network of millions of papers to ask an important question: is scientific research living up to its potential? Their analysis, conducted with UCLA’s Jacob Foster and CI Director Ian Foster, found that science increasingly explores more incremental and conservative questions, avoiding the riskier research that carries higher probability of both failure and field-changing discovery. Their conclusion, that institutional pressures on scientists to publish often and consistently bring in funding, nudges researchers toward “safer” experiments, slowing the pace of innovation.
A long feature by reporter Paul Voosen at The Chronicle of Higher Education picks up that thread, putting it in the broader context of concern about the lack of risk-taking in science. Voosen traces the history of research universities from their post-war mission of cutting-edge science, to today’s practice of making hiring and tenure decisions based on publications and grant success, incentivizing more conservative science. The argument is supported by the recent papers from Knowledge Lab and CI, which find that science as a whole is less adventurous.
So incentives may deter high-risk research, but that may not mean the actual work has changed in recent years. Is this just how science has always been practiced? Intrigued by that question, Mr. [Jacob] Foster, the UCLA sociologist, and three colleagues have found a new way to judge the balance between tradition and innovation in science. And it suggests that in recent years science has become more hidebound.
Their method, put simply, involved scraping the abstracts of millions of biomedical papers published from 1983 to 2008. Examining papers that dealt with biochemistry, they built a network, with each chemical compound as a node in it; when two molecules were mentioned in the same abstract, they added a link between the corresponding nodes. By 2008 the network had 181,078 nodes and more than 10 million links.
Analyzing such networks in papers published late this year in the American Sociological Review and the Proceedings of the National Academy of Sciences, the team found scientists increasingly exploring existing links, rather than venturing out in a rapidly expanding terra incognita. From 1987 to 2008 that preference increased as much as sixfold.
Voosen also highlights several recommendations for reversing this trend, such as funding individual investigators or spreading risk across departments and institutions. While none of these options would completely restore the risk-taking spirit of science, they would reverse some of today’s stifling conservatism, he concludes.
Science is a balance. It will never be perfectly efficient, as in Mr. Foster’s model. But every day — because of the incentives universities and funding agencies create — sees an invisible loss as bold ideas are postponed, ignored, or never even conceived.
The recent PNAS paper by Rzhetsky et al. also inspired an anonymous editorial in the prestigious journal Nature, which again urged more risk-taking research, saying “it’s time to oil the gears” of science. The editorial cites the authors’ recommendations for reform, while adding its own vote for a wider mix of grants and less reliance on “bibliometrics” that treat publications and citations as the best way of measuring scientific accomplishment.
Attempts to hit the publishable ‘sweet spot’ by avoiding both the prosaic and the risky are likely to reduce the efficiency of scientific discovery. But a fashionably despairing cry of ‘Science is broken!’ is not the way forward. The wider virtue of Rzhetsky et al.’s study is that it floats the notion of tuning practices and institutions to accelerate the process of scientific discovery. The researchers conclude, for example, that publication of experimental failures would assist this goal by avoiding wasteful repetition. Journals chasing impact factors might not welcome that, but they are no longer the sole repositories of scientific findings. Rzhetsky et al. also suggest some shifts in institutional structures that might help promote riskier, but potentially more groundbreaking, research — for example, spreading both risk and credit among teams or organizations.
For more on this research, read our article on the papers from last month.