Science replications are fun

Reading: We Should Do More Direct Replications in Science, Stuart Buck, Harvard Data Science Review, 1 Aug 2024.

It seems a no-brainer to me:

[…] science funders such as the National Institutes of Health (NIH) and the National Science Foundation (NSF)—which will spend nearly $60 billion this year, collectively—should dedicate at least 1/1,000th of their budgets to doing more replication studies. Even $50 million a year would be transformative, and would ensure that we can have higher confidence in which results are reliable and worth carrying forward into future work.

The article is a great review of all the reasons why we should do more replication work, including flushing out tacit knowledge or identifying gaps in descriptions of work.

How hard could that be? Pretty hard. In the Reproducibility Project in Cancer Biology project: 

In every single case, the team had to reach out to the original lab, which often was uncooperative or claimed not to recall what had actually happened in the study.

What the article doesn’t mention—maybe because it’s not true for most people—is that it’s fun. In software-related reproduction, yes there’s tons of frustration, but you learn a lot and sometimes find bugs.

Related to this, there are “new science” programmes that have replication built in. The one I’m aware of is The National Institute on Aging’s  Interventions Testing Program which carries out tests simultaneously at three labs.