For Yet Better Science

When science is viewed in isolation from the past and politics, it’s easier for those with bad intentions to revive dangerous and discredited ideas.  

At the heart of the Open Science movement is the conviction that  Open Science is better science . More rigorous. More inclusive. More efficient. More trustworthy. More reproducible. And more impactful for society.  

The “New Statistics” emphasizes effect sizes, confidence intervals, meta-analysis, and the use of Open Science practices. We present three specific ways in which a New Statistics approach can help improve scientific practice: by reducing overconfidence in small samples, by reducing confirmation bias, and by fostering more cautious judgments of consistency. We illustrate these points through consideration of the literature on oxytocin and human trust, a research area that typifies some of the endemic problems that arise with poor statistical practice.  

I often like to discuss science online and I’m also rather partial to topics that promote lively discussion, such as  climate change ,  crime statistics  and (perhaps surprisingly) the  big bang . This inevitably brings out the  trolls .  

To understand disease, scientists are producing comprehensive  omics  datasets. However, the majority of these are Eurocentric. Recently, the inclusion of patients from Asia and the Middle East in genomic analyses uncovered unique loci linked to COVID-19 severity. This demonstrates that focusing on diversity and underrepresented populations can benefit all.  

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.  

Collaborations between the academy and governments promise to improve the lives of people, the operations of government, and our understanding of human behavior and public policy. This review shows that the evidence-informed policy movement consists of two main threads: ( a ) an effort to invent new policies using insights from the social and behavioral science consensus about human behavior and institutions and ( b ) an effort to evaluate the success of governmental policies using transparent and high-integrity research designs such as randomized controlled trials. We argue that the problems of each approach may be solved or at least well addressed by teams that combine the two. We also suggest that governmental actors ought to want to learn about  why  a new policy works as much as they want to know  that  the policy works. We envision a future evidence-informed public policy practice that ( a ) involves cross-sector collaborations using the latest theory plus deep contextual knowled