For Yet Better Science

The panel discusses what inclusive research is, how to conduct it, and what issues and challenges exist about engaging in it. “Inclusive research” has its history as a participatory research method designed to ensure people closest to the issue or problem under study are authentically engaged in the research process rather than simply being “research subjects.” While community-based participatory research has begun to take on greater prominence in the criminal justice realm, such efforts are largely confined to qualitative research inquiries. This panel makes the case that inclusive research can and should apply to a wider array of research questions and methods and that employing it can yield more accurate and policy-relevant evidence. Panelists will also engage in a “myth busting” discussion to address possible challenges to conducting inclusive research and how to overcome them. 

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.

The "New Statistics" emphasizes effect sizes, confidence intervals, meta-analysis, and the use of Open Science practices. We present 3 specific ways in which a New Statistics approach can help improve scientific practice: by reducing over-confidence in small samples, by reducing confirmation bias, and by fostering more cautious judgments of consistency. We illustrate these points through consideration of the literature on oxytocin and human trust, a research area that typifies some of the endemic problems that arise with poor statistical practice.  

Collaborations between the academy and governments promise to improve the lives of people, the operations of government, and our understanding of human behavior and public policy. This review shows that the evidence-informed policy movement consists of two main threads: ( a ) an effort to invent new policies using insights from the social and behavioral science consensus about human behavior and institutions and ( b ) an effort to evaluate the success of governmental policies using transparent and high-integrity research designs such as randomized controlled trials. We argue that the problems of each approach may be solved or at least well addressed by teams that combine the two. We also suggest that governmental actors ought to want to learn about  why  a new policy works as much as they want to know  that  the policy works. We envision a future evidence-informed public policy practice that ( a ) involves cross-sector collaborations using the latest theory plus deep contextual knowled

George Washington Carver was an American agricultural scientist and inventor who promoted alternative crops to cotton and methods to prevent soil depletion. He was one of the most prominent black scientists of the early 20th century.

Hilda Bastian considers post-publication commenting and the cultural changes that are needed to better capture this intellectual effort. 

Although criticisms of the undertakings of science are always needed, especially when it concerns matters of data integrity, publishing practices, and perverse incentives in academia, I am a bit weary of uncritically listening to someone who appears to have no established career in either science or science journalism (the only pieces published under his own name appear only on the blog he runs). And his pieces do come off with a bit of crankiness – reading them alone one would think that the scientific establishment is infested with data forgery, devised by scientists whose only wish is to publish in fancy journals at any cost.