on scientific consensus

Consensus is the most important part of science that many people don’t understand and that isn’t in the high school text books.’ – Naomi Oreskes [1]

Astrophysicist Geraint Lewis writes,

Science has little to do with “proving” anything… the media continually tells us that science proves things… [Rather,] science is like an ongoing courtroom drama, with a continual stream of evidence being presented to the jury… the jury is constantly updating its view… But no verdict of absolute guilt or innocence is ever returned, as evidence is continually gathered and more suspects are paraded in front of the court…’ [2]

Employing a similar metaphor, Oreskes articulates that scientific consensus is not an ordinary jury. Rather, ‘it’s a special kind of jury… it’s a jury of [people] who have Phds… [It’s] the collective knowledge of all of the scientists who have worked on a particular problem’. [3]

A traditional jury has conversations, shut away in a room. This Phd jury has conversations which happen through activities such as: creating experiments in collaboration, conducting blind peer reviews of theories and experiments, publishing research in community-edited and read journals, replicating and reproducing previous experiments, participating in conferences, and more.

To be clear, not all scientist involved in peer review processes have Phds. Various levels of scientists contribute to some community discussions. Some processes are very formal, with ‘gate keepers’, while other aspects of consensus are more open. Among scientists, there is a great about of debate about which sorts of peer review are effective. [9][10]

All that said, consensus conversations are a community-wide effort. They are slow and they are on-going. This science-jury does not offer a verdict of absolute guilt or innocence.

Nonetheless, they do articulate their findings. They do speculate about which theories and experiments they find compelling, which they do not, and why.

Even without a definitive verdict, these juries do say something. Sometimes they mostly agree. In other situations they slightly disagree. In the midst of such agreement-and-disagreement, how do you know how much consensus there is about any given theory or experimental finding?

To explain this, I put to rest the metaphor of a jury.

Imagine that someone added a drop of ink into water. There are no clear, clean cut borders between the ink and the water. Still, you can see that there is an area, a cloud of ink. Consensus is a bit like this ink droplet.

If there is particularly strong evidence, sometimes a cloud of ink-agreement will dissipate throughout an entire glass. For example, the scientific consensus that the Earth is spherical would create a very ink-full glass. [4]

In many other situations, only a portion of scientists agree. For example, some scientists agree that mindful practices are an effective treatment for people suffering from depression. [5][6] Meanwhile, other scientists suggest that the current research does not necessarily support such a finding. [7] Other scientists may take a different position altogether. [8][11][12]

The question moves from ‘what does science say…?’ to ‘How much agreement is there about the theory I like, or the experiment I am interested in?’ And, ‘What do the other ink-clouds have to say? What reasons do they give for occupying a different position?’

The more ink-filled a cup is with one ink-color, the closer this cup is to scientific consensus, the stronger the evidence or theory is considered to be. The smaller the ink cloud, the less strong, the further from scientific consensus.

source notes

1. https://www.npr.org/transcripts/516709308

2. Lewis, G. (2021). Where’s the proof in science? There is none. The Conversation. Retrieved 28 March 2020, from https://theconversation.com/wheres-the-proof-in-science-there-is-none-30570.

3. Staff N. Naomi Oreskes: Why should we believe in science? WBUR. https://www.wbur.org/npr/516709308/naomi-oreskes-why-should-we-believe-in-science. Published February 24, 2017. Accessed October 27, 2021.

4. Youtube.com. (2021). Retrieved 28 March 2020, from https://www.youtube.com/watch?v=hLPPE3_DVCw.

5. KL; LSES. Mindfulness-based therapies in the treatment of somatization disorders: A systematic review and meta-analysis. PloS one. https://pubmed.ncbi.nlm.nih.gov/23990997/. Accessed March 28, 2020.

6. How mindfulness may change the brain in depressed patients. https://www.mindful.org/how-mindfulness-may-change-the-brain-in-depressed-patients/. Accessed March 28, 2020.

7. Farias M, Wikholm C. Has the science of mindfulness lost its mind? BJPsych bulletin. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5353526. Published December 2016. Accessed March 28, 2020.

8. Lindahl JR, Fisher NE, Cooper DJ, Rosen RK, Britton WB. The varieties of contemplative experience: A mixed-methods study of meditation-related challenges in western Buddhists. PLOS ONE. https://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0176239. Accessed February 28, 2020.

9. 4, R. A. A., 5, L. P. A., & 9, M. A. (n.d.). 108: Carl Bergstrom on information, disinformation, and Bullshit. Sean Carroll. Retrieved September 2, 2020, from https://www.preposterousuniverse.com/podcast/2020/08/03/108-carl-bergstrom-on-information-disinformation-and-bullshit/

10. Andrew, Andrew, Phil, Phil, Lizzie, Lizzie, Hullman, J., Hullman, J., Anoneuoid, Weissman, M., Schechter, C., Joshua, Alper, P., Chetan, Higgs, M., Lakeland, D., Littleboy, D. J., (Smith), M., C, B., … Kass, M. (2022, January 2). Statistical Modeling, causal inference, and social science. Statistical Modeling Causal Inference and Social Science. Retrieved January 2, 2022, from https://statmodeling.stat.columbia.edu/

11. Farias, M., Maraldi, E., Wallenkampf, K. C., & Lucchetti, G. (2020, August 21). Adverse events in meditation practices and meditation‐based therapies: A systematic review. Wiley Online Library. Retrieved January 2, 2021, from https://onlinelibrary.wiley.com/doi/full/10.1111/acps.13225

12. Willoughby B. Britton, J. R. L. (n.d.). Defining and measuring meditation-related adverse effects in mindfulness-based Programs – Willoughby B. Britton, Jared R. Lindahl, David J. Cooper, Nicholas K. Canby, Roman Palitsky, 2021. SAGE Journals. Retrieved January 2, 2022, from https://journals.sagepub.com/doi/10.1177/2167702621996340

%d bloggers like this: