Study Size
Size is an aspect of experiment design which is important to consider. At times, this is fairly straightforward: A study with 1,000 test subjects is generally more reliable than a study with 5 test subjects. [1]
Yet, a small study size does not necessarily mean that it has not been done well, or that the science was shoddy. Often times small studies are small simply due to a lack of funding. Typically such studies are pilot projects, conducted with an intent to be expanded later. Nonetheless, smaller studies need to be regarded as bearing a smaller amount of weight within the frame of scientific consensus.
A minimum sample size should include 30 test subjects. As Christopher Rout, of University of KwaZulu-Natal, notes, “It’s not that “30 in a sample group should be enough” for a study. It’s that you need at least 30 before you can reasonably expect an analysis based upon the normal distribution (i.e. z test) to be valid.” [2]
An article from Science Buddy puts it like this, “… a [general] survey with only 10 random participants [is] not reliable… If you increase the sample size to 100 people, your margin of error falls…” The bigger your study size, the smaller your margin of error. “The bottom line is, you need to survey a lot of people before you can start having any confidence in your results.” [3]
A few websites, such as FlexMR [4], or Survey Systems [5] provide sample size calculators. They show, in a simple way, about how many subjects might be needed for a given population. Such considerations can help you get a clearer sense about how study size relates with reliability.
Study Length
Another aspect of study size to consider is the duration or length of study. Studies vary a great deal in how they are created, with big differences in their time length. [6][7]
Often we practitioners are looking into clinical trials. We want to know more about – if, how, and for whom – a particular treatment is helpful to people.
For example, if a practice is intended to be done regularly, for a year or more – a science study which only investigates a three week, or a three month, period may not provide a robust picture of that treatment. Science-related media articles tend to exaggerate findings, stating conclusive remarks about whether or not a particular practice ‘works’.
When you bring science studies about clinical research to your audience, my hope is that you let them know the size of the study, as well as the length of study. Rather than adding to the sort of hype common in media hubs, I hope you help your audiences understand research in nourishing ways.
–
sources
- Brysbaert, M. (2019, July 19). Journal of Cognition. Retrieved January 2, 2022, from https://www.journalofcognition.org/articles/10.5334/joc.72/
- Rout, Christopher. (2015). Re: What is the rationale behind the magic number 30 in statistics?. Retrieved from: https://www.researchgate.net/post/What_is_the_rationale_behind_the_magic_number_30_in_statistics/56103b806307d9970e8b4613/citation/download.
- Sample size: How many survey participants do I need? Science Buddies. (n.d.). Retrieved March 2, 2020, from https://www.sciencebuddies.org/science-fair-projects/references/sample-size-surveys
- Martin, C. (n.d.). The Quantitative Research Sample Size Calculator. Retrieved January 2, 2020, from https://blog.flexmr.net/sample-size-calculator
- Sample size calculator. Sample Size Calculator – Confidence Level, Confidence Interval, Sample Size, Population Size, Relevant Population – Creative Research Systems. (n.d.). Retrieved January 2, 2020, from https://www.surveysystem.com/sscalc.htm
- Kral, T. R. A., Schuyler, B. S., Mumford, J. A., Rosenkranz, M. A., Lutz, A., & Davidson, R. J. (2018, July 7). Impact of short- and long-term mindfulness meditation training on amygdala reactivity to emotional stimuli. NeuroImage. Retrieved January 2, 2022, from https://www.sciencedirect.com/science/article/abs/pii/S1053811918306256
- How long do clinical trial phases take? Massive Bio. (2021, October 15). Retrieved January 2, 2022, from https://massivebio.com/how-long-do-clinical-trial-phases-take/
Relational Awareness 