Wow I’m already surprised at some of the results. It would be awesome to get 100 responses to this survey but I can already see some underlying trending. I finished tons of this kind of statistical analysis during my PhD dissertation. G*Power was (still is) a simple and widely available app to calculate statistical power. The “Power” value is a computation that says for a given type of research - especially surveys - the answers from X number of people very closely would represent the answers that would be the same for a much larger audience. Because of the type of research, it is commonly not necessary to count the larger population. Much of the parameters are established from types of errors rather than total people.
A very high percentage of “survey” style research is looking for trends. How many people think “A” is true? How popular is choice “B”? What percentage of the population thinks “C” is a bad idea? In the statistics world, the method for much of this research is conducted using a two-tailed regression analysis. Faul, Erdfelder, Buchner, & Lang (2009) wrote some great material of this type of research in a book called, “Behavior Research Methods”.
Back to the “Power” value… Figuring out how many survey answers you need is called the Sample Size. The “effect size” and the chance of Type I & Type II errors usually is enough to set the Sample Size. Describing the research error Types is a large and separate discussion but suffice to say:
a Type I error is a pretty bad error so researchers set the tolerance pretty low (only 5%, α = .05)
a Type II error is not so bad so researchers let it slide more (20%, ß = .20)
A research author named Weller wrote about this stuff in 2014 in “Field Methods” called Sample Size Estimation: The Easy Way.
Jacob Cohen was the wizard who defined much of this stuff in terms of computing sample size and effect size. He wrote “Statistical Power Analysis for the Behavioral Sciences” in 1988 - it is a major work and has set the stage for many modern statistical research methods. If I use some of these widely accepted parameters in a software tool like G*Power, I get a Power value of 92.
That means if you got about 100 responses to this survey (92), your results would very closely resemble the results you would get if you were able to survey the entire world.
That’s very interesting! Looks like we have a long way to go and this morning I thought of a new option I’m curious about. Unfortunitly it will be behind the others in responses but I’ll add it anyways.
Ive been updating the title to reflect how many more people we need to make it to 92. I’ll try to keep bumping this post but maybe we can get facer to pin it.
