Tuesday, 1 July 2025

Design Psychocologist - Sample size

Scientists need to decide what question to ask; sometimes the question needs to be more narrow
It also depends on how strong they want their claims to be.


* effect size: something that has a small effect (traffic light impact on commute) will require more samples than something that has a large effect (car accident)
* statistically larger effect sizes produce bigger differences in outcomes, which are visible even with smaller samples
* smaller effect (small button changing conversion 0.5%) - this will need more participants to detect reliably
* how much variability exists
* how certain you want to be
* the more variable your data, the more samples you need; you need enough data points to statistically say this will be repeatable

For UI/UX, "can someone complete this form without getting stuck" does not necessarily need hundreds of people