Sample size in experimental advertising research

The sample size is a term used in research to define the number of participants who took part in the experiment. This group of participants is selected from the general population. It is considered a representative of the actual population for that specific experiment. A subsample is the group of participants attributed to one of the experimental conditions.

Type II errors: The problem of having too few participants

According to Geuens and De Pelsmacker (2017), an experimental study must have an adequate sample size. In other words, the sample size should be sufficient to find a statistically significant effect when this effect should occur because it is theoretically relevant. Small samples decrease the power in statistical testing, and weak power leads to type II errors. This type of error occurs when existing theoretically relevant effects cannot be confirmed. When the sample size is too small, a researcher may not find statistical significance when testing a hypothesis. However, the effect exists, and the hypothesis was supposed to be validated.

Type I errors: Too many participants also lead to errors

At the same time, Lenth (2001) argued that the sample size should not be too large because one would find a statistically significant effect when this effect should not occur because it is theoretically not relevant. Large samples increase the power in statistical testing, and too much power can lead to type I errors. This type of error occurs when a nonexisting effect is found to be statistically significant. That is to say; when the sample is too large, the statistical power is too big that even a relatively small mean difference may be statistically significant while it is not theoretically relevant.

How many participants are required?

A size of 30 to 40 participants per experimental condition indicates sufficient statistical power in experimental advertising research, especially if the population is homogeneous (Geuens & De Pelsmacker, 2017). However, the authors emphasize that larger subsamples per experimental condition are needed if a moderator is used in the model. For instance, a total sample of 80 participants is sufficient when a researcher designs an experiment with two experimental conditions. In this case, 40 participants are randomly exposed to advertisement A, and 40 participants are exposed to advertisement B. Now, suppose that the researcher wants to test the moderating of the gender between the type of stimulus and consumers' responses. In that case, 80 participants will not be enough. In terms of analyses, there are now four conditions instead of two, and the interaction effect would be based on subsamples of 20 participants on average. The research would need 40x4 participants, thus a total of 160 participants.