Focus groups are frequently conducted in a roundtable format. Photo: United States Department of Veterans Affairs.
If you’re planning focus group research, you’ll probably want to know how many groups you should run. Sometimes the question will be answered simply and quickly by your available budget; if you can’t afford to run more than two groups, then that is all you’re going to plan for. But what if you can afford more? Is there a rational basis for settling on any particular number of groups?
Most of the suggestions that I have seen are variants on a single theme — sample until you stop hearing anything new. Lunt and Livingstone [1], for example, say “…one should continue to run new groups until the last group has nothing new to add, but merely repeats previous contributions.” That sounds simple enough but what does it really mean? If you run two groups which proffer essentially the same three opinions, does that mean you should stop? Or should you assume that a third group might come up with some, as yet unstated opinions, and continue running more groups? Would it make any difference if each group gave exactly the same 20 different opinions, as compared with the possibility that each group offered only two opinions?
One way of looking at the decision problem is to try to create a model of the process that is assumed to underlie the formation of opinions and their subsequent revalation in the focus group. The closest that I have found to an explicit statement of the model that underlies the previously described rule of “sampling to saturation” is in another remark by Lunt and Livingstone, namely, “A useful rule of thumb holds that for any given category of people discussing a particular topic there are only so many stories to be told.” That might not sound like a description of a model, but here is my attempt at enlarging upon their statement.
Regarding any particular topic, there are a finite number of stories (or beliefs, or opinions) floating around in the ether. Each person acts as an “story trap”; stories that approach too closely to the trap are caught, and when the person is subsequently interviewed, in a focus group for example, the contents of the trap are revealed.
The reason that I have phrased the model in these unusual terms is because it then becomes obvious how the activity of running a focus group to discover opinions is similar to the activity of a biologist who is trying to discover how many species of animal there are in a particular environment. The biologist sets special traps which capture, mark and release the animals that are ensnared and at the end of the day the biologist has information on how many species were captured only once, and how many of them were captured multiple times. Statisticians have developed various methods [for example, 3] for estimating, from the capture data, the number of undiscovered or untrapped species.
Similar approaches could be made, first, to determining how many opinions one has not managed to tap by the focus groups one has run so far, and second, estimating how many more focus groups one should run to increase the probability of capturing those opinions with some arbitrary likelihood. I know of only one paper [4] that touches on first problem and I know of no research that has attempted to tackle the second problem. Given the very large sums of money that are devoted to market research, both problems seem to me to be worthy of more attention than they have so far been given.
[1] Lunt, P., & Livingstone, S. (1996). Rethinking the focus group in media and communications research. Journal of Communication, 46(2), 79–98.
[2] Karanth, K. U. (1995). Estimating tiger Panthera tigris populations from camera-trap data using capture\u2014recapture models. Biological Conservation, 71(3), 333–338.
[3] Efron, B., & Thisted, R. (1976). Estimating the Number of Unseen Species: How Many Words Did Shakespeare Know? Biometrika, 63(3), 435–447
[4] Griffin, A., & Hauser, J.R. (1993). The voice of the customer. Marketing Science, 12(1), 1–27.
Contributors: Mark R. Diamond