pro-tips-qualitative-sample-sizes

POV: Qualitative Sample Sizes

Purpose

The purpose of this content is to provide a point of view on the ideal sample size for a qualitative market research study, because that’s one of the most common questions even the most seasoned researcher asks us.

Industry norms

Adequate sample sizes have been debated extensively in the market research industry for many years, in part because quantitative sample sizes are statistically easier to measure. Often, quantitative research frameworks get applied to qualitative research because quant norms are concrete (e.g +/- variance), whereas qualitative sample sizes are difficult to prove, mathematically.

A number of studies have been published over the years on the topic of adequate sample sizes for qualitative research. Here are a few:

    • Creswell, Glaser, Morse – all pretty much agree that sample sizes of 30-50 respondents reaches a point of saturation, where adding more respondents does not significantly alter the findings
    • Springer – Springer puts forth the argument that anywhere from 5-50 is adequate, but that 25-30 is considered to be the right number
    • InterQ – recommends 20-30, or even as few as 10

Fabric POV

Qualitative research is more art than science. Bill Bowerman (Nike co-founder) tested his shoe designs on a relatively small sample size, yet was able to scale the innovation to millions of people, in large part because it was the right sample of runners.

Typically, clients turn to qualitative methods to understand meaning – the How and the Why. 

So while we believe there are minimum sample sizes that should be employed in qualitative research, there are some critical variables in the equation we consider:

    • where you are in the process; in our opinion, earlier stage projects (e.g. exploratory product innovation) can employ smaller sample sizes, because more iteration and development will be conducted as the project moves along
    • the business impact: we feel that the higher the business impact of the research, the greater the sampling (qualitative and quantitative combined) should be
    • geography: it can be as easy as domestic and international, but one critical question we consider is how broad the target audience is, geographically
    • research design: how the study is designed can have a huge impact on results (e.g. which questions to ask in what order)
    • research vendor: when you buy a qualitative research resource, you are buying a filter (like a water or air filter). One important variable is how equipped that resource is, down to the people, to field the study and interpret the results
    • quality of recruit: back to the Bowerman example, it’s very important that the people in the study be the right people; beyond technically whether they qualify, we ask ourselves whether the people we interview “feel” like the right consumer. And we will over-index the input of those whom we deem more influential (for example, a starter on an AAU basketball team in Brooklyn may be more insightful than a practice player on a D3 team in the suburbs)
    • analysis: who is interpreting the data is highly important. 
    • methodology: there are a number of qualitative techniques to choose from, including IDIs (individual interviews), focus groups, in-homes, friendship pairs, small group interviews, intercepts, observational research, ethnographies, and digital (online, mobile). In Fabric’s case, the methodology is unique in that it’s effectively “1-on-none” – meaning it’s asynchronous, and there is no moderator present. A lot has been written about the effects of group-think in focus groups, where an ‘alpha’ respondent will influence others. Similarly, the confessional style of the Fabric methodology enables what researchers have called the “online disinhibition effect” where respondents are more open to express themselves because there is no fear of disagreement or conflict with a moderator. For consumers, communicating asynchronously via video is a very comfortable and common medium

Fabric POV

Our short answer when clients ask us about the right sample size is to go with 30-50 respondents, but the context as outlined in the “Fabric POV” section above guides us beyond.

Follow up questions are welcome: please email tom@fabric.is.