pro-tips-qualitative-sample-sizes

POV: Qualitative Sample Sizes

Purpose

The purpose of this content is to provide a point of view on the ideal sample size for a qualitative market research study, because that’s one of the most common questions even the most seasoned researcher asks us.

Industry norms

Adequate sample sizes have been debated extensively in the market research industry for many years, in part because quantitative sample sizes are statistically easier to measure. Often, quantitative research frameworks get applied to qualitative research because quant norms are concrete (e.g +/- variance), whereas qualitative sample sizes are difficult to prove, mathematically.

A number of studies have been published over the years on the topic of adequate sample sizes for qualitative research. Here are a few:

    • Creswell, Glaser, Morse – all pretty much agree that sample sizes of 30-50 respondents reaches a point of saturation, where adding more respondents does not significantly alter the findings
    • Springer – Springer puts forth the argument that anywhere from 5-50 is adequate, but that 25-30 is considered to be the right number
    • InterQ – recommends 20-30, or even as few as 10

Fabric POV

Qualitative research is more art than science. Bill Bowerman (Nike co-founder) tested his shoe designs on a relatively small sample size, yet was able to scale the innovation to millions of people, in large part because it was the right sample of runners.

Typically, clients turn to qualitative methods to understand meaning – the How and the Why. 

So while we believe there are minimum sample sizes that should be employed in qualitative research, there are some critical variables in the equation we consider:

    • where you are in the process; in our opinion, earlier stage projects (e.g. exploratory product innovation) can employ smaller sample sizes, because more iteration and development will be conducted as the project moves along
    • the business impact: we feel that the higher the business impact of the research, the greater the sampling (qualitative and quantitative combined) should be
    • geography: it can be as easy as domestic and international, but one critical question we consider is how broad the target audience is, geographically
    • research design: how the study is designed can have a huge impact on results (e.g. which questions to ask in what order)
    • research vendor: when you buy a qualitative research resource, you are buying a filter (like a water or air filter). One important variable is how equipped that resource is, down to the people, to field the study and interpret the results
    • quality of recruit: back to the Bowerman example, it’s very important that the people in the study be the right people; beyond technically whether they qualify, we ask ourselves whether the people we interview “feel” like the right consumer. And we will over-index the input of those whom we deem more influential (for example, a starter on an AAU basketball team in Brooklyn may be more insightful than a practice player on a D3 team in the suburbs)
    • analysis: who is interpreting the data is highly important. 
    • methodology: there are a number of qualitative techniques to choose from, including IDIs (individual interviews), focus groups, in-homes, friendship pairs, small group interviews, intercepts, observational research, ethnographies, and digital (online, mobile). In Fabric’s case, the methodology is unique in that it’s effectively “1-on-none” – meaning it’s asynchronous, and there is no moderator present. A lot has been written about the effects of group-think in focus groups, where an ‘alpha’ respondent will influence others. Similarly, the confessional style of the Fabric methodology enables what researchers have called the “online disinhibition effect” where respondents are more open to express themselves because there is no fear of disagreement or conflict with a moderator. For consumers, communicating asynchronously via video is a very comfortable and common medium

Fabric POV

Our short answer when clients ask us about the right sample size is to go with 30-50 respondents, but the context as outlined in the “Fabric POV” section above guides us beyond.

Follow up questions are welcome: please email tom@fabric.is.

pro-tips-creative-cases-mindswarms

Creative and inspiring use cases of Fabric

The following is a short list of ways in which we and/or our clients have employed the platform creatively to extract new and unusual insights.

Baby wash basin

For a leading brand of baby strollers that was looking into extending their product footprint into the baby washbasin space, we had parents of kids <1 years old wash their babies on camera (their privates were covered with a washcloth) and talk about the experience. In this instance, the partner held the phone to capture the experience. There is literally no other way to do this: imagine sending a videographer and interviewer into their home and asking them to wash their baby while you film it? Right; we don’t think so either.

Prepare and eat food

For one of the country’s leading packaged food innovation companies, we shipped a potential new product to consumers’ homes, then had them record their out of box experience, the preparation, and finally tasting of the product. Interestingly, while their initial impressions bordered on repulsion (“it looks like rabbit poo!”), the taste of the product was incredibly well received, providing the client with insight on how to tweak the product extrusion.

Write a love letter to the brand and read it aloud

For one of the world’s biggest furniture brands and retailers, we had loyalists write a love letter to the brand, and read it aloud on camera. It was for an exercise to get at what the core of the brand stood for, so having consumers – in their own words – craft and read a love letter on camera, it helped surface some of the deeper emotional connections they had with the brand.

Go to a drive thru

For a major QSR brand that was looking at re-designing its drive-thru menu board, we had consumers of the brand report on the before, during and after of the drive-thru experience at their stores. Recording video from inside their cars in the moment, the client was able to get a first-hand look at how and where the messaging was connecting and missing.

Shop for spray paint

For a global ad agency pitching a new potential client, we had consumes shop for the client’s product at a chain of major big box stores. Because like many categories, the product and landscape was crowded and competitive, so having consumers report on their thoughts on the buying process as well as packaging and displays helped drive a deeper understanding of how hard it would be for the brand to stand out.

Show us your favorite sports bras

For a project to help re-launch and re-position a line of sports bras for a leading brand of sports footwear and apparel, we had women show us their collection of sports bras and talk about them. Any time people hold something and talk about it, it not only helps validate them as the right consumers, but it also makes them more animated and articulate in their responses.

Step outside of a party

For a project with truth (anti smoking), we had teens step outside from a party they were attending and record their responses to key questions that would arise for them in a party situation. Although the lighting quality was variable (a number of the responses were quite dark), the insights were powerful because they were so close to the moment (specifically a situation loaded with peer pressure).

SONOS diary

To understand how consumers’ relationship with music changed in the home using SONOS, we shipped speakers to people all over the world. And had them record their thoughts over a period of one month.

Laundry journey

To understand potential product extension zones of opportunity for a major European appliance brand, we had consumers deconstruct their before, during and after laundry journeys.

Political advertising

Early on in the Trump campaign when the first TV commercial launched, we were curious to understand if/how the messaging was resonating and connecting. While we are politically neutral, we predicted early on that he had tapped into a strong emotional undercurrent in the country, and our founder was quoted as such in Mother Jones magazine.

Introduce us to your cat

For a pet food study for one of the country’s biggest and best known brands, we had respondents introduce us to their cats in the first response. The technique helped not only validate that owners did indeed have cats, it made respondents more emotionally engaged, and helped add color/texture to their insights.

Show us the biggest villain in your home

Working with one of the world’s largest furniture brands and retailer, we had consumers show us what they deemed to be “the biggest villain in the home” with the goal being to understand how that connected to the American Dream.

Home use test

We have shipped food products, SONOS speakers, shoe prototypes and make-up to people’s homes and had them record everything from their “out of box” experience to preparation/set up, use and even removal of make-up.