Analyzing Video Responses: Qualitative Data Analysis

Your Study Results

7 min read

By far the hardest part of the qualitative research study process is analyzing the incoming data because it’s unstructured compared to quantitative data. Researchers appreciate our platform’s unique tools, which speed up the process and give researchers a head start. Leverage our six key features and learn how our proprietary Fabric AI can help.

Six Fabric features for qualitative data analysis

1. Video responses are organized by question

- You can hyper-target your analysis by reviewing how all respondents answered any one question.
- Unlike focus groups or moderated one-on-ones, the same questions get asked of everyone in the same order, providing consistency.
- Sometimes a specific question is THE question you’re trying to understand, and the other questions lead up to it. In that case, you can skip right to a specific column of responses, to get straight to the heart of the issue.

2. Enhanced transcript experience

- If you upgraded to human + machine transcripts (from machine only), then those transcripts will populate within 24 hours of each respondent’s completion.
- Google Speech transcripts are available to all researchers almost immediately. Note that while machines are good and getting better all the time, they aren’t perfect. We make the transcript field editable so that you can correct any Google Speech errors.
- Transcripts can be downloaded to Google Sheets or Excel.

3. Comments can be added to individual study responses

- As on any social media platform, there is a comment field where collaborators can share feedback, ideas and insights. The field can also be used for note-taking.

4. Respondents can be rated

- Our rating system makes it easy to remember who the best respondents are. It’s also a way to keep track of which respondents the researcher has reviewed.

5. Respondent names can be made anonymous

- In an era of increased sensitivity to PII, our default view is to make users’ names or email addresses anonymous. However, if you bring your own recruits, the default is to show respondents’ email addresses because we presume they gave you access to email them. That field is editable if you would like to make respondent names anonymous yourself.

6. Unlimited tags can be created

- There is no limit on the number of tags that can be created in a study.
- After tags are created, researchers can sort by tags individually or in combination.

Working with our proprietary Fabric AI

One of the top frustrations of qualitative data analysis is the time it takes to digest the information, make sense of it, and bubble up key insights.

So we developed our own proprietary sentiment and emotion-based AI that provides three primary data analytic sources that free up researchers to spend more time developing insights that link to their brand, message, product or design.

Fabric AI includes:

The paragraph summary
The paragraph summary encapsulates the responses to any individual question in a study. To be clear: this does not summarize metadata across the entire study, only question by question.

The paragraph is written in plain English and it identifies:

- Whether responses were positive, neutral or negative
- Up to three sources of the sentiment
- Key themes
- Top emotions

Sentiment, sources of the sentiment, themes and top emotions are all hyperlinked. The videos and verbatims surfaced by our AI respond to the topic being explored in the hyperlinks.

Total mentions & emotional mentions
Fabric AI shows a count of total mentions, as well as mentions with emotional intent behind them (either positive or negative).

- Total mentions is a count of literally how many times a word was mentioned.
- Emotional mentions is a count of how many times a word was mentioned with emotional intent behind it.
- Strength bar indicator shows whether the sentiment was strong or weak.
- Mentions are also correlated to one of 8 primary emotions Fabric AI tracks.

Additional features
Researchers also benefit from our AI’s ability to surface videos and verbatims with the highest degree of emotion, and there is a search field to quickly direct researchers to specific terms of interest.

December 5, 2022December 27, 2023

Creating a Video Highlight Reel

Your Study Results

7 min read

Why use video?

Video highlight reels can be incredibly powerful tools in a debrief session. Standing in front of a group of people, telling them what consumers said—it’s not uncommon for attendees to get defensive or dismiss insights they don’t agree with.

Here’s where your highlight reel comes in. Press “play” and let consumers say it in their own words. Not only does the room go quiet for a few minutes, it puts any naysayers on their heels.

How to create a powerful video highlight reel

In our years of market research experience, we’ve learned how to create videos that pack a punch.

Here’s our method:

1. Keep it short
2. Tell a singular story
3. Bring the story to life
4. Make a paper edit
5. Edit video & post-production

1. Keep it short
Videos should be kept under 2–3 minutes. Anything longer won’t hold the viewers’ attention in a culture dominated by TikTok-length videos. Long videos also become a mini documentary film requiring too much production effort.

2. Tell a singular story
How do you go from a dataset of dozens or hundreds of individual video responses to a 2–3 minute video?

The most important part of the process is telling a singular story. This means distilling down the ideas and statements found in your survey respondents’ videos. Sum up the singular story of your video highlight reel in one definitive sentence.

Our rule of thumb: if you can’t articulate the story of the video in one sentence, it’s probably not a story. Viewers are accustomed to video highlights that tell a single, coherent story, rather than a disconnected set of facts.

Asking the video to do the work of a slideshow presentation generally fails, too. We know for a fact that trying to do PowerPoint in video format does not work.

The video highlight reel and the presentation are two different narrative forms. Your reel should have its own internal narrative that tells a singular story.

3. Bring the story to life
Now that you have a singular story sentence, bring the story to life.

As a starting point, try structuring the story using three chapters: beginning, middle and end.

As you go through the process, keep the end in mind; what is the key message you want viewers to take away when they have finished watching the story?

Within those three chapters, consider using an effective storytelling device called a pivot point. Is there a moment where you are leading viewers down a path, and the unexpected gets revealed? For example, is there a surprising attitudinal, perceptual or behavioral insight that challenges convention? A new way to see the world? We think about it as the equivalent of the needle scratching across the record on a vinyl LP.

In classic ad agency pitch theater, the arc of a story might look something like:

- Things were great, everybody loved us
- Then something unexpected or bad happened and we lost them
- But here’s what they love about us or we are overlooking that could bring them back

4. Make a paper edit
A paper edit is a written outline of your video. It provides structure, tightens your focus, and helps you keep your video the right length. Think of it as a kind of script, storyboard or shot list, but with time codes. The time codes show exactly which interview clips you plan to use.

To make your life easier, use our paper edit tracking sheet where you can enter the key quotes you want to include, and it will automatically calculate the run time length. Note that it’s based on averages, so there may be some variance between the estimated and actual video length.

5. Edit video & post-production
With a paper edit prepared, the video editor can now download the relevant clips. Transitions, title cards, music and even B roll can be added.

Examples

Some of the more compelling video highlight reel stories we have seen or been part of:

- For a study about kids attending a football combine, for a major footwear brand working on a new shoe: These athletes would give their eye teeth for one tenth of a second improvement.
- In a study for a major national fast casual restaurant: People’s connection to bread goes way beyond taste and texture to near-spiritual associations with ethnicity, ancestral origins and even religion.
- For a study on the redesign of an ultrasound machine: Despite the anxiety physicians experience as part of their jobs, they have personal superhero moments using ultrasound that provide a massive surge of confidence.

Conclusion

Including individual clips in your debriefs can be powerful. But creating a 2–3 minute video highlight reel with a clear underlying story will help put your final presentation over the top. Our five steps will help get you there.

December 5, 2022December 27, 2023

Determining the Right Study Design

Creating Your Study

7 min read

Because qualitative market research is more of an art than a science, it’s important to think through study design—or as we call it, the research strategy.

Study preparation arc

Broadly, the study preparation logic should follow this arc: 1) Research objective; 2) Research strategy; and 3) Research tactics.

1. Research objective
Determine the research objective. Ask, what is the singular, overall objective of the study? Usually, your answer will take the form of a single sentence.

2. Research strategy
Articulate your research strategy. Broadly speaking, given the objective, what is the strategy for the study’s architecture? For example, if the study objective is to understand how a new product concept resonates, the research strategy could be “Compare the ideal to the actual.”

3. Research tactics
What are the research tactics? Given the objective and strategy, determine which specific tactics should be employed.

For example: we wanted to understand how people felt about the fact that when they are in public, chances are very good they were being recorded on video. We asked several questions involving how they felt about the proliferation of video cameras in public, to tap into the left brain where the speech center is located. Then we had them then answer one question using only gestures and body language—no words—to tap into the more emotional right brain.

Study design example

By way of example, imagine this scenario:

- The research objective is to understand core consumer perceptions of the client’s brand.
- For the research strategy, we recommend that the study deconstruct all of the different elements that make up the core brand perceptions.
- Hence, the research tactics would involve breaking down the top most important associations people might have with the brand, and focusing each question on one aspect (what they sell, who they think buys it, who they associate with the brand, what imagery they connect with it, who works at the company, what does the company’s brand say about the people who wear or use it, if there is one word that describes the brand, what would that one word be and why).

December 5, 2022December 27, 2023

Setting the Research Objective

Creating Your Study

7 min read

One of the most important aspects of any market research project is setting the right research objective. Researchers often mistake the questions they want answered (which are of course important) for the research objective itself.

Establishing a research objective can be done in various ways. We recommend articulating the research objective in one single sentence. This will provide a clear focus for your team.

If you list a number of questions, neither you nor the team will know exactly what it is the study is aiming to achieve.

Below are some examples of clearly articulated research objectives:

- Understand how the current messaging campaign resonates in different countries
- Understand if/how existing users understand our key point of differentiation
- Bring to life the muse target audience for the brand
- Understand which concept resonates most with consumers and why
- Identify drivers of churn

December 5, 2022December 27, 2023

Determining the Best Sample Size

Creating Your Study

7 min read

The #1 question we get asked by researchers is this: How many people should I include in my study? We’ll answer that in two ways:

- What the academics say
- What our experience has taught us

The academic POV

What constitutes an adequate sample size has been debated extensively in the market research industry for many years, in part because quantitative sample sizes are statistically easier to measure, including variance.

Often, quantitative research mindsets get applied to qualitative research because quant norms are concrete, whereas qualitative sample sizes are difficult to prove mathematically.

Studies have been published over the years on the topic of adequate sample sizes for qualitative research. Here are a few:

- Creswell, Glaser, Morse (the recommendation: 30–50 participants)
- Springer (Springer puts forth the argument that anywhere from 5–50 is adequate, but that 25–30 is considered to be the right number)
- InterQ (recommends 20–30)

Broadly, the academic research suggests that a sample size in the 30–50 participants range achieves what experts call “the point of saturation” where adding another participant doesn’t add materially to the insights generated. We agree with that analysis.

What our experience has taught us

Qualitative research is more art than science.

Typically, clients turn to qualitative methods to understand deeper meaning: beliefs, attitudes, perceptions, feelings, emotion and more—deep intangibles that help add up to the “why.” So while we believe that minimum sample sizes should be employed in qualitative research, getting to the “why” requires more nuance. For this we recommend taking 10 critical variables into consideration.

The 10 critical variables

We have found that there are 10 critical variables to consider for determining the right sample size for a qualitative research study.

1. Where you are in the process
Earlier stage projects (e.g. exploratory or generative studies) can generally employ smaller sample sizes, because more iteration and development will be conducted as the project progresses.

2. Business impact
The higher the business impact of the research, the greater the sample size should be. For high impact research, we would likely recommend a hybrid quant/qual approach so you’re not relying solely on qualitative research.

3. Geography
One essential variable we consider is the geographic diversity of the target audience. It can be as easy as needing domestic and international respondents, or it may be more complex. In the US, most national brands have very broad distribution, so making sure to include the coasts as well as the interior is not only great practice, it’ll signal to clients that you don’t favor one type of demographic audience over another.

4. Research design
How the study is designed can have a huge impact on results. Which questions to ask in what order changes how respondents answer, affecting a study’s insights and conclusions.

5. Research platform
When you leverage a qualitative market research platform, you are buying a tech stack. One important variable is how well equipped that resource is technologically to identify the right recruits, field the study in a relevant way, and interpret the results. In the case of Fabric, our proprietary AI is an industry first, employing sentiment and emotion analysis to help researchers rapidly make sense of video responses.

6. Quality of recruits
It’s very important that the people in the study be the right people, regardless of how many respondents are included in the study size. Beyond the issue of whether they technically qualify, the people included in the study need to “feel” like the right consumer.

Using Fabric’s video-centric platform is particularly useful in this regard. Seeing the target in motion, on screen, telegraphs a great deal of information, versus seeing numbers in a graph or having a faceless/voiceless respondent using text replies.

7. Analysis
Who is interpreting the data your study will generate? All insight is qualitative, so considering how equipped you or your research partner is in the process has a tremendous influence on both sample size and study design.

8. Methodology
A variety of methodologies are available to the researcher, and each will suggest its own approach to sample size. Qualitative techniques to choose from include IDIs (in-depth interviews), focus groups, in-homes, friendship pairs, small group interviews, intercepts, observational research, ethnographies and/or digital (online, mobile).

In Fabric’s case, the methodology is unique in that it’s effectively “1-on-none”—meaning it’s asynchronous, and there is no moderator present. Much has been written about the effects of group-think within focus groups, where an ‘alpha’ respondent will influence others; Fabric has none of that. Fabric also removes the moderator from the study; therefore there is no moderator bias.

Instead, the methodology and technology employed by Fabric free up respondents to behave in a more open manner. Fabric’s confessional style enables what researchers have called the “online disinhibition effect” where respondents are more open to express themselves because there is no fear of disagreement or conflict with a moderator or fellow panelists.

9. Company culture
Some organizations are more comfortable with small sample sizes, whereas some look for larger samples because distribution is wide and/or global. For example, we have worked with Nike’s Innovation Kitchen on early exploratory studies using a small sample size. When we work with Xbox, having 10 markets each with 10 respondents, the sample size can drift to 100+ easily.

10. Segments
Often we’ll see that a client has a number of segments to understand. In that case, determining sample size depends on whether or not you’re looking for a rollup of all segments. If you’re going for a rollup, the 30–50 number is fine.

However, if you’re seeking to understand similarities and differences amongst segments, we would recommend 15 respondents per segment. This enables your study to take advantage of our AI, which kicks in at a minimum of 15 respondents.

Conclusion:

What is the right sample size for your qualitative study?
If you’re looking for a short answer on the recommended sample size, it’s 30–50. But keep an open mind to key variables that may influence higher or lower numbers.

December 5, 2022December 27, 2023

Writing Recruitment Screeners

Creating Your Study

7 min read

There are two ways to bring recruits into your Fabric study: provide us with the right information to write the screener for you, or bring your own screener.

Since respondents aren’t paid to answer screeners, try to limit the number of screening questions to ten data points (including age, gender and geography).

1. Let us build the screener for you

When building your DIY study, select “Let Us Recruit”, and then on the following page define the recruits you’re looking for responses from. This may include:

- Demographic information like age, gender, geographic location, household income and education
- Behavioral criteria like product usage, recency of purchase, competitive products owned, brand affinity, amount spent, and frequency
- Psychographic criteria such as personality questions, agreement to attitudinal statements, general preferences, or other intangibles
- Our team recommends keeping recruit criteria simple to allow for the quickest possible responses and the highest likelihood of approval from the recruiting team.

2. Provide us with your screener

The content of your screener can be cut and pasted into the “Define Your Recruit” field within the “Let Us Recruit” option, or you can paste a link to a document containing your screener for our team to program.

3. Fabric will accept/reject your criteria within 24 hours at the most

Once you have submitted your recruitment request through the Fabric platform, the team will review it for feasibility within 24 hours. Most requests are approved within 1–2 hours. If there are any questions or potential sticking points, we will reach out via email.

4. Once the screener is approved, finish building your study and launch

Once your screener request is approved, you will receive a notification. In your dashboard, the study status will now read “Approved – Awaiting Payment”. From there, the next step is to do a final review on your study questions.

Then proceed with launching your study. Your recruits will populate the study dashboard, with the first respondents usually coming in within 24 hours of launch.

Author: Tom Bassett

Analyzing Video Responses: Qualitative Data Analysis

Analyzing Video Responses: Qualitative Data Analysis

Six Fabric features for qualitative data analysis

Working with our proprietary Fabric AI

Fabric AI includes:

Creating a Video Highlight Reel

Creating a Video Highlight Reel

Why use video?

How to create a powerful video highlight reel

Examples

Conclusion

Determining the Right Study Design