Testing Creative Stimulus Materials

Creating Your Study

7 min read

Whether it’s a new product design, the development of a new ad campaign, or the iteration of new UX or features, we often get asked about how to test different types of stimulus materials

How to attach stimulus

You can add a link to any individual question or multiple questions. Just highlight the word you want to link, then (as with Google Drive) paste in the link.

Users will click on that link and be directed to the destination. We generally recommend using Google Drive because most people are familiar with it, but links can take users to:

- Websites
- Videos (e.g., YouTube, Vimeo, etc.)
- Cloud storage locations (Google Drive, Dropbox, Box, etc.)

What kind of stimulus can be attached

Researchers use Fabric to test a broad range of assets. Some examples:

- PDFs
- Videos
- Sites/apps

How long should a piece of stimulus be?

The kinds of stimulus that have been tested include:

Product design:

- Sketches of product ideas from a designer’s notebook
- Descriptions of a new product
- Proposed layouts
- UX/UI
- Packaging
- Beta products/apps and finished products

Advertising:

- Platforms
- Campaigns
- Tag lines
- Campaign elements
- Manifestos
- Positioning statements
- Value propositions

Since each respondent is served up 10 questions, there are a number of ways to leverage the Fabric platform for testing stimulus. As a general rule of thumb, if you have four different pieces of stimulus to test, here’s how the arc of the study might look:

- Q1: baseline perceptions of X
- Q2: first reactions to Stim 1?
- Q3: what resonates with Stim 1?
- Q4-Q9: repeat Q2 + Q3 for the other Stim
- Q10: compare and contrast or pick fave*

*Note: for Q10 in the above example, it’s a good idea to include a rollup of all the stimulus to remind them of everything they have already seen. Otherwise they might have trouble recalling the first few concepts.

Testing statements or paragraphs

When testing product descriptions or positioning statements which can run longer in text form, do your best to keep the concepts highly differentiated. Present 3–5 concepts max. If there is significant overlap in the concepts and/or the statements are long, consumers will have trouble distinguishing one from the others. In that case, we recommend that your wrapup include a rollup PDF of all of the statements/concepts. The rollup will refresh the respondent’s memory after they’ve seen each individually.

Avoiding order bias

To avoid your entire sample seeing the stimulus in the same order, therefore biasing their reaction depending on the sequence in which they see the stimulus, break your studies down into smaller sample sizes and switch up the order.

For example, with a sample size of 15 people (n=15) and three pieces of stimulus, a suggested approach would be to structure it like this:

- Cohort 1 (n=5): Stimulus A,B,C
- Cohort 2 (n=5): Stimulus B,C,A
- Cohort 3 (n=5): Stimulus C,A,B

Confidentiality

As with everything in an online environment, confidentiality can be compromised. A few notes on how to protect your ideas:

- Our user Terms and Conditions have built-in confidentiality; but as you know, a lot of folks don’t read them all.
- Serving your concepts up without a logo or brand can help make it brand-blind, eliminating not only security concerns but also may give you a purer read on the relevance and resonance.
- Serving up the same concept with multiple logos on it can help head-fake consumers, and also give you a read on the influence of the brand associated with it.
- Lastly, if the risk of the idea somehow leaking is high, we recommend you NOT use Fabric to test your concepts. You have to do the risk/reward calculus. If a 17-year-old teen can hack into the Pentagon, taking a screen grab of a concept is not beyond the realm of what consumers may do.