“Master the topic, the message, and the delivery.” — Steve Jobs, Co-Founder, Apple
Do you want to know what wording to use in your marketing campaign, what hero image to add to your landing page, or what features and benefits to emphasize in your social media ads? Maybe you have three wildly different packaging ideas and you need to see which one generates the most consumer interest? Doing some simple experiments can answer these questions and help ensure that you’re connecting with and converting customers at every touch point.
In a recent survey, Moonshot Collaborative found that 36% of companies listed “developing effective messaging and marketing” as one of their three most important business challenges. In this blog, we focus on using experimental research for optimizing things like advertising, branding, and packaging. Experiments (as opposed to simple survey questions) are also very useful for other areas, notably product development.
Ready? Let’s start experimenting!
A/B Testing
A/B testing is the practice of changing one element of your message for a subset of your audience and evaluating the difference in results. If your research question boils down to “this option or that option,” then A/B testing might be a good fit. It is one of the most common forms of experiments and is widely used in web development and email optimization, where clicks and conversions can be easily tracked to measure differences.
For instance, companies might send an email with Subject Line A to 10% of their list and Subject Line B to another 10% of their list. Whichever one performs best (e.g., leads to more clicks and sales) will be used with the remaining 80% of the people on their list.
But what if you don’t have a large email list or lots of website visitors? Fortunately, you can also run experiments with consumers — and Moonshot Collaborative can help.
Let’s take a look at a basic example, which is similar to a client study we just completed. In our hypothetical scenario, a new alternative dairy company is trying to decide which term to use in their marketing materials – chickpeas or garbanzo beans. A simple way of answering this question would be to ask consumers how interested they would be in buying an alternative milk made from X, using the term “chickpeas” with half of respondents and “garbanzo beans” with the other half (randomly assigned among all respondents in your sample size).
A company might take this a step further and A/B test the two different terms using mock-ups of their product packaging. The closer to a “real world” experience you can get with your research, the better. But it’s far too expensive to do multiple production runs for different types of packaging just for testing purposes. Instead, you can get solid answers to key messaging and branding questions with just a few hours from a graphic designer and a single survey of about 500 people.
A few tips on A/B testing
- Have more than two variations? You can do A/B/C or A/B/C/D testing if you have a large enough sample size (we typically recommend at least 100 respondents per variation), but be sure to stick to testing just one element at a time (e.g., different packaging colors or descriptive terms, but not both).
- Randomly assigning the variations to respondents is important, and best practices suggest verifying that the different groups are similar when it comes to key metrics such as age, gender, and household income profiles.
- Give your A/B test enough time for results to come in before acting on the data. Those who respond to your test first may not be representative of your entire target audience and could skew the results.
- Rather than “one and done,” use A/B testing iteratively to continue refining your messaging and your marketing materials. You’ll learn much more over a series of A/B tests than with a single test.
Experiments that Answer More Complex Questions
A/B testing is the most basic form of consumer research experiment, with a focus on testing one element at a time with a single group of people (which is then split into smaller groups to view different versions). But what if you want to know how exposure to a message impacts people’s attitudes and behavior (e.g., greater purchase intent)? Or what if you want to test multiple elements at a time? Fortunately, there are experimental methods to get those kinds of answers too, though they’re not always simple to carry out. Here are a few examples:
- Pretest-Posttest: Asking participants the same set of questions before and after being exposed to two or more different messages (or “treatments”) and seeing which message produces more positive results.
- Controlled Trial: Depending on circumstances, you may want a “control group” of respondents who are not exposed to your message for comparison purposes to see if your message is effective.
- Conjoint Analysis: Part experimental design and part analysis technique, conjoint lets you evaluate multiple elements or variables at a time and understand the importance that consumers give to each element.
Lastly, a few words on cost. A/B testing is a simple and straightforward approach to consumer research and can be done relatively inexpensively. A basic A/B test of a couple of concepts with 300-400 targeted respondents per concept might cost as little as $2,500. More complex experiments take more time and expertise to implement, but can start at around $5,000, with full conjoint analyses starting at around $10,000.
~~~~~~~~~~~
Is your company ready to craft messaging that captivates your target audience, resonates with them, and seamlessly turns them into paying customers? At Moonshot Collaborative, we have the tools and know-how to help ensure that your consumer research experiment is cost-effective and leads to actionable results.
Get in touch today to discuss how we can help
