You’ve probably heard of quantitative and qualitative research. Both are tools to better understand customers, consumers, and people in general. But each takes a somewhat different approach and should be used for different research goals. Larger research projects usually employ a mix of both quantitative and qualitative methods.
Examples of data sources for qualitative research include in-depth interviews, focus groups, some observational studies, and even product reviews and customer support tickets. The purpose of analyzing this kind of data is usually to gain a deeper understanding of how or why people think or behave a certain way or react to a product, advertisement, etc., whereas quantitative data is best for telling you what people are thinking.
Sample Sizes & Challenges
As noted in our blog on sample sizes, Moonshot Collaborative recommends that studies involving one-on-one interviews include at least 25-30 people, but typically not more than 50-60. For focus groups, we think that 3-4 groups is usually sufficient, with each group including 5-8 participants. These numbers usually produce sufficient data for either manual coding or automated text analysis (see below), without being overwhelming.
Some forms of qualitative research can be relatively straightforward for businesses to conduct themselves, but making sense of the results is deceptively hard. A set of 25-30 consumer interviews or 3-4 focus groups might yield hundreds of transcribed pages of people’s comments — what we researchers call “unstructured data.” Effectively interpreting that data and distilling it down to something meaningful is especially challenging.
It’s so challenging, in fact, that Moonshot Collaborative generally does not recommend that companies analyze their own qualitative data, unless they have that expertise in-house. Otherwise, it’s easy to allow bias to influence the analysis and it can be hard to resist the temptation to quantify the results (e.g., “55% of our 18 focus group participants said they would buy our product”). But we thought it might be useful for clients to understand the four steps we typically take when analyzing qualitative data.
Step 1. Clean and Familiarize
For any kind of qualitative data analysis, the first step we take is to clean and familiarize ourselves with the dataset. This includes transcribing the data (if needed), correcting spelling errors and other inconsistencies that might make interpretation more challenging, and taking note of the initial themes that arise from a quick review of the data. The result is a dataset that we can easily work with and code, and some preliminary ideas of what people think.
Step 2. Code All Comments
When manually analyzing qualitative data, our next step is to code the comments using a systematic process. At Moonshot Collaborative, we tend to prefer an “inductive” process that lets the data determine the codes and themes (rather than defining themes ahead of time with a “deductive” approach, though it depends on the project). We do this by taking each comment and parsing it into different codes or “mini themes.”
For example, let’s say we’ve conducted a study to understand consumers’ perceptions of plant-based meat options currently on the market, and one person made the following remark. Here is how we might interpret and code the comment:
In the above example, we’ve color-coded the various phrases that a researcher might identify in the comment and the corresponding codes on the right. Each code should describe a specific idea or feeling expressed in that part of the comment. The actual codes will depend, in part, on the specific research question we’re trying to answer for a given project. Ideally, we have two researchers code the data and compare notes to ensure reliability.
When we manually code qualitative data, we go through every participants’ comments using the approach above (or similar to it). The process is organic, with new codes added as needed to fully explain the comments, and all meaningful feedback is assigned one or more codes. This is just as time-consuming as it sounds. We typically spend three times as many hours on analysis as we did collecting the data. So if we conducted 10 interviews lasting an hour each, we might spend 30 hours or more on coding and analysis!
At Moonshot Collaborative, we know clients can’t always pay for researchers to spend that much time manually coding qualitative data. Thankfully, we also have excellent tools to partially automate text analysis to analyze for key topics and themes like the codes in the example above. We also have the ability to automatically code all comments (and open-ended survey results) according to overall sentiment — positive, negative, neutral, or mixed.
Have qualitative data you need analyzed? We can help.
Step 3. Turn Codes into Themes
The next step in qualitative analysis involves translating the codes you developed in step two into themes and narratives. To do this, we divide the codes or topics into different thematic groups and subgroups, with the goal of developing an overall, cohesive taxonomy of themes for our particular research question. If our study involves multiple, distinct research questions, then we repeat this process for each one.
As an example, let’s assume we received the sample comment about plant-based meats above in a set of interviews, along with other, similar comments. We might find the following themes emerging from the data: for some plant-based foods, taste is already comparable to their conventional versions; consumers express reluctance to eliminate specific foods (e.g., bacon, cheese); and uncertainty about net health benefits for plant-based foods.
As a final check, we review the themes and the taxonomy that we developed against the original dataset. After reading through the transcripts or open-ended responses one last time, we look for any themes that we missed or that we may have emphasized too much in our analysis. Ideally, a second researcher will also review the themes and the original dataset and provide another set of eyes for quality assurance and to limit bias.
Step 4. Write Up the Results!
The result from all of our work in steps two and three is a detailed and hierarchical set of themes derived from participants’ direct feedback. As we worked through the earlier stages, some themes were already probably becoming evident. The final step in the analysis is to confirm and expand upon the most prominent themes and use them to answer the research question(s) posed by the study and the client.
Then we write it all up! The report for qualitative data often includes a discussion of the key themes that we found during our analysis, with supporting verbatim comments from participants. While qualitative analysis usually does not include charts and graphs, the sentiment analysis mentioned earlier allows us to visualize participants’ feedback and even associate their overall positivity or negativity with the specific themes we identified.
For example, if you asked customers of a vegan restaurant to give feedback on their experience, you would be able to examine sentiment (positive, negative, neutral, or mixed) relative to different things they mentioned. This could include the food itself, the customer service, cleanliness, internet access, etc. The chart below (produced by Qualtrics) shows what the results of this kind of sentiment analysis might look like.
The approaches described in this blog can provide a deep understanding of how research participants think about a given topic, whether it’s plant-based food, cruelty-free products, or sustainability in general. Qualitative data is rich in information, but also very challenging to analyze without expertise or the right tools.
How can your company get the most out of qualitative data? For a free consultation, contact Moonshot Collaborative, or sign up for our newsletter for tips and insights.
