How to Use Qualitative Research to Drive Conversions

When marketers think of using data to come up with test hypotheses, they often turn to their digital analytics. 

Yet often times, qualitative research can offer more insight than anything else working to come up with winning test hypotheses.

Where quantitative stuff tells you what, where, and how much, qualitative tells you ‘why.’ The goal of qualitative research is to gather an in-depth understanding of user behavior and why they took those specific actions.

Consider this scenario.

You have a 4-step sales funnel on your website. Most people have no problem taking the first step, but almost all of them drop off at step #2.

Why?

This is where analytics data can fall short, and won’t be enough to figure it out. You need to know what your actual users are thinking when they see that page, and leave. And the only way to get that information is through asking your users.

Qualitative research enables us to peek inside the mind of the people we’re trying to sell to (data through buyer intelligence). Without knowing what they’re thinking, you’re in the dark. You can speculate all you want, but it’s not going to get you anywhere.

Qualitative research sample size requirements

Qualitative research sample size requirements are different from quantitative research. You don’t need anywhere close to the same amount of people.

Qualitative research doesn’t need the same numbers as quantitative research because it is focused on understanding the depth and complexity of people’s experiences, rather than making generalizations about the general population. 

In most cases, a sample size of 15 people is already good enough to get strong insights.

How you conduct qualitative research matters

Qualitative research involves people.

What you ask them, when you ask them and how you ask them will all influence their response. They might lie to please you, they might be influenced by how you frame a question (“Don’t you think this looks more secure?”).

Michael Summers, Global User Experience Research team at PayPal:

There are two very precious cognitive resources in the mind that influence how we study optimization:

1. attention,
2. and external motivation.

We all have a sincere desire to understand how our UI is supporting conversion—or what obstacles or barriers exist.  For example imagine a young child with a magnifying glass—who leans close to study the fragile flower—only to accidentally concentrate the suns rays and cause the flower to burn.

Many times when we try to “study” something, we get in our own way.

When we try and watch people encounter our UI or our conversion funnel, perhaps arriving at a landing page, evaluating options, we many times artificially distort attention and motivation.

Which causes us to we will walk away with unrealistic values for things like comprehension, retention, decision to act, time on task, success rate, and error rate.

For this reason, the way we study people matters, more than you may think.

How do top experts use qualitative research to get insights?

I reached out to fellow conversion optimization people, and asked them how they are going about qualitative research. Which research methods have been most useful to them, and how what specifically have they asked the users, or have them do to get insights?

User testing

User testing and usability testing are the favorites for many optimizers.

Dr. Karl Blanks from Conversion Rate Experts praises user testing:

User tests allow us to see how visitors interact with a site, but they also have other benefits. Each user test gives us a deeper intuitive understanding of how people use websites. And they allow us to collaborate with the user, so we can work out solutions to difficult problems. No other technique delivers so many “insights-per-minute” on a prolonged basis.

Craig Sullivan, optimization guru:

By far the most useful technique is user testing, with as early stage of a product or new feature as you can get.  This is where the best blinding insights, product pivots, winning tests or great ideas came from in my work.

Start on paper or using some of the great prototyping tools out there—and get feedback, early and often as you iterate and build something out.  I must also wave the flag for Voice of Customer or Feedback tools, particularly where you are measuring customer satisfaction (e.g. NPS) and service ratings for multiple elements of the product.   Working out where to invest for increasing delight works perfectly with an optimisation programme.

Moderated user research

User testing has to be handled with care—and often moderation is needed to make sure we’re not just having the testers go through the motions.

Paul Rouke from PRWD:

Over the last 15 years of conducting user research, I’ve witnessed a range of different approaches and moderation styles, but unless you’re careful you can invalidate the research. It’s crucial when using this research technique to do the right preparation, the right recruitment and use the right style of moderation. Follow this and you can’t fail to gain powerful and highly actionable user behaviour insights, which to date, have led us to developing some of our most impactful A/B test hypotheses.

Moderated research comes under criticisms such as “people will tell you what you want to hear” or “this isn’t an environment where people usually browse.” These can be negated with the right facilities and welcome questions prior to conducting any research scenarios, allowing participants to provide their thoughts in a neutral setting. Plus get the right level of observation, probing and non-leading questioning and people have a phenomenal ability to unlock and share the hidden motivators in their brain.

A quick tip that we often use to gain more considered feedback and insights from people – provide at least two different digital experiences to compare to each other. This allows people to progress their thinking to what really does matter for them, and helps to identify strengths and weaknesses across the different experiences.

One final often overlooked opportunity using moderated research, is how it can be used to gain insight or validate the fundamentals of your businesses. Conducting research on your proposition to see if it resonates and persuades your target audience can provide you with strategic insights and the opportunity to gain advantage over your competition.

Michael Summers, Global User Experience Research team at PayPal chimes in again:

The best way to learn about your UI, is to mix it in with competitors, or even non-related tasks, and ask people to conduct a very general goal.

By doing rapid on-line tasks in sequence, and not even revealing to participants what you are studying, you are less likely to artificially slow them down, and also less likely to impose external motivation such that they understand more than they would under natural conditions.

Also, before you even do your blind set of tasks—ask users to spend some time on-line without you directing them at all.  Allowing them to read the news, check Facebook, etc., will help them acclimatize to an unfamiliar environment and begin to move at a more realistic tempo to what they would do at home on their couch.

Live chat transcripts

Do you use live chat on your website? It’s a proven tactic for increasing conversions. You can also use it to get insights.

Brian Massey, the Conversion Scientist:

Live Chat transcripts tell us what people are looking for who don’t want to search. It also tells us what is missing from a site or what is hard to find.

Sometimes you can feed the transcript into Wordle.net to tease out words that are used frequently, giving you clues to larger issues.

Interviewing customers,  sales and support people

Interviewing customers and stakeholders is without question one of the most insightful qualitative research methods in any CRO toolkit.

In my experience, nothing beats actually talking to your target audience. You know, calling a customer or jumping on a video chat and having an actual conversation.

The insight you get are priceless, and no amount of quantitative data will let you reach the same level of understanding.

When it comes to copywriting and messaging, I find customer interviews to be especially insightful. In many cases there’s a huge difference between what marketers think the target audience needs to hear—and what the target audience actually needs to hear in order to make a positive buying decision.

I’ve been involved in several optimization projects where customer interviews revealed that the core value proposition was fundamentally flawed. Furthermore, the answers I got from these interviews got me much closer to the winning optimization hypothesis.

I also find it extremely useful to conduct interviews with employees from Customer Service, Support and Sales. These guys and gals spend all day talking to customers and have in-depth knowledge of the problems and issues that they are dealing with—both in relation to the website and the product itself. Moreover, they are familiar with the decision-making process of the target audience and can help us build better optimization hypotheses.

When I interview employees from Sales and Customer Service, one of the most useful questions is a simple: “What are the five most common questions you get from (potential) customers?”

Followed up by this question: “What do you answer when you get these questions?”

Try it out next time you are conducting qualitative research for a CRO project. I bet you’ll be amazed at how much insight you get.

Customer surveys

One of my own personal favorites is customer surveys. Surveys are mostly about learning who the customers really are, what they want, what’s causing doubts and hesitations, and the language they use. This is critical for copywriting, understanding common pain points, and learning what matters to them about the products you sell etc.

When conducting surveys, its important to only survey recent first-time buyers where it has been no more than 30 days since their purchase (better if it’s right after completing the purchase.) Survey them past 30 days and they can forget what they were thinking when they shopped on your site (and then bullshit their answers).

The best online surveys have open-ended questions. No (or very limited) multiple choice or scale-based responses.

What to ask? It depends on the business/website. Broadly speaking, you want to understand 4 key things:

• Who they are: Pay attention to how they self-identify, useful for putting together personas;
• User intent: What’s the specific problem they were solving;
• Shopping process: What mattered to them when choosing the product, what kind of comparisons did they do, how many / which other sites they looked at, and so on;
• Friction: Fears, doubts and hesitations they experienced before making the purchase.

I have an article on learning about your audience here.

Dr. Karl Blanks from Conversion Rate Experts:

We survey people who have just signed up or purchased something. The golden questions for them are these:

• What’s the one thing that nearly stopped you buying from us?
• What was your biggest fear or concern about using us?

Web traffic surveys

Most people on your site will not buy anything, but even those who don’t can provide valuable insight.

There are 2 ways to survey your web traffic:

• Exit surveys: hit them with a popup when they’re about to leave your site.
• On-page surveys: ask them to fill out a survey as they’re on a specific page

Both are useful. What should you ask? Since our goal is to get more people to take action, start with learning about friction. What are the FUDs (fears, doubts, hesitations) they are experiencing—while on a specific page?

Step 1: Determine the most wanted action for the page.
Step 2: Come up with a question that asks about friction.

If for example, you have an ecommerce product page, the goal would be cart adds. So the question to ask could be something like “What’s holding you back from adding this product to the cart right now?” or “What’s keeping you from buying this product right now?”

Take note of the responses then spend time reviewing any potential themes. See if your customer survey responses about friction are similar to web traffic survey results. You’re mining for insights!

A bunch of people will tell you that pricing is an issue (“too expensive!”)—that’s to be expected. If that’s a dominant response, you’re either driving too much unqualified traffic to the site, or you’re not doing a good enough job communicating the value of your product(s).

Conclusion

Qualitative research helps you look inside the mind of your customers. You will learn what they really think, what they want and, how they want it. If you can’t figure out why people are dropping off here or there, or how to increase motivation to take action, qualitative research can help.