There are few things in marketing we know for certain.
Show a landing page to a panel of experts and ask what’s wrong—and everyone having an answer is one of them.
While there is often no shortage of opinions on how to improve a landing page, the question remains how valuable is most of the feedback?
Even if the panel consists of experts, opinions still aren’t worth the weight of in-depth research.
On-page surveys, however, can be incredibly useful in unlocking insights for conversion optimization.
Table of contents
- On-Site surveys defined
- Using on-site surveys to remove friction, increase conversions
- Before anything else, define your objectives
- When to pop the question?
- Which questions should you ask?
- How to get more people to respond
- Mind your cognitive biases
- Using on-page surveys: A swimsuit case study
- On-Site survey tools
On-Site surveys defined
Some time ago, I wrote an article on customer surveys. While both on-site and customer surveys are under the broad category of ‘qualitative research,’ the former is significantly different in their goals and execution.
While customer surveys ask questions from people who bought something from your site, you can use on-page web surveys to ask questions to a variety of different segments of visitors who use your website.
In conversion research, the main goal is the same for both types of surveys — you’re trying to identify sources of friction. On-page surveys are useful because you’re getting real-time feedback as visitors are experiencing your site.
Web & exit surveys are a kind of pop-up box that appears to the visitor based on certain rules—like time spent on site, the number of pages visited, activity (e.g. moves the mouse cursor next to the browser window closing X).
This is what they look like:
What can you learn with on-site surveys?
You can learn a lot from on-site surveys. For instance, you can:
- Identify the demand for new products or improvements to existing products.
- Figure out who the customer is to help define accurate customer personas.
- Uncover UX issues;
- Locate process bottlenecks;
- Understand the root causes of abandonment;
- Distinguish visitor segments whose different motivations for similar on-site activity are undetected by analytics;
- Decipher what their intent is. What are they trying to achieve?;
- Find out how they shop (comparison to competitors, which benefits they are looking for, and the language they use, etc).
What fears do they have about handing over their credit card number? What doubts do they have about your product or service? What’s stopping them from buying—emotionally, functionally, or otherwise?
Note: there are other things you can learn from on-site surveys, of course, like NPS. There are tons of good articles on those other purposes, so we’ll just focus on the conversion research side of things.
Using on-site surveys to remove friction, increase conversions
On-site surveys are critical for conversion research. Here’s how Dustin Drees, optimization consultant, put it:
Take, for example, this case study that Optimizely wrote up on Teespring. Teespring collected qualitative feedback in various ways, including customer surveys, live chat, and on-site surveys.
Through their research, they discovered that credibility was an issue. Because Teespring has an unconventional commerce model (they only ship the shirts once the minimum order size is hit), they needed to establish trust.
In their research, they heard anecdotes like: “Not sure if I should give my credit card information,” and, “Not sure if I’ll get my shirt.”
With this in mind, the team set up a test between two CTAs. The original:
And the variation:
The subtle change in microcopy ended up increasing conversions by an impressive 12.7%. Not bad for changing a single line of copy.
Voice of customer research using on-site surveys
In doing conversion research for LearnVisualStudio.NET, they discovered that a vast amount (almost 2/3) of respondents considered themselves beginners, and 69.74% of respondents said they were “most interested in finding their first developer job.”
They took these insights to form a basic copy experiment. Here’s the original:
By simply telling visitors who the lessons are designed for (and where the lessons will get them), conversions increased on the Courses (+9.2%), Plans and Pricing (+24%), and Curriculum (+23.9%).
They then dug deeper into the survey responses and made a few more changes (below the fold copy, CTA, headline, etc.) Their variation ended up outperforming the original on the main call to action button above the fold by 66.3%. All primarily fueled by VOC insights from on-site surveys.
Before anything else, define your objectives
The effectiveness of your on-site survey strategy hinges on clearly defined business objectives. Otherwise, you’re just wasting your time.
What goals do you want to achieve? Be specific.
As HotJar puts it: “Before you start picking survey questions (and we’ll have you covered in just a few paragraphs), think critically about what you want to accomplish and where the best place to ask your questions may be:
- What pages are crucial to your business?
- Which over- or under-performing page(s) could most benefit from additional insight?
- Where in the conversion funnel have you spotted a leak that needs investigating?”
To ask the right questions, you need to decide what kind of data you are interested in collecting and what you will do with it. If you are not planning on acting on any of the answers, then all the effort (yours and your customers’) is wasted.
Dustin Drees factually considers a lack of clear focus as the largest mistake you can make with on-page surveys. Starting with no clear goal compounds the inefficiency of the surveys:
When to pop the question?
Since you can target when and to whom you’re showing the on-site survey, keep in mind two things when deciding:
- Qualifying the visitor (is this a random visitor or someone that is considering purchasing?)
- Asking the right question at the right time (e.g. if you ask someone why they didn’t buy right when they land at the site, there will be lots of friction and confusion, and zero insight gained).
Look at your average time on site and page views per person metrics: ask questions from people who have above average engagement (for qualification reasons).
One heuristic to follow is to target people just above the average engagement. That way, you’re getting users at least in consideration of a purchase.
Ask the right question on the right page. Don’t ask, “why are you here today?” during the checkout. You get the idea.
Which questions should you ask?
Avinash Kaushik once wrote about the “three greatest survey questions ever.” The story is that, when asked which analytics tool he’d recommend to a VP on a short time frame, he answered that she should not install an analytics tool. Instead, install an on-site survey and ask these three questions:
- What is the purpose of your visit to our website today?
- Were you able to complete your task today?
- If you were not able to complete your task today, why not?
He then explained more about web survey strategy in another article, explaining that the greatest wisdom is to be gained from open-ended questions:
That said, depending on your strategy, there are many more questions that can bring insight than those three. Think about trying to answer two categories of questions, in general:
- Why did they come to the site? Does our site match their needs? If not, are we attracting the wrong traffic? Or is there an opportunity here we aren’t capitalizing upon?
- What are the sources of friction? This is more specific that “why they didn’t buy” (understanding that is our main objective, but we have many goals to understand the big picture).
Examples of questions to ask
Here are some example questions you could ask (feel free to tweak the wording as necessary):
- What’s the purpose of your visit today?
- Why are you here today?
- Were you able to find the information you were looking for? This can identify missing information on the site—best asked on product pages.
- What made you not complete the purchase today?
- Is there anything holding you back from completing a purchase? This helps you further identify any friction.
- Do you have any questions you haven’t been able to find answers to? Y/N
- Were you able to complete your tasks on this website today? If the answer is no, allow your visitor to expand on the reason why.
There are so many different questions you could ask and get actionable insights; it all depends on your strategic goals.
How to get more people to respond
There’s no single magic question that resonates with every audience, so you’ll have to experiment.
Typically, though, there are two ways you can go about setting up your on-site survey:
- Ask a single open-ended question.
- Ask a simple yes/no question, and ask for an explanation once they’ve answered it.
The second often almost always works the best for us. As for why that is, it’s probably because of a psychological principle known as “commitment/consistency” (remember Cialdini?)
Once the user starts on the path by answering the easier Y/N question, they are compelled to continue by following up with an explanation.
Here we asked two questions that got almost the same amount of views, but one got significantly more responses:
What was the difference?
- Winning question (517) responses: Do you have any questions you haven’t been able to find answers to? Y/N
- “Losing” question (182) responses: Is there anything holding you back from making a booking? Y/N
The lesson is simple: you need to experiment with different wordings. Some work better than others with specific audiences.
Here’s another example. We asked the following three questions:
- Why didn’t you complete a purchase today?
- Is there anything holding you back from making a purchase today?
- Do you have any questions you haven’t been able to find answers to?
Can you guess which question yielded which result? The results are below (pay attention to views/responses ratio):
The first question performed overwhelmingly better than others. Which questions lined up with which results?
- The first question (winner) was “Is there anything holding you back from making a purchase today?”
- Second question was “Do you have any questions you haven’t been able to find answers to?” (worst performer).
- Last one: “Why didn’t you complete a purchase today?”
The takeaway? There are no universal questions that will resonate with all audiences. Our previous winning question performed worst on this audience. Keep experimenting.
Sometimes small changes to phrases can have a big difference on response rate. For instance, Groove changed their question from “why did you cancel?” to “what made you cancel?” and got nearly double the responses.
No one can know for sure why the increase occurred, but the theory is it was because the former was accusatory and put the customer on the defensive. Either way, small changes can be big increases sometimes.
Mind your cognitive biases
Just as customer surveys are susceptible to cognitive biases, on-page surveys are as well. Actually cognitive bias can strike easily on both parts of the on-page survey process: creating the survey and analyzing the results.
For example, confirmation bias causes you to overtly focus on responses that back up what you already think (and you ignore all those that say otherwise). Dustin Drees points out how to avoid some of the common blind spots:
Using on-page surveys: A swimsuit case study
Ecommerce agency, Growth Rock, had a client who sold plus-sized swimsuits for women. One of their first tasks was to help improve the product pages.
While sending a customer survey via email was considered, they ultimately decided to use on-page surveys to avoid emailing the client’s customers too much.
Using the tool Hotjar they asked:
And the initial responses were revealing:
As shared on Grow and Convert, they were incredibly surprised at the results, “Hold the phone. They don’t have enough sizes? This is a plus size swimsuits site, the right size selection is a core part of the value proposition.
Obviously, that’s not a UI/UX issue, it’s a merchandizing issue. So it’s not something we can “fix” with an AB test, but it’s incredibly insightful.”
But they didn’t stop there. They decided to take their learnings one step further by asking a few additional questions.
And yet again the responses were incredibly valuable.
A few strategic questions were able to uncover issues that would have never been solved through an A/B test.
On-Site survey tools
The tool should be a secondary consideration to your strategy. Still, there are some options when it comes to tools (or read our huge conversion optimization tools guide, all reviewed by experts)
On-site surveys are a critical piece in the conversion research puzzle. They help you identify and remove friction from the purchasing process, and increase conversion rate in the process.
Because you’re polling traffic, and because of advanced targeting and segmenting capabilities of polling tools, you can do a lot of different things with on-site surveys.
But, make no mistake: just throwing up a HotJar exit survey with a random question does you no good at all. You must have a very clear goal in mind with your qualitative research, a goal that will inform your execution of the surveys. In addition, keep these ideas in mind:
- Ask the right question at the right time (important!)
- Experiment to maximize response rate;
- Mitigate cognitive biases;
- Make sure your on-site surveys fit into and inform the rest of your conversion research efforts.