Men have become the tools of their tools.Henry David Thoreau
We’re asked often whether a tool is good, bad, or downright awful, and until now, haven’t put it all together into a cohesive post.
To navigate the list, you can filter tools based on function as well as company size. We also gave the option to view only Peep’s favorite tools.
AB Tasty is another testing tool designed for the masses, this one hailing from France. They offer client-side, bayesian-based A/B testing along with full factorial multivariate testing—the common experiments.
The things I like about AB Tasty I like a lot:
The dashboard is just a plain list of tests on a white background. Each test has only two main options: Edit or Report. There is a separate page for the Editor and a separate page for the Report. Simple and intuitive.
The Report is a one-pager showing all goals one under the other with confidence interval graphs. The report load time is slow, but it has some powerful features, like the ability to use goals as filters. You can, for example, isolate the conversion rate for all visitors who played the video you’re testing. You can fire a goal on mouse movement and use that to exclude bots. It opens the door for detailed analysis.
The Editor works smoothly and allows you to interact with each variation. AB Tasty has considered some details overlooked by more mature tools. You can add a custom goal right in the code, and it automatically shows up on the report—you don’t need to create the goal first.
Each variation in the report has a preview link, and the preview URL is simple. There is even a Planning screen in Beta to track testing ideas.
Unfortunately, AB Tasty has some major show-stoppers. There are flicker problems, even with synchronous code, and the ability to control for flicker is limited.
VWO, in contrast, has a solid workaround to hide specific elements temporarily to avoid flicker. AB Tasty has no full data export, so you’re limited to what the tools shows. Time charts on reports are rudimentary and disappear once a filter is applied.
Support is friendly but slow compared to VWO or Optimizely. It can take days to hear back and weeks for back-and-forth about an issue.
I’ve logged these issues with AB Tasty, so hopefully they will get that fixed sooner rather than later. It has the potential to be a fantastic tool.
Adobe Analytics is part of Adobe’s Marketing Suite, and it’s a powerful platform.
Formerly known as Adobe SiteCatalyst, the tool has pretty much everything you could want, including real-time analytics, mobile app analytics, funnel tracking, and Predictive Intelligence. It’s basically only for enterprises, though. It’s not cheap.
Adobe offers a selection of analytics products named “Adobe Analytics” as part of the “Adobe Marketing Cloud.” Their analytics tools can be used for collecting, reporting, visualizing, targeting, testing, and predicting with data across different digital experiences: site and mobile.
Other Adobe products offer capabilities for marketing automation and email, media optimization, audience insights, creating and delivering experiences across digital and television, and social. They have a complex ecosystem and are a veritable one-stop-shop for many marketing requirements, but not all.
Some of their products have significant install bases and can be considered top of class, while other products are not fully integrated or don’t interoperate in obvious ways.
Great salemanship is to be expected at Adobe. Adobe has even been rumored to give away free product that may not be considered best in class to beat competitors simply by giving it away for free. Free often wins versus buying a competing product.
Adobe products can have some complexity to implement, but that should be expected given Adobe’s feature set and customizability.
The services offered post-sale are not necessarily well-regarded and, in the worst case, may be considered “nickel and dime.” Adobe may not even talk with customers to answer deeper questions, instead explaining that consulting time may need to be purchased to answer the question.
Account managers can tier customers. High-value accounts are rumored to get more attention; the rest are purported to get little to none. But don’t worry. It’s a strong sales culture with great salesmanship that may certainly help you out at renewal time.
Adobe Target is an enterprise testing tool that does a great job of leveraging a visitor-based metric system. It’s one of the most popular enterprise tools out there.
Here’s a review from Andrew Anderson:
Pros: Offers the best in-product segmentation available. Can offer much better data due to true user basis then tools like GO or Optimizely. Most flexibility of any tool out there. Part of larger Adobe Suite and can take part of other tools such as Adobe Tag Management, AEM, Analytics, and 360 profile. When paired with the adobe consulting team or people who actually deeply understand the tool, it can deliver by far the most value of any tool out there. The beauty of the tool is that it can do a million things…
Cons: …The curse of the tool is that it can do a million things (and that Adobe pushes you constantly to do the least valuable ones). Steep learning curve to do the things that matter most. While they offer WYSIWYG editing, it is still at its best with manual HTML overwrite. Ties to Adobe Analytics and other poor product decisions have crippled the once-best tool and made it just an offering for analysts (read: one of the two worst groups to do testing).
Interface is heavily focused on doing the wrong things. Target Premium offers the second best advanced adaptive-learning system to Conductrics but requires a massive amount of data to work ideally. The Adobe Consulting team is now little more meaningless Yes Men who offer little additional value. More than any other tool, it is almost impossible to plug and play and get any sort of real value (though in many ways that is true of all tools). High cost and barrier for entry means that a lot of people dismiss it without really understanding the true benefits of the tool.
Overall: What was once by far the best tool on the market is now almost an afterthought. At each step of the way from Offermatica to Omniture to Target it has progressively lost value, which is pretty hard to do. Even worse, a tool like Touch Clarity, which was a decade ahead of its time, has been crippled as it moved to Test and Target 1:1 and now Target Premium. Throughout the 2010s, it lost value at every stage due to poor product management and positioning in the Adobe Marketing Suite. It now requires the most data discipline and the best understanding of causal versus correlative information in order to achieve its true value.
The value pieces are still there but they are hidden under a multitude of product sins. At this point all those features in the hands of the average optimizer are a bit like a baby with a handgun, you’re not sure what will happen but in most cases it is all bad. It still can do more and can present more value than just about any other tool, but at this point the barrier to get to that value is not worth the additional cost for the tool.
Adobe has a great sales team and a decent analytics offering, which is about the only thing keeping this once mighty king afloat. If you’re willing to get past the high barrier for entry, it can and will give you more flexibility and value then tools like VWO or Optimizely.
But, for almost all organizations, there are better cheaper options out there.
Browserling offers live-interaction, web-based browser testing.
It gives you a real visual from each browser. From there, you can also make annotations and report design errors on any browser.
BrowserStack is a live, web-based browser testing tool.
With BrowerStack, you fire up different configurations of operating system, display size, and browser variation. For cross-device testing, you can emulate a range of mobile environments across Android, Windows, and iOS.
Google Analytics’ device and browser reports will guide you in terms of which combinations to use in your QA.
A common complaint is that it can be slow, which becomes a pain in the ass when you’re testing multiple configurations. Obviously, it’s not as fast a local virtual machine, but that’s a small trade-off for an affordable, comprehensive browser testing system.
ChangeAgain.me is a good solution for companies that already have Google Analytics and want to try split testing.
They bill based on the number of tests you run, not the number of visitors, as other split-testing solutions do.
No coding skills are necessary. You can launch new tests with a point-and-click visual editor.
Clicktale is an awesome tool that does session replays, heat maps, and conversion funnels.
Clicktale records users’ sessions, which can be replayed. This qualitative data is quite a divergence from the quantitative data that users are so used to seeing in Google Analytics and provides unique insights into user behavior.
Clicktale provides very nice heat maps of mouse movements, clicks, as well as mobile-device tracking (though I haven’t tested the mobile functionality myself). Clicktale’s form analytics are second to none.
The ClickTale “Event” API, a way of tagging a visit with a custom variable, allows robust tagging and custom segmentation of traffic within the tool (though using the term “event” isn’t very intuitive). There is significant native segmentation capability within the user interface as well.
Clicktale measures user behavior very differently than clickstream tools, so there are some questions it cannot answer well. The data model does not lend itself to tracking how many times certain actions happened over time.
Merchandising analysis would be difficult, at best, with the tool. You won’t find reports for Revenue per Channel or Return on Ad Spend over time like you will in other analytics tools. (Don’t expect tabular data you can export.)
Multi-channel analysis and attribution modelling is non-existent in the platform. Clicktale also doesn’t have any cohort or retention analysis reports. No pricing about the product is available on their website, but from what I can gather from speaking with people, they are focused on the “enterprise” (i.e. $$$$).
Clicky is free (if you have less than 3,000 daily pageviews) and directly positions itself as an alternative to Google Analytics.
Clicky has a few features that differentiate (all real-time analytics, heat maps), but I don’t think it’s worth switching from GA. If you want heat maps, there are other free tools.
Conductrics is an advanced tool that blends A/B Testing with machine learning to deliver optimal experiences for each user, automatically.
You can use it as either a server-side or client-side tool, and you can customize it pretty much any way you’re trying to use it.
Here’s a review from Andrew Anderson:
Pros: Offers a lot of flexibility as far as ways to interact with your systems, from APIs to one-tag solutions. Is at its best when it is not being used for just straight A/B Testing, but instead as an active decision agent that is constantly adapting and creating experiences.
Because of the ability to select factors, instead of just blindly adding them all in, it allows for this functionality on far smaller sites than other similar tools (Adobe Target Premium/1:1/Touch Clarity, X+1). Very robust system with very low variance and a wide variety of features. Great support, although the help menus are a bit technical for most groups.
Cons: Requires more knowledge about decisioning agents then most programs have. QA is less than ideal. Data reporting can use some improvements, especially graphically. Has a bit of a learning curve when it comes to reading the Targeting report. Still requires a fair amount of traffic to reach full potential.
Overall: Conductrics is a great tool for programs that really want to tackle adaptive-learning personalization or want a lot of flexibility when it comes to interactions.
It does require a lot more knowledge and has more of a learning curve than basic tools like Optimizely or VWO, but in the end it can provide far better functionality and more flexibility than just about any other tool on the market.
It’s a great tool for programs that are above the standard tools but not at a Fortune 100 size.
Convert.com has joined the likes of VWO and Optimizely in making an easy and affordable testing tool—though Convert aims more at the enterprise crowd.
Here’s Julien Le Nestour, an applied behavioral scientist and CRO consultant based in Sydney, giving his opinion:
Convert.com has a fantastic tool that should be considered and evaluated by any CRO agency or end-user client that also evaluates Optimizely and VWO. They offer a level of functionality that is globally on par with them for testing but at a lower price once you go above the free level offered by Optimizely.
The Convert.com team is very experienced in testing, and they go to some incredible lengths to offer an unparalleled amount of support when you are building or running your tests. In my experience, they will assist you in all technical aspects and issues and solve them for you as part of their plans.
You can also ask them to create tests directly for you if you don’t have anyone on your team to do it as an additional service. The only (potential) limitation would be a more traditional approach to measuring statistical significance, but whether it’s important to you depends on your views on this subject.
Crazy Egg offers heat maps, click maps, scroll maps, etc., to offer a visual glimpse at how users navigate your site.
Here’s what Peep had to say about Crazy Egg:
The poor man’s mouse-tracking tool. It has the basic heat maps but no session recordings or form analytics, and limited segmentation capabilities. Annual payments only.
Decibel is a high-powered, enterprise-level customer experience analytics tool. They offer session replays, dynamic heat maps, form analytics, behavorial alerts, and error reporting.
With Ethnio, you can recruit your own testers from website visitors—very useful for recruiting testers.
Watching actual customers interact with your website can generate powerful insights for optimization. It’s a solution to the notoriously “canned” nature of responses associated with semi-professional panels on popular usability testing platforms.
We use Ethnio on every CRO project to recruit real users in real time. An entry pop-up invites visitors to participate in the study. Candidates are then qualified based on their responses to a list of screening questions. Setting up and launching a campaign is quick.
You can auto-pilot the study by integrating with Usertesting.com. Our experience with that hasn’t been great though. Things fall apart when users are asked to install the third party tool, so you end up with abandoned videos. The magic happens when you phone up candidates immediately, and set up screen sharing with a tool like GoToMeeting there and then. You get to virtually peer over the shoulders of a real customer in an authentic scenario.
A word of warning: Response rates tend to be low, so you may need large traffic numbers or a big incentive.
Google Analytics is the most common analytics tool used by marketers.
It integrates with almost everything. It’s easy to use. There’s no reason not to have it, really. Install it in less than 10 minutes.
Pros: Google Analytics can be used to answer a range of business questions. The tool has huge adoption, which means that many people know how to use it (at least on a cursory level) and are comfortable with the interface.
It also means that there are tons of resources available for GA, some free and some paid. Google Analytics is known for its excellent UI and great data visualization capability.
The tool is extremely powerful and can crunch massive data tables at tremendous speed. Indeed, it’s so fast that people often become frustrated by how long it takes to get a report because they’re so used to Google’s fantastic processing power. (Yargh! Why is it taking 15 seconds to calculate 3.5 million rows of data on the fly?!)
Some well-implemented features include attribution modelling, multi-channel analysis, motion charts, and enhanced ecommerce reporting. The on-the-fly segmentation capability is terrific. GA’s integration with Google Ads is very tight.
Google Analytics has a very robust team, and they keep the entire analytics industry on their toes with continual product updates. The software license for Google Analytics Standard is free.
Cons: Because the software license for Google Analytics Standard is free, people don’t invest the requisite time and money into the tool to make sure that it’s configured properly for their business.
GA out of the box is robust, but it’s not “business ready”; the tool needs proper strategic setup. The Google Analytics data model is fairly rigid; they have somewhat strange concepts, such as data scope, which makes the tool complex and sometimes difficult to understand.
Data is almost never reprocessed, which means if you get some bad data into GA or misconfigure a goal, it’s like a bad grass stain on a nice pair of white pants. User segments (which are great!) are limited to a 93-day report window.
Cohort analysis in GA is relatively weak compared to other tools. Google Analytics Standard begins sampling data in reports at 500,000 sessions in the date range, which impacts larger sites in particular.
Most importantly, GA does not retroactively stitch user IDs. As a result, the most important user/customer analytics that an analyst would want to perform in GA are crippled.
(Yes, crippled is a strong word, but understanding user behavior when they are in the awareness and consideration phases of the customer journey [i.e. pre-login] is business critical.)
Google Forms is a great survey tool. It’s free. You can get almost anything done with it.
There aren’t any bells and whistles, so the surveys look kind of plain.
Google PageSpeed Insights analyzes the content of a web page, then generates suggestions to make that page faster.
It’s free, but it’s harsh. Don’t be surprised if your site’s mobile experience scores in the low double-digits (or single digits).
Heap automatically captures every user action in your web or iOS app and lets you measure all of it after the fact.
Here’s a review from Peep:
What I like about Heap is that it captures all events (e.g., clicks, submits, etc) without any configuration. You do need to set up the reports later when you need them, but that’s easy.
It’s super valuable for funnel analysis and finding correlations between user behavior and higher conversions.
Hotjar does (almost) everything. In addition to their triggered polls, they also offer funnel tracking, session cams, heat maps, and more.
They’re available for a good price, too, and actually have a pretty extensive free version.
Here’s Michael Aagaard from Unbounce talking about the tool:
I’ve been using HotJar since it was in beta, and I totally love it.
The fact that they provide seven tools in one bundle makes life so much easier for me! I don’t have to pay for a bunch of different tools; I don’t have to go through a bunch of different accounts; and I don’t have to implement a bunch of different scripts on the website or landing page to get the insight I need.
If you’re a consultant, you know how much of a hassle this can be.
Also, the interface is delightful and very easy to interact with. I mention this because similar tools I’ve used have had a horrible UX design, and that really bugs me.”
And here’s a second opinion by Alex Harris:
Mouse-tracking videos are probably one of the best features in Hotjar. They give you the ability to see how visitors are going from page to page. This helps you see the user journey from the customer’s perspective.
To make this even more useful, you can filter reports by different recording and visitor attributes. For example, you can identify which recordings show a visitor who has gone past more than a few pages. That way, you don’t have to watch visitors that visit only one or two pages.
You can even watch recordings based on a specific user ID, device, geo location, operating system, browser, or device.
Their funnel tracking is much easier to understand than the funnel report inside Google Analytics, plus it’s much easier to set up. You can also click to watch videos of all the visitors that reached a certain point in your funnel. This helps add to your hypothesis to understand the insight of why the visitors may be dropping off.
There are still many feature requests needed to make this the perfect tool. For instance, you can’t do tracking across domains. If your blog is on a subdomain, then you can track the flow from your main site to the sub domain only.
Inspectlet is feature rich and affordable, and you can gather and analyze a higher volume of clickstream data compared to other tools in the same price range or category.
They do form analytics, session recordings, and heat maps.
Iridion is a workflow tool focused on expert optimizers.
They’ve also added their knowledge from thousands of A/B tests to help growth hackers and optimizers build stronger experiments with higher uplifts.
Lucky Orange offers an “All-in-One Conversion Optimization Suite,” including analytics, recordings, live chat, heat maps, funnel analysis, form analytics, and polls.
Oracle Maxymiser is a powerful tool, one of the more expensive enterprise tools out there.
It offers lots of different solutions, lots of flexibility, and focuses on personalization as well. Almost across the board, reviews say that they have great service and support, and people tend to like their UI.
One big con is, of course, that Maxymiser is expensive and, therefore, not as accessible to smaller organizations.
It can tell you pretty much anything, but you have to configure it yourself.
We use Mixpanel for funnel tracking and mobile analytics. The great thing about it is that it’s super easy to set up, and super easy to gain high-level insights. I also love the ability to trigger push notifications, SMS, and emails. They also make it easy to get data out, though it is a bit tricky.
There are a couple downsides, however. For one, we’ve encountered some bugs on iOS, but the support team has been very helpful.
Additionally, it doesn’t offer the ability to segment your user base based on events, only properties. For example, if you want to see if people who watched your explainer video convert better, you can’t do that as easily as you can with other tools.
It’s also annoying that I can’t set more than a 90-day window for the funnel.
Read more about how Mixpanel compares to Google Analytics.
Mouseflow offers a whole suite of mouse tracking tools: click maps, heat maps, scroll maps, recorded user sessions—you name it.
Optimizely is the leading A/B testing tool, by a pretty wide margin.
It’s easy to use, you don’t need to be technical to get small tests running, and their Stats Engine makes it harder for noobs to mess up tests.
Here’s a review of Optimizely from Stephen Pavlovich, CEO of Conversion.com:
- Performance and delivery: We switched to Optimizely back in 2012 because of their focus on performance. You can’t have a test that excessively slows down the load time, or a platform that doesn’t track accurately.
- Stats Engine: Optimizely led the way when they launched Stats Engine. They helped the industry move on from traditional—and often badly applied—statistical analysis. It’s been great to see other platforms following their lead.
- Integrations: It might not be a feature they shout about too often, but their tech partnerships are awesome. Being able to pull data into tools like Mixpanel can make a big difference to sophisticated testing programs.
- Developer support: Our development team loves Optimizely’s large documentation library, which makes it easy for new developers to get up to speed quickly. Tests are also simple to set up, especially compared to many other enterprise platforms.
- Stats Engine: Their Stats Engine is both a pro and a con for us. We like the emphasis on better statistical analysis, but the “black box” of Stats Engine can be frustrating at times. It can be hard to predict, so sometimes we rely on traditional (and properly applied) statistical analysis instead.
Alhan Keser from American Express also gave a short review on why he loves Optimizely:
We like Optimizely because it’s the most reliable tool for testing at high velocity. For strategists, experiment setup is easy to understand. And through its API, our development team can quickly build and QA a test with many variations.
Also, you can’t overlook the fact that Optimizely has great documentation and an active community making it better.
PageSense from Zoho offers several optimization tools under one roof to help you with your website conversions.
PageSense’s primary offerings include
- Goal tracking and funnel analysis to track key website metrics;
- Heat maps, form analytics, and session recordings to analyze visitor behavior;
- A/B Testing and split URL testing to optimize for conversions.
This is a comprehensive approach to the CRO process. All features also come with a segmentation facility for filtering reports, which is a quality addition to this bang-for-the-buck CRO software.
We were impressed by the ability of the heat mapping tool to capture data behind interactive elements, like hover-enabled mega menus.
With Pingdom’s Website Speed Test, you can test the load time of a page, analyze it, and find bottlenecks.
Pingdom’s Website Speed Test is one of many tools that Pingdom produces.
Here’s Peep Laja on the tool:
What makes this one different from other speed-measurement tools is that is shows you the waterfall report—how long each individual file takes to load.
That’s very handy for detecting culprits that slow down the site. I use it in conjunction with other speed-analysis tools.”
Sean Ellis started it, and it’s grown a lot, getting acquired by ProProfs in 2020. It’s great at targeting the right customers at the right time, and it’s easy to setup. It also integrates with common tools like Optimizely, Marketo, etc.
They’re onboarding is great. Nothing was confusing at all, unlike their competitors. The walk-through made everything clear.
Here’s a review from Andy Hunt, founder of UpliftROI:
Overall, it’s the best research tool in the market for quick, low-cost qualitative research. Many times, you can uncover insights quickly and cheaply with Qualaroo surveys rather than doing in-depth customer interviews.
- Pre-made templates: Comes with pre-made surveys for qualitative research. For example, “What’s the one thing that almost stopped you from buying from us?” These make it extremely fast and easy to get started.
- Regular updates: You’ll get notified each time you get a new response, as well as a summary of responses for that week. This makes staying on track of qualitative research easy.
- Low response rate: Expect 1–3% of visitors to complete a survey. To get 10–20 responses, you’re going to need to work on sites doing at least 100–200 conversions a month, or put surveys higher up the funnel if customers aren’t there yet.
- Mixed quality: Expect that ~20% of your responses will be low quality. (Customers won’t take these as seriously when they’re online vs. doing in-depth qualitative research.)
- Quarterly billing: Qualaroo charges on a quarterly rather than monthly basis. Normally this is okay, but it can be difficult if you need it only for a short-term project.”
Qualtrics is a powerful survey tool common among enterprises (and universities). The data collection and analysis is unmatched for large data sets.
Rob Balon, a market research consultant in Austin, has worked with tons of enterprises. Here are his thoughts:
This Provo-based company has been at the forefront of online survey–based research since 2002.
They have been instrumental in introducing new user-directed methodologies drawing from an accumulated panel of respondents. Qualtrics is a favorite of many CMOs who lease the software.
SessionCam is a popular usability tool that offers a range of capabilities, which center on session playback and mouse tracking.
Some users ignore the Funnels feature because everyone already has funnel visualization in GA. The difference is that SessionCam gives you a more granular view of where things go wrong.
Not only can you easily play back sessions related to a particular stage in the funnel, but it helps you to understand exactly how forms may have contributed to drop off rates. So you start with a holistic view and drill down from there.
Like other tools in this category, SessionCam also lets you do heat maps, scroll maps and mouse-movement maps.
My favorite part is that it integrates neatly with GA, UA, Optimizely, VWO, and a few other tools. This means you can examine in detail the difference in behavior between the control and variations in your experiments. If you treat your split tests primarily as learning opportunities, you’ll also be excited about this.
With prices starting at $1,500 per month, it’s unfortunately out of reach for many small businesses.
And a more opinionated review from Peep Laja:
SessionCam is an ugly workhorse—it gets the work done, but you don’t want to look at it. The usability and user experience are quite terrible. I used it with many clients, and every single time was frustrated by how cumbersome it was.
The data was good, but using the tool was a terrible experience. They weren’t very receptive to feedback either.
When you stack different mouse-tracking tools against each other, the fact that the SessionCam interface is stuck in the past wasn’t a deal breaker.
They offer out-of-the-box solutions that can be customized to your business needs; their support turnaround is outstanding; they treat their customers like partners; have integration with most of the big players; and offer a suite of customizable features to really help with session replays, funnels, and heat map analyses at a fraction of the price ClickTale offers.
SiteSpect is a testing tool that uses a proxy server, so it avoids many of the client-side problems and retains the robustness of server-side tools.
SiteSpect is one of the global leaders in testing and optimization. They are pioneers in the space, with a decade of experience. Their patented technology provides the ability to test and measure testing results without having the change code on your website or mobile experience.
The technology pattern matches patterns the customer defines and matches test content based on pattern recognition in real-time, with no latency and total failover. Features exist for speeding up the performance of tests and running many different tests simultaneously.
While SiteSpect used to require specialized hardware, the capability can now be deployed in the cloud or on-premises. Like other vendors, they have figured out the “flicker” problem and continue to innovate mobile testing.
Detractors of SiteSpect report that the product has technical complexity and isn’t something that can instantly used by a marketer without engineering support. It’s also a tool that requires people to understand what they are doing and plan for testing.
The reporting interface has been criticized but is widely regarded as industry standard. Finally, it’s not a tool for people who want to spend $49 or more per month. It’s powerful software for enterprises that are serious about testing and optimization and want technology that can accommodate enterprise requirements under an SLA.
SurveyGizmo offers easy-to-create online surveys.
They’re much like other tools, except that they have limited features on the free version. You can get basic stuff done, and the UI is great.
If you’re looking for advanced customization, you’ll have to pay.
SurveyMonkey is one of the most commonly used tools for creating and distributing surveys.
It’s really easy to set up surveys and equally effortless to analyze the data. I’ve also used this tool in the past for easy form submissions, such as nominating people for awards or taking surveys for research projects.
I’ve never had any complaints about it, but it’s never blown me away either—just got the job done.
Here’s a review from Rob Balon, market research consultant:
This is the idea of research made easy: Come up with a few questions, determine a sample size, and SurveyMonkey conducts a survey (from their own sample base) and the subsequent analysis.
Seems simple, but Survey Monkey does not pretest or validate customer questions. They will produce a survey, with the price depending on the number of questions and respondents.
The problem with this type of approach is that it allows customers with extremely limited experience to produce something that may look like a valid survey.
Careful with this!
TryMyUI lets you can have as many people as you want to be part of a usability test (when you recruit your own testers).
Typeform surveys are awesome and beautiful. It definitely has the best UX of any of the survey tools on this list.
It’s also really easy to drag and drop questions and design your survey. Overall, a great experience.
Here’s a review from Peep:
Typeform has the most beautiful forms, period. People don’t generally enjoy filling out surveys, but the user experience of using Typeform is delightful—and we’ve seen better response rates with them. The admin UI is pretty slick as well—kicks every other tool in this category strongly in the butt. Competitive pricing, too (including free).
UsabilityHub offers a suite of popular usability tools.
Michael Aagaard from Unbounce is a big fan of UsabilityHub and offered his opinion on a few of their tools:
UsabilityHub.com has an awesome suite of UX tools. I use these for Conversion Research and especially for pre-qualifying hypotheses before I decide whether to test them in the wild.
You can recruit your own panel for a private test via email invites, or you can use Usabilityhub’s own panel either randomly selected or based on demographics.
There are 3 tools I find especially helpful:
Users see a screenshot (e.g., homepage hero area) for 5 seconds and then have to answer questions afterward. This is a fantastic way to understand how users intuitively react to a specific page or piece of content.
I use this a lot for new landing pages stuff—show the first screenful of the landing page and ask users to answer: What do you think the page was about? What do you think the company sells?
The insight you get is fascinating and can help you identify fundamental problems that could be hurting your conversions.
With a click test, you upload a screenshot or a wireframe and ask people to click on a page element.
For example, I was doing research for a new pricing grid and ran a number of tests experimenting with how different visual cues affect behavior. I uploaded the grid wireframe and asked users to click on the plan that seemed to provide best value.
This is a very basic but super cool tool for getting insight from your customer base.
I recently ran a question test on a new headline treatment. I uploaded a screendump of two variations and asked for qualifying questions to get insight on how customers in different segments reacted to the messaging in the two variants.
UserTesting.com is probably the biggest marketplace to find testers.
Here’s a review by Claire Vo from Optimizely:
Way back in the dinosaur age (pre-2010), I used to run real, live, in-person usability studies. This meant sourcing a panel of customers—usually off a shady craigslist post—bringing them into your office, and having them walk through your web experience.
When (or if!) it worked, you could do screen and audio recording of your session, but that wasn’t always reliable, so you had to assign someone to take notes. By hand. For a startup, it was a terrible process but the best we could do.
UserTesting has done away with all of that nonsense and is an awesome way to get real user feedback from remote testers on your web experience. You just plug in your site, a couple tasks you want a user to try to accomplish, and get great, high-quality videos of users testing your site in less than a day.
It’s a great way to overcome your internal blindness or bias about what is and isn’t working on your site, and has provided great insights for every experience I’ve worked on.
- Cheap for a few sessions;
- Quick turn around;
- High-quality video and audio recordings of sessions;
- Works on desktop, mobile, and tablets;
- No shady craigslist postings and managing schedules, etc.
- Can feel expensive at scale;
- Unless you’re on the “pro” version, you can be matched with testers outside your target demographic;
- Feedback is anecdotal and qualitative, must be tempered with large-scale data.
VWO is a powerful, popular, and easy-to-use A/B testing tool.
Their “What You See Is What You Get” (WYSIWYG) editor makes it incredibly easy for marketers to set up tests without the help of developers.
VWO also has a stats engine based on Bayesian decisions called SmartStats. It’s supposed to make it easier to make business decisions and not mess up tests.
VWO is pretty all-inclusive. It also offers heat maps and click maps, as well as personalization tools and on-page surveys.
VWO is an excellent conversion testing platform. One of the main pros is the visual interface. In my opinion, I think the UX is more user friendly and the reports are more visually appealing. The reports and charts are easier to analyze than other testing platforms. Another good thing about VWO, is that is very easy for anyone to walk through creating a new test. It is a simple step by step process that I think makes more sense, especially to someone who is new to A/B testing.
Previously I would would have recommended another A/B testing tools if you were a developer, but VWO recently improved their code editor which in the past was really buggy and caused more headaches than needed.
The only downside that I have ran into is related to the WYSIWYG, which a good optimizer shouldn’t really use anyway.
In the past, when I didn’t use the code editor the WYSIWYG was really buggy. For example, I would create a test and if I didn’t make a copy of the code that was created, sometimes the variation would break and I would lose all my work. This may not be the case anymore since I only use the code editor now, but it was rather frustrating when I lost my work in the past.
Also in the past their support has a slow response time related to tickets created for support.
WebEngage is pretty neat because it includes notifications (like discount pop-ups), as well as surveys and the like.
Their free plan actually goes decently far if you’re a startup or smaller organization. It’s not as customizable as some other triggered pop-ups and surveys, but the simplicity is great, depending on your purposes.
WebPageTest can run site speed tests from a variety of locations.
It’s by far the most in-depth free tool—has charts of content breakdown, first view, repeat view, etc. Pretty neat, but not as easy to read as Google’s version.
Woopra is a real-time digital analytics tool that is easy to set up and start running.
It also offers some cool features, like real-time customer analytics, customer funnel analytics, event tracking, and cohort analysis.
Basically, it tracks users across digital touchpoints (website, mobile app, help desk, marketing automation, etc.) and builds a comprehensive behavioral profile for each user. This makes it really helpful for startups.
Woopra also tracks anonymous website and mobile app users from their first touch until they identify themselves. A con is that its integration with other tools is limited, especially when compared to universal tools like GA.
Here’s Peep’s review:
If you have forms, you need to analyze their performance. Zuko does a very good job at providing in-depth information that really helps with optimization efforts—useful data, simple interface.
Its form analytics actually work, especially when compared to some mouse-tracking tools that have form analytics as one of many features (and often fail to work).
Learn how to run conversion optimization experiments the right way. In this video, I sit…
Testing in an enterprise is truly a team sport. If a testing program was a…
Conversion optimization is hard; it’s constantly changing and you need to know a lot about…
In this 2016 State of Conversion Optimization Report, we gave a 48 question survey to…