5 Hardest Things in Conversion Optimization

5 Hardest Things in Conversion Optimization

Conversion optimization is hard; it’s constantly changing and you need to know a lot about a lot. Keeping up with the technology changes and managing business’ expectations can be tricky.

Here’s what some of the top experts in the field are saying about the top challenges in conversion optimization.

#1 Having to be an expert in many things.

CRO is multidisciplinary and cross-functional. Conversion optimization specialists must be “polymaths,” having a working background in at least 2-3 disciplines (and being good enough at others).

Optimization polymaths often come from a variety of backgrounds, including IT, design, UX, analytics, marketing, translation, photography, etc.

Chad Sanderson, a conversion optimization, experimentation, and personalization specialist for Subway, argues that developing a system that includes the many aspects of optimization is one of the biggest challenges.

Chad Sanderson, Subway

“Imagine you’re a heart surgeon at a world-renowned hospital. Usually, you’d be brought in at the end of a lengthy series of consultations if (and only if) all the medical experts on staff agree that a heart operation is necessary.

However, due to recent budget cuts, it falls on you to diagnose the patient, conduct medical tests, prescribe antibiotics, find a suitable donor, perform the surgery, and conduct the follow up exams. Now imagine repeating this ordeal a hundred times a month.
CRO’s are faced with a similar dilemma…

It’s not enough to have a general understanding of Statistics, Development, UX Design, or Analytics. In order to run a successful program you must be extremely capable in each of these areas (and many more) or face potentially disastrous consequences.”

#2 The industry is constantly changing

As technology evolves, so must our tests.  This is especially true as mobile engagement via smartphone has skyrocketed. In 2017, more than half of the world’s web traffic now comes from mobile phones.

However, mobile is a relatively new domain; not every company has caught up.  Catching up is the trick to getting to most out of your consumers, and the only way to do that is by having your finger on the pulse of the industry, reading new research, and conducting your own experiments.

Angie Schottmuller, Interactive Artisan

“Balance is the toughest challenge with conversion optimization. How much science versus art? How much change versus consistency? How much structure versus agile process? How much training versus tactics? Optimization is challenging pursuit that demands perpetual change. To be successful, it’s imperative to find a balance ‘center point of gravity’ for yourself, your team, and your customer.”

Matt Roach agrees:

Matt Roach, Sanoma

“Unlike other jobs (accounting, law, etc.) where once you know it, you know it, with CRO it is a constant challenge – and requires significant investment of time and effort – to stay up to date with your knowledge […]

Every business is different and presents unique challenges – what works for one business may be a disaster for another. This makes it harder to take shortcuts, to simply apply what you have done elsewhere.”

So, in order to remain competitive, you must watch trends, watch your peers, and prioritize the data.

#3 You need to challenge your own beliefs and expertise

Oftentimes, experimental data counteracts our original assumptions or changes over time.  Even years of experience in the field of analytics can’t always help CROs predict the outcome of tests or forecast future trends.

Els Aerts says to check the arrogance of experience:

Els Aerts, AGConsult

“It’s important to keep your own biases in check. And to always, always do the research. It’s very tempting sometimes to think you know what is wrong with a page and how to fix it. But that’s just not how it works. You need to dive into the data to pinpoint the real opportunities.”

CRO is an inward-looking discipline, Andrew Anderson suggests, requiring objective knowledge of probability theory, economics, sociology, statistics, and psychology, just to know where to focus in the first place.

Andrew Anderson, Recovery Brands

“My life would be so much easier if all I did was try to validate other people’s ideas instead of putting through a system that compares them to as large a pool as possible […]

It isn’t about what I think matters, or what anyone thinks matters, it is about the building of a disciplined system that takes all inputs, maximizes the beta of them, and then lets the system spit out what wins.”

Ego shouldn’t even come into play.

Nick So, WiderFunnel.com

“One of the most difficult things to master when you’re doing conversion optimization is the ability to be objective and humble when facing ‘losing experiments.’

Sometimes, you’ll test a hypothesis that seems brilliant or like plain common sense, or you’ll test a design that seems like a vast improvement over the original experience, and just can’t lose… only to see these variations lose, or worse, show inconclusive results.

You have to put ego aside when you’re testing. You have to be able to look at the data and results objectively, from a scientific perspective. […] If you can do that, you can turn those losing experiments into winning experiments.”

André Morys says that not only is it important to remove your ego, it’s essential to keep your goals customer-centric:

André Morys, Web Arts

“Each uplift is a result of a behavior change of the website users. Most optimizers forget about that and start tweaking buttons and templates. That’s not our job. Our job is to change websites to influence user behavior. If you forget that, your A/B tests are just ‘trial and error,” but not optimization. [The hardest thing is] to stay really customer-centric.”

#4 Resistant organizational mindsets

The sticking point is not just with individual optimizers, but with the companies they are hired to work for professionally.

Bryan Eisenberg, NY Times bestselling author

“[The struggle is] to transform the organization looking for a short-term boost to a learning organization looking for long-term innovation and sustained growth.”

Changing a company’s cultural lifestyle requires examining foundational beliefs about CRO, and honestly, that’s difficult in a large scale organization. In many places, politicking is king.

Paul Rouke, PRWD

“The biggest barriers are often driven by the type of leadership and organization has – so often egotistical and opinion-driven, with little if any true humility to truly respect and value the ideas of employees, and most often little if any perceived value in speaking to, listening to, and understanding customers.

[…] Instilling more humility within businesses, and slowly removing egotism, presents a huge challenge for conversion optimization – but the greater the challenge, the greater the opportunity to have a material positive impact on both the commercial performance of the business and employee satisfaction and enjoyment.”

In our 2017 State of the Industry Report, we interviewed 333 conversion rate optimization professionals in a 26-question survey. One of the resounding responses to industry challenges was a lack of structural support and dedicated budgets for conversion rate optimization.

Brian Massey agrees that corporate politics are at fault for some of the budgetary issues, and that top decision-makers in the company aren’t always educated in how to interpret data in a meaningful way.

Brian Massey, Conversion Sciences

“We have a name for this leadership style: “HiPPO,” or Highest Paid Person’s Opinion. Joel Harvey calls it Helicopter Management.

This is the management style of charismatic or autocratic leaders who drive action in their organizations by helicoptering in, expressing a lightly-informed opinion, and enforcing their opinion in one of the following two ways: they bestow budget upon the loyal, [and] they threaten the jobs of the disloyal.”

It’s a vicious cycle. In order to get valuable results, optimizers need a budget. However, optimizers don’t get any money because the value of the results they do get isn’t evident to the leadership looking at the numbers.

Renee Thompson explains that iterative testing is the way to realize the most gains, but if a “win” is realized initially, many times people think that’s good enough and don’t think they need to keep pushing.

Renee Thompson, TechTarget

“ […] There are many challenges, but one that stands out is managing expectations – of stakeholders and management. Everyone wants big wins and they want them quickly.

Sometimes in order to justify resources or prioritization, an expectation of results is required, and if it isn’t perceived as big enough, it’s difficult to move forward.”

Without that initial investment, growth is stalled, because marketing without data is like driving with your eyes closed.

Andre Morys says that 80–90% of big companies do not aim for bigger goals, and certainly don’t have enough staff assigned to work towards them.  This more short-term solution presents a problem for an industry that is designed to forecast the future.

#5 Employees have limited CRO knowledge

Without employees with experience in data management strategy, it’s hard to measure success, especially if they don’t know what metrics matter.

About 58% of CRO professionals have been in the field for 2 years or fewer.

In fact, around 30% of companies in our survey only had one dedicated employee assigned to the task.

Tim Stewart, TRS Digital

“You can’t add it to the responsibility of a junior analyst, give them no resource or time to develop and execute a roadmap, and then expect to out-perform targets because you have ticked a conversion optimization box.

If you believe that, I have some magic beans to swap for your prize cow and plenty of others happy to sell you oil extracted from reptiles with no legs and forked tongues.”

Therein lies the challenge.  CRO is seen as optional, not necessary, and because of this, time and energy aren’t invested in employing people with experience in CRO.

Organizational pushback happens because companies don’t understand the quantifiable value of conversion rate optimization data, and don’t want to expend resources on employing CRO specialist unless they see the immediate value.

Tim Stewart, TRS Digital

“Optimization is something that is frequently oversold as to what is realistic at low maturity but is also undersold on the real benefit from the longer-term potential of a mature testing culture […]

If you have a massive, impossible task like [incorporating CRO] in front of you, don’t try to swallow it all at once. Start with one wafer-thin slice and keep going.

Because if you don’t start, it’ll still be there, taunting you with the massive opportunity cost and risk mitigation you have not managed to swallow.”

The key is to recruit talented, persuasive employees who can work within this framework and shed light on the value of data analysis to marketing efforts and website optimization.


Conversion optimization is a challenging industry, and the specific challenges for optimizers often can be:

  • Having to be an expert in many things.
  • The industry is constantly changing.
  • Data can be challenging to your own beliefs and expertise.
  • Resistant organizational mindsets.
  • Employees have limited CRO knowledge.

The key is keeping letting your data guide you, and finding skilled employees who can persuasively convey the value of CRO work to leadership.

Related Posts

Current article:

5 Hardest Things in Conversion Optimization