Key Insights from Every Speaker of Elite Camp 2017

Elite Camp is a traffic and (mainly) conversion event held in Estonia every summer. It’s among the very best CRO events in the world, and of course in Europe.

Elite Camp 2017 had an enviable line-up of heavy hitters and rock stars. Here are top insights from every speaker of this year’s event.

Thanks to the inimitable Karl Gilis for all the photos below. Take his course on UX and Usability here.

Day 1

Peep Laja: Opening Keynote

• Dunning–Kruger effect is a cognitive bias where someone is so incompetent that they don’t know that they are incompetent, they think they are tremendous.
• A/B testing is a great way to see that sometimes the changes you think will work don’t work at all.
• Websites are contextual. Websites selling different things are different and even if they sell the same products, their target audience might be different.
• Research is the key to figuring out which best practices to use.
• Bottom line – we don’t know what’s the best solution. We can’t predict it. So we want to come up with a bunch of radically different ideas & test them.
  • Treat each idea as fungible – because we don’t know what works what doesn’t.
• The bigger the change – the higher the likelihood that something will happen. Bigger changes are more likely to change user behavior.
  • Example – you probably shouldn’t test adding a period at the end of the sentence.

Morgan Brown: Building a Growth Organization

• Organize growth into a cross-functional team.
• Build your team – make sure they are focused on driving growth through testing.
• Love your Data – Track full funnels, and fix your data! Data is often fragmented, so fixing it can’t be emphasized enough.
• Know your North Star Metric. What’s your core value? Optimize for that. (Hint: it’s not usually a company-valued metric like revenue. It’s usually something like Daily Active Users, or something that signifies user value).
• Prioritize ruthlessly.
• Move fast, and learn things.
• Aim for impact. Test big things, create real change.
• Never stop moving. Growth is never done.

Daria Nepriakhina: Problem-Solution Fit Canvas

• If you don’t know your customer well, 99% of your assumptions are wrong.
  • Are we optimizing the right thing?
  • Do people care about this product?
• The world we live in, we’re constantly on the move, distracted – it’s really hard to stay focused.
  • So how to target a customer in this world?
  • We’re over-triggered, constantly bombarded by brands.
    • You need to be/stay relevant
• Things change:
  • Customer state changes.
  • Situation changes.
  • Market shifts.
  • Perception of your brand changes.
• Understand your customer and the existing experience to design for improvement of people’s lives.
• If you don’t know your customer well, then 99% of your assumptions are wrong. – You can simply ask your customers and talk to them.
• Problem-solution canvas (source)
  • Problem
  • Behavior
  • Awareness
  • Customer State.

Sujan Patel: Content As A Growth Engine

Stop creating mediocre content and start creating epic content.

11 epic content marketing tactics:

• Create epic content. Focus on quality not quantity.
  • All that matters is that it’s better than your best competitor.
• Content Circles. broaden your topic scope.
  • Example: Employee engagement -> small business -> productivity -> Management -> Culture -> Human resources
• Epic content + Content Circles.
• Use your best content and Epic content to nurture & onboard new customers.
  • Just because you create for the top of the funnel, don’t assume that people actually see it.
• Guest Post + SEO:
  • Make a list of the top sites in the industry.
  • Use SEMrush to research what keywords they rank for.
  • Write s guest post based on related keyword + topic that benefits your business.
• Invite influencers to guest post on your blog. You don’t have to be the author be the source.
• Create “behind the scenes” content for bottom of the funnel. Show the work, document and share what you do.
• Interview customers and tell their success story (with or without your product/service) – How are they successful? Advocate the cause, not the tool!
• Survey your customers or audience about their top problems & create an ebook about solutions – What to they do every single day?
• Solve their REAL problems! This works for both top and bottom of the funnel content.
• Repurpose your best content into webinars, videos, micro-sites & ebooks People learn differently, so reach more people!
• Teach your sales & customer support team to send helpful content from the blog. Sales/support follow up with useful content. Leaves a good taste in the customers mouth.

Bob Ruffalo: How IMPACT Used the ResearchXL Model to 3x Conversions from Content Marketing

ResearchXL for content

• Heuristic analysis:
  • Friction – above the fold is too big.
  • Clarity – there is no meta description.
  • Distraction – are people seeing our content.
  • Value – Why is Ramona the only writer?
  • Relevancy – How are blogs categorized?
• Technical analysis – Crush all the bugs on your site:
  • Check Analytics – identify issues, cross browser & cross device testing
    • audience → technology → browser/OS report
  • Speed analysis -> Hubspot
• Web analytics analysis:
  • Most popular pages – optimize these!
  • Leaks on your site
• Mouse tracking analysis:
  • Heatmaps – where do people click?
  • Scrollmaps – How low do customers scroll?
  • Session replays – see actual user interaction
• Qualitative surveys: get answers to Why
  • On-page questions “Why you are not subscribing” – Why did they subscribe?”
    • Can be done using tools like Qualaroo, Hotjar etc
    • Track responses using Hubspot!
• User testing: observe interactions with your site – think out loud testing
• Takeaways:
  • Listen to Peep and CXL
  • Implement Agile!

Els Aerts: Advanced User Research – Right Time, Right Place, Right Method

•  Warning: A focus group is not a user test
  • Only good for generating ideas
  • Not good for identifying usability issues
• Unmoderated user testing – there is no interaction with the tester.
  • The questions you ask have to be crystal clear.
  • Can test with a lot of people in the same time.
  • Look for patterns, not yet statistical significance.
• What to choose:
  • Depends on what is your research question (what’s the problem)?
• Types of moderated user testing:
  • Two room setup – Testing room (moderator+subjects), observation room (your team, observing). Record everything.
    • You don’t need a two way mirror.
    • You can test in their office – It might be the first time they see somebody use their site/product.
    • Eye tracking – helps understand what element they are talking about (the one they are looking at).
    • Guerrilla user testing – in the wild user testing.
      • Great if context of use is important (Doctors at hospital)
  • Remote moderated testing – basically a web meeting, record their desktop & webcam
    • Facial expressions don’t lie
• Tree testing (Treejack) – Data structuring
• Screenshot testing/ first click testing (Chalkmark, Usabilla)
• To test for immediate clarity, use Five Second Test. (ask only simple easy questions – that should be clear in the first 5 sec.)

Christoph Lütke Schelhowe: Data for everyone?

• Goal: bring data-informed decision making to the next level in Zalando.
• Hybrid Product analytics organization:
  • Central teams for tools, methods, governance.
  • Team -embedded product analysts to empower everyone.
• Self-service tooling: make data access and experimentation quick and easy for EVERYBODY.
• Data education – Train your team:
  • Bootcamps for experts on all levels.
  • Self-learning videos.
  • Community of practice.
• Data governance – hated but necessary:
  • Rules of play.
  • Data event overview.
  • Data quality standards and monitoring.
  • Example: was not clear who uses what tracking events – app feature failure.
• Their recent learnings:
  • Spark a fire: Teams intrinsically get into the flow of iterating optimization cycles when seeing the first encouraging results.
  • Have the right balance between small + big ideas: Don’t only do front-end testing, think about bigger experimentation use cases that involve backend testing.
  • Think beyond data: Use qualitative research to get a the why. The value is in the insights behind data.

Kristoffer Ewald: Modern Measurement Strategies

• 11 years ago, online space management:
  • Based on analytics
  • Optimize the average value of a visit
  • Therefore the ROI on online investments
• Now, it’s about real time understanding/action
  • Powered by AI – machine learning.
  • Data is everywhere.
  • Data is silo’d.
  • Data is not used in context – you need to put it into context.
  • Complexity of context is increasing.
  • Integration is the core. Get closer to the customer – what do people do at a physical location?
• Collect → sync → wrangle → analyze and predict

Jeff Sauer: Attribution is Bullshit

• Is attribution worth your time?
  • Less than 100-1000 conversions: No.
  • More than 100-1000 conversions: Maybe.
  • More than 10,000 conversions as well as multi-channel distribution: Yes!
• Attribution is hard.
• Complexity of attribution strategy = complexity of marketing strategy.
• It is not for everybody.
• Test all the models! See what model fits you best. You can do this for free in Google analytics. Don’t just passively accept the numbers.

Day 2

André Morys: The Magic of Combining Neuromarketing & Data Sciences

• How digital transformation works:
  • Strategy+engagement+innovation+technology+data&analytics = digital transformation.
  • But where is the customer? Don’t ignore real people.
  • “Start with the customer and work backwards”
• If you want to sell something, you need to make people feel like this is made for them.
  • Relevance -> Resonance: “I love it. This is made for me!”
• Search for the connection between insights and data.
• What you should do:
  • Develop customer types based on personas.
  • Analyze your existing Customer Data and build clusters.
  • Validate existing customer types online.

Nicolas Visiers: User Experience Testing – Practical cases

• Makes sure decisions are made, and FAST.
• Get everybody on board and make sure they are supportive of the team. Have a single goal, go all in.
  • Everybody has a vote
• Try to fail fast, recover faster
• The best learning is experiential – change people.
• Emphasize diversity, team dynamics, make sure people are not “getting bored.”
• Align goals, KPI’s, bonuses – align with optimization goals.
• Spend time and money in making sure your team is engaged.
• Your team will drive more value than your tool: 20%/80%.
• Value = ( knowledge + Skill ) x Attitude.
  • Attitude makes a difference

Manuel da Costa: The Shocking Truth About CRO Programmes

Common pitfalls that can derail your optimization program:

• #1 Lack of knowledge. CRO has only been popular 4-5 years, so there’s a knowledge gap.
• #2 Not Structured process. Find a framework that works:
  • Define business goals
  • Collect data
  • Analyze
  • Hypothesis
  • Design
  • Build
  • Test
  • Learning / improving
  • Models – WiderFunnel Infinity framework, ResearchXL
• #3 No Buy In – Lose the us vs them mentality with the stakeholders
  • Everyone has their own agenda
  • Break down internal silos / departments
• #4 It’s a Team Sport. So engage & educate.
  • Teach CRO through examples
  • Hold a testing hackathon
  • Get people to vote on ideas
  • Display results & leaderboards
  • Always provide visibility of your work
• #5 Lack of Resources – 57% companies struggle with resources
  • Bottlenecks – identify them!
  • Get external partners – 3rd parties or freelancers – to help scale.
• #6 The Current Approach is Broken
  • Don’t start with tools
  • Conversion optimization tools list here.
  • People, Process, Priority – the 3 pillars of a good program
• Optimization is not a project – Don’t time box it – It’s business as usual – Don’t stop optimizing

Manuel has a course starting August 15 for CXL Institute.

Raffaela Rein: Becoming a UX-led Company

• It’s not technology itself that disrupts the world, the industries do it to themselves.
  • “Amazon did not kill the retail industry, It did it to itself with bad customer service” – Alberto Brea
• Don’t make it about the features or the product – make it about the customer
• Insights from 60+ UX experts:
  • Build case studies to influence stakeholders – show the numbers.
  • CRO needs to more customer centric (20% Customers use voice search).
  • User experience need to come to the executive table.
  • It’s a journey to become UX lead.
  • Check out their courses.

Stephen Pavlovich: Testing Beyond User Experience

• Companies invest huge amounts of money in products the customers do not want.
  • Success is a result of the decisions we make.
• The advantage of experimentations is to make multiple choices and validate them.
• Testing was the same a 100 years ago – Stephen gave an example of a Pepsodent newspaper ad
• Using experimentation to solve complex problems is the gold standard.
• Experimentation lets us challenge assumptions with radical concepts.
  • Nothing is new. Everything is changing.
• If we can hypothesize, analyze and iterate, we can experiment.
• In a fast changing world experimentation becomes an unfair advantage.

Unn Swanstron: News Media CRO

4 favorite experiment tips:

• Setting design limitations – creativity is weird
  • Put your idea in different scenarios
  • Start with the data you have and ask questions
  • Example: opened a recommendation box when people scrolled up on an article. So place your recommendations where people can actually see them.
• Always be iterating – even on the things that are really good – Find smart solutions
  • Mailto button – is super useful on mobile. Example with instant subscribe on mobile where to subscribe people have to send one email and once they do they are thrown back to the original page.
• Do we have the right KPI’s?
  • Common KPI’s : Page Views, Unique browsers, Number of visitors – they ignore the confused, angry customers – NOT A GOOD IDEA
  • Time spent is another bad KPI to follow
  • Don’t be blind with one metric
  • It’s your job to test Usability & Accessibility
• How soon can we have a proof of concept? – not time to market.
  • Iteration example: Making a personal app
    • Crude scenario/sketch
    • Clickable prototype
    • Test with users
      • If you get sad – you are learning
      • You have to kill your darlings
    • Test with low cost? Maybe inside your team?

Guido Jansen: How to Involve the Whole Team in the Optimization Process

• When starting, get shit done. Start proving the stuff you do.
• Show value. Things don’t need to be perfect, but you need to prove business value (specifically in monetary value if possible.)
• It’s ok to get people mad in the beginning.
• RegEx challenge. challenging people to be interested
• Things that work:
  • Explain all research/experiment learnings – over and over again
  • Take care of your shit umbrella – The one who keeps the managers/investors away so you can continue to work.
  • Transparent workflow tool. – they use Effective Experiments.
  • Shared cross-team KPIs – North Star Metric
  • Calculate ROI in MONEY – estimations, super hard to reach a one true answer
    • “There is 2% uplift in Sweden” – boring.
    • “We made 100,000€ in Sweden” – Aww yeah!

Erin Weigel: Concept != Execution

• Failure rate 9/10
  • “We have optimized the S*** out of our site”
  • Strict on statistics.
• When a test doesn’t work, check the following:
  • Check the technical implementation. There are often issues here.
  • Page load time.
  • Noisy tracking.
  • Low traffic.
  • Poor design.
  • Usability issue.
  • Ahead of your time?
• Page load time matters a ton. The invisible elements impact UX just as much as the visible.
• Check for: unintentional increases in page load time…
  • Are custom web fonts necessary?
  • Big hero images?
  • People want more information, but they don’t want to wait for it.

Pauline Marol: From Fantasy to Reality – How to do AB Testing in a Small Company

So how do you do A/B testing with low traffic?

• Invest in training.
  • CXL Institute.
• Work harder on your homework.
  • Don’t test any random ideas.
  • Listen to the voice of customers:
    • Customer service.
    • NPS feedback.
    • Sales.
  • Basic analytics:
    • Funnel.
    • Exit pages.
    • Form analysis – check out Formisimo for this.
    • Event tracking.
• Focus on micro-conversion:
  • Next page path.
  • Pages per visit.
  • Exit pages.
  • Clicks.
  • Traffic sources.
• Consider low statistical significance.
• Don’t be tempted by multivariate testing.
• Test only your top pages.
• Test radical changes.
• Keep the process simple.

Angie Schottmuller: Hybrid Hacks – Deep Growth Insights for Multiple Disciplines

• There’s often little correlation with success and hard work… But lots of with solving other people’s problems.
• First questions when starting a marketing campaign?
  • What’s the problem? (This should always be the first question)
  • Who’s the customer?
  • What’s your product?
• Look for Hybrid Hacks.
  • Clever solutions to tricky problems. Hacks that are:
    • Applied to another marketing channel.
    • Merged with non-marketing channel.
    • Applied to a different context.
• How to do Hybrid Hacking:
  • Identify patterns
  • Assess problems
  • Brainstorm creative solutions
    • Common goals
    • Different context
    • “Connectable dots”

Conclusion

Elite Camp was an all-around amazing event – quality networking, cutting-edge learning, and a great venue. Come next year!

Also, sign up for updates about next year’s CXL Live