CXL LIVE 2020

This session focuses on 10 marketing principles that will unlock new growth for your business by focusing on the one thing that hasn't changed in thousands of years: people. Each lesson in this talk is practical with real examples from some of today's best marketers.

After this session, you'll be able to:

• Make faster, better decisions about new marketing strategies and campaigns without having to get buy-in from everyone in your company.
• Launch new campaigns with confidence knowing that they will succeed because you learned how to test ideas quickly and without spending a dollar.
• Use the framework Steve Jobs used for his keynotes to tell stories that attract customers to your business like a magnet.
• Write more case studies than you can keep up with by tapping into customers and superfans on social media and use their words to drive more sales conversations for your business.

Online controlled experiments have become ubiquitous in evaluating the impact of changes made to software products and services. To understand the top practical challenges, representatives with experience in large-scale experimentation from thirteen different organizations (Airbnb, Amazon, Booking.com, Facebook, Google, LinkedIn, Lyft, Microsoft, Netflix, Twitter, Uber, Yandex, and Stanford University) were invited to the first Practical Online Controlled Experiments Summit. All thirteen organizations sent representatives. Together these organizations tested more than one hundred thousand experiment treatments last year. Thirty-four experts from these organizations participated in the summit. The talk will summarize the top challenges faced across the industry and some common solutions.

Experimentation is the most rigorous way to answer business questions. After rolling out a series of positive test results we should see a meaningful improvement to the bottom line—but that’s often not the case. Ever wonder why? A stringent experiment implementation is only one piece of the puzzle; we can still fool ourselves when it’s time to analyze the results. Choosing which response metrics to track, as well as when and how to measure them, is just as important. This talk will discuss measurement lessons learned while building an experimentation program at Shopify, including common pitfalls to avoid and best practices to get more accurate results from your own experiments.

In online controlled experimentation, it is imperative that we conduct analyses in trustworthy ways and produce credible results for decision making. One aspect of such effort, rarely discussed in the A/B testing literature, is outlier detection.

In this talk, we share empirical evidence that extreme outliers can indeed affect A/B testing. To address this issue, we propose a conceptually intuitive and computationally efficient outlier detection approach, and demonstrate its performance by simulated and real-life examples. I also share best practices in outlier detection in industrial A/B testing, in particular how to transform the learnings to actionable business insights.

With initiatives like Intelligent Tracking Prevention (Safari/WebKit) and Enhanced Tracking Protection (Firefox/Mozilla), persisting things like analytics and experiment identifiers in the browser has become difficult to do reliably. Well, truthfully, it’s never been reliable to begin with, but now there’s a concentrated effort by browsers to make client-side state intentionally fragile.

In this talk, Simo will explore the impact of these changes on your data collection. Particularly sensitive areas are web analytics data, but similarly experiment and personalization identifiers are difficult to persist reliably, especially if the site does not have an authentication system in place.

Simo has some solutions for you as well, but don’t be lulled into thinking there’s a turnkey solution out there that will bring instant gratification. We’re looking at a new paradigm of constant evaluation of state-management mechanisms, and without an agile workflow in place, there’s no tool in the world to help you navigate the turbulent waters of the client-side web.

Aligning with Hilton’s customer-centric product development philosophy, Paula and her team ensure feedback mechanisms always put the guest first. Her team advocates for the customer from exploratory and discovery work through to measurement of products at scale. In this discussion, Paula will share more about how Hilton delivers best-in-class customer experiences, using Human Centered Design principles powered by research.

• Collecting feedback through human centered design principles to pioneer exceptional customer experiences
• The marriage of qualitative and quantitative data to drive optimal outcomes
• Case studies of the good, the bad, and the ugly
• Practical takeaways: how you can do this, too

• Learn how context can be a strategy for asking people the right questions
• CXL's playbook for gathering voice of customer data specific to mobile
• Methods to get qualitative data on customers perceptions of UX, friction, motivation, and uncertainties
• Learn how to prioritize the VoC research findings and create a process for testing and action

This is your chance to explain your business challenge to speakers, and get their advice for solving it.

Have you ever thought about the ROI state of your optimization program? Do you know if you should:

1) Increase the budget for A/B-testing?
     or
2) Invest in more knowledge for teams that do A/B-testing?
     or
3) Decrease the budget for A/B-testing?

An analyst can properly calculate the answer to this question. In this presentation Ton will show you how.

Do you also know that an analyst can have a really big influence on the outcome of the answer to the investment question above?

Ton will show you how an analyst can make a big impact on the number of winning A/B-tests.

The days of "Launch and See" are over. No longer will we exhaust our budgets to find out if a campaign is going to deliver. Collecting behavioral data is easy, cheap, and fast. Brian Massey will take you through the modern digital design process, showing you where simple experiments will ensure that your campaigns go out the door with high conversion rates and fast ROI. We're going to cover a lot of very important ground for digital marketers and business owners.

Introducing the growth hacking concept and a culture of experimentation is hard anywhere; let alone at one of the oldest and largest tech companies in the world. Hear from Jason Barbato, one of the chief evangelists and lead strategists in building and scaling a growth hacking stack and best-in-class enablement program applied by 200+ users, and stories of some of the multi-million dollar experiments and milestones that have helped transform digital strategy and improve product experiences across IBM.

Over the last year, intent data providers have been cropping up left and right, but the overwhelming majority of demand generation marketers have been slow to appreciate the hard, ROI-driving power of intent data. In other words, the first-mover advantage is still yours for the taking! The only question is: which provider is best for you? And growth marketing expert Guillaume Cabane can help you there. Drawing from his experiences at Segment and Drift, where he bought over 10 intent data providers, he’ll give you first-hand, insider knowledge on what features to look for, how much to budget for them, and how to make the most of the overall intent data market shift in this practical, tell-all session.

In this talk, Russell will go through what data we can use to forecast the future and understand much more about how our future customers will engage with us, how we can find more of them and get the best value from their engagements. Forecasting the future isn't just possible - its essential to the modern marketer!

When tools and systems are not easy to use, people lose trust and understanding. They lose their bearings and end up relying on other methods to make decisions. It doesn't have to be this way. We can create empathetic, intuitive systems for sharing information.

• How to choose accessible tools
• Creating intuitive events and reports
• Collaborative practices to ensure trustworthy data and data analysis

Common wisdom about what motivates customers, builds loyalty, and encourages positive word of mouth is wrong. Drawing on the ideas in his award-winning book, Friction, and his best-selling Brainfluence, Roger Dooley will share the data that shows what really drives positive outcomes and show you practical ways to get there. This keynote will be fast-paced, entertaining, and full of examples.