9 Lessons Learned from Running a CRO Agency

9 Lessons Learned from Running a CRO Agency

Run an agency? Offer conversion optimization services? Then you know it can be hard.

I’ve been running an optimization agency for 6 years now. Here are my top lessons I’ve learned along the way.

1. Build a name, build an audience

It’s so much easier to run an agency business if you have a personal brand name and an audience. Think about it: you’re in the business of expertise. You’re not in the business of selling services, per se. Clients don’t pay you for your time, they pay for your expertise.

So when a potential client thinks, “I need someone to come in and work on my site,” who will they respond to better: the one with a big audience and lots of social proof or someone that no one has heard of?

All things equal, there’s less friction and uncertainty in going with the one that has the brand name.

Cold outreach can work in getting clients, but in my experience, warm is 10x better. If leads come to you, then the sales call that follows is now about qualification. You’re in the position of power. It makes a massive difference.

If you have to sell them the idea that they should do CRO and that they should hire you, there’s way more friction.

How do you build an audience? Today, there are so many ways, and the channel isn’t the important part. There’s blogging – my top choice, but also podcasting, YouTube, Snapchat, Instagram, etc. You can also get creative and organize communities online. It’s easy to do that now with Facebook groups or Slack groups.

Though, blogging has the benefit of SEO, which lasts a lot longer than other channels.

If you’re only working in a local market, you can easily host meetups and build events around what you do. Speaking at events works as well, and there are many that allow newer speakers, like Measure Camp, Product Camp, and local marketing meetups.

Relying only on referrals is a tough life. You’re not in control of the inputs or the outputs when you only rely on referrals for business, so if referrals stop coming, then what can you do? My advice: referrals are great, but have another lead generation channel as well.

If you don’t know where to start building a name for yourself, guest post for the CXL blog.

2. Getting results is hard. Competition is intense.

If you read a lot of hacky blog posts, you might think this is easy. Change this button color, make it bigger, change your headline – 200% increase!

It reality, it’s hard work.

When running an optimization program, consider three metrics that contribute to success:

  • The number of experiments you run.
  • The percentage of winning tests.
  • The average uplift of successful experiments.

The first lever is all about resources and politics. How much traffic do you have to work with? How many designers and developers do you have working on the problem? How fast does the client allow you to work?

These things are fairly limited in terms of their ceiling. Once you max out on tests given traffic allotment, you can’t really push through more just because you want to.

However, the second two levers rely on expertise, and that’s where you can really bring value.

Competition is tough, and getting harder all the time. You might think your competitors are not as smart as you are, but bear in mind that your client might not know this. Your client hears the same pitch from everyone, so how do they know you’ll deliver better results? And if you can’t deliver, know that you’ll be fired very fast.

In order to win in this business, you need to build your brand, but outside of that: Have the damn expertise.

If you’re just winging it – “oh, I read a blog post or two so I know what I’m doing” – then you’re going to get fired. Sure, everyone has a bad month. Even the best agencies do. But word travels, and if you’re consistently bringing terrible results, you won’t be in business long.

The best agencies invest in learning at all levels of their team, and they don’t stop learning once they hit a basic threshold of knowledge.

3. Work with bigger businesses

Conversion optimization doesn’t align with small business. For one, there are inherent limitations with traffic. If you don’t have the traffic to test, there’s not much we can do.

Second, you can’t bring as much ROI to small businesses.

Here’s a quick exercise to prove this. When taking on a potential client, ask yourself, “If we improve their sales by 1%, what does that mean to their business?”

If it’s a company that does $1M in revenue per year, and we increase sales by 1%, that’s $10,000 extra revenue (who knows about profit?). Maybe you can get more than one percent, but how long does it take you to get that lift? How much do you charge per month? The ROI is really hard to achieve with this size of business.

You may think, “oh hey, I charge $200 a month!” Well, A) you’re an idiot, B) No company that makes over $1M a year is going to hire someone that charges $200 to do CRO. It’s just not the way the world works.

But if your company does $1B a year in annual sales, you’re going to achieve a crazy ROI with only a 1% increase. And with that crazy ROI comes happier clients, better testimonials, and more referrals.

As a ballpark, try not to do business with companies that do less than $10M a year in online revenue. Of course, there are exceptions, but that’s a good heuristic to follow.

In my experience, one optimization project manager can work with a maximum of 3-4 clients without sacrificing quality. Getting results is hard in the first place, so if you don’t have the focus, you won’t get results.

Also, think about it from a business perspective. There’s not much margin on an agency business anyway, so how much do you think these clients should pay you for an engagement, given the assumption of 3-4 clients per project manager?

Whether you’re working for a mom and pop shop with low revenue or a $500M a year ecommerce giant, the work you put into get results is the same. It’s just that for one of these, they can pay you more and they receive more ROI for your efforts. Why waste your life with the more stressful and frankly less valuable of the two types of clients?

4. Differentiation is hard. Branded methodology helps.

I hear it all the time from potential clients: “The other guys have the exact same pitch. What makes you different?”

If there’s no difference, why should they hire you? Of course, you can differentiate in nominal ways: better case studies, better testimonials, slicker landing pages, etc. And you can compete on price by being the cheapest. But it’s a race to the bottom: there’s always someone cheaper than you.

What’s the solution, then? Branded methodology.

We have the PXL prioritization model and the ResearchXL conversion research framework – both widely adopted in the industry. It definitely helps to position us as the knowledge leader in the space.


Building frameworks also helps standardize the industry practices.

In CRO, this is particularly important because of the loose association of “experts” that may or may not be doing things the right way. When there are rogue consultants promising big lifts and just throwing ideas at the wall and seeing what sticks, it’s bad for the industry.

Models and frameworks give everyone a standard process, and it helps everyone (and gives you a brand boost).

5. Qualify

Lead quantity can be a vanity metric. We used to waste so much time and money on clients that weren’t qualified. It’s a huge issue.

How do you eliminate that waste of time? The easiest way is to improve lead qualification.

One way is with lead scoring. There are so many tools out there for this that it’s easy to get lost, but the essence is that they score based on a series of inputs (company size, industry, visited specific pages, etc.). That way you can automate lead qualification in a way.

Another way is to filter out bad leads before they can even fill out the form. The best way we did this was by adding a form on our landing page that asks for their budget for the project. The lowest value is high enough that we eliminate the tire kickers and wastes of time:

6. Fire shitty clients. It’s not worth it.

No matter what stops you put in place, you’ll get a few clients from hell.

Clients from hell will suck the life out of you and your business.

In building a conversion optimization agency, we’re chipping towards our dream and our vision, and a lot of that is daily grind. And to get to our goals, we need to enjoy the grind. When you have a horrible client, this chips away at that happiness and you sacrifice the long term goal for the short term cash flow. Sometimes this is necessary (you have to feed your family), but if you can avoid it, avoid it.

Signs of a client from hell:

  • They want to use real time communication with you (Skype chat, Slack, etc.)
  • They’re anal retentive about every little detail that goes into the contract or the proposal.
  • They want 17 meetings before they’re ready to move forward.
  • They try to haggle the pricing, hard

If we get a lead that comes in with the above signs, I usually drop it pretty quickly now. It’s as simple as putting out an ultimatum: “Guys I don’t think this is working out. Either we change the way we’re working and you give us more freedom to do our stuff, or we part ways. If we can come to an agreement, that’s great. Otherwise, no hard feelings.”

7. Productize. Don’t sell hours.

Most people sell by hours. Don’t do that.

You’re in the business of expertise. You sell your expertise, not hours.

Selling hours is a dick move to your clients. Why? People that are way better get the job done faster and more junior people take longer. In addition, it incentivizes you to take as long as possible to get the job done. You know, you get paid by the hour, so why hurry?

The hero of this story is value-based pricing. As long as the value is there, no one should care how many hours it takes you. Read The Consulting Bible, and start pricing your services based on the value you bring.

The goal with our agency is 5-7 times ROI per client, so we plan out our value-based approach using that. That’s how we figure out our monthly flat retainer as well as our other project-based offerings.

To do this yourself, you can look at inputs. How many tests per month can you run? If it’s a low traffic site, you’re limited by statistics (another reason to work with bigger clients). Your monthly retainer should reflect a certain number of tests launched per month. Think about:

  • How many for desktop?
  • How many for mobile?
  • Analytics and measurement included?
  • Research and insights provided?

You can also productive other services. Do you provide one-time analytics or conversion audits? These are great, because you usually find a million problems and then you can productive the implementation/fix as well.

On another pricing note, I’d stay away from pure revenue share/performance-based pricing.

Companies that want you to do that instead of flat fee generally don’t have the money to be worth it. It’s only the small businesses that want it – because they don’t have the cash to pay you. No large enterprise has ever asked for it.

If they were a big enough company, the upside for you is too huge with revenue share. Avoid revenue share at all cost, mostly because of the fights that will inevitably happen. Once you get wins, your client won’t want to pay that much and will start to find reasons why you shouldn’t be paid that much. It often takes lawyers more time to agree on a language in the contract than the client engagement itself.

I have seen some successful implementations of hybrid models though: flat fee + performance bonuses. So, if you reach certain milestones. Stay away from pure performance though.

8. Training CRO folks takes a lot of time

If you’re going to hire CRO folks that have never done it before, expect a long ramp up period.

I used to believe in the phrase, “hire for talent, train for skill.” You have young and ambitious people available for an affordable price. But I realized that the ramp up period was incredibly long. Why is that the case? To be good at CRO you need to know a lot about a lot:

  • Testing
  • Statistics
  • Psychology
  • Copywriting
  • Design
  • Analytics
  • Front End Development

That’s just the CRO-specific skills. It takes years to become decent at this.

If you’re client-facing at an agency, you also have to have account management, presentation, and project management skills. You need to be excellent at navigating political waters. Think of it like a graph, where on one axis is the quality of work and the other one involves the quality of the relationship. You want to be in the quadrant with the good work and good relationship, otherwise you won’t be working with that client for long.

Your people need to be good at so many things. I’ve had many great analysts that were skilled at specific things, but when it comes time to talk to a client, things fall apart. It’s hard to find a unicorn that is good at both sides of the equation.

Because of the difficulty in hiring, it’s necessary to retain employees for longer, and also to hire experienced CRO people. Turnover and constantly training new employees will be too costly and too time consuming for your business.

However, you can speed up your training. On-the job training works, but also formal training like what we offer at CXL Institute. We now have hundreds of companies sending their conversion optimization employees being trained there. It works.

9. Find mentors who have been there before

When I first started my agency, I was constantly reinventing the wheel because I didn’t know any better. Then I started going to conferences, and you know the value there isn’t necessarily the talks, it’s the people and the networking.

I would find the people who were running successful agencies and interview them. I asked what they were charging, what their processes were, etc. and we implemented the lessons at my agency.

The greatest thing you can have: a one-on-one mentor, someone who has been there and is a master of the craft. Or if you don’t have a real one, you can get a virtual one – like the Conversion Optimization Minidegree.


Running a conversion optimization agency is hard, make no mistake. But it’s rewarding. No matter what, you’ll make mistakes and learn from them. But do what you can to avoid them by learning from others.

Related Posts

Join the conversation Add your comment

  1. Thank you for all the great advise Peep! If you’re willing to share I’d love to learn more about how you productized your offering and your approach to value-based pricing.

  2. Fantastic list! For #6 – I’ve sometimes mislabeled “clients from hell.” Especially for smaller clients, their livelihood depends to some extent on your ability to make them more money, so some of them are going to be questioning/pushing non-stop. I don’t put up with people second guessing our strategy or how we do the manage their accounts, but if they just want more handholding and are willing to pay for more meetings external to the actual work, I’m usually willing to bill out as their marketing therapist too.

    1. Fantastic reply. Marketing therapist? I should be billing a lot more people after this.

  3. Hey Peep. Thanks for sharing. As a relatively new business owner I identify with a lot of what you have said (and made many of the mistakes)

  4. Sterling article, thank you. I can confirm many of these points to be universal lessons for many small businesses in online marketing, as they were true for me in my own business too. Solid advice; thumbs up.

  5. Great post Peep, the issues you raise are not limited to a CRO agency, all agencies are the same and have experienced the same myself running a marketing agency, which does CRO too, maybe it’s time to focus!

  6. Solid content!

    Speaking the language of ROI and talking about the value of that 1% increase is something we inherited from you a year or two ago with great results. Building on that, we also find that showing a potential client a couple of possible outcome scenarios before starting the work (0%, 10%, 20%, 30% gains) is a great way to build trust and make the sale.

    The other thing we learned and I can vouch for is to move away from selling single tests (sprints) towards a multiple month engagement (marathons) where more time is available to fail, iterate, learn and succeed.

    Oh and thanks for giving a few positive points to the hybrid pricing model. :)

  7. When you pitch your services to a client, do you estimate the potential lift that you can get them? Or do you advise against that?

Comments are closed.

Current article:

9 Lessons Learned from Running a CRO Agency