Post by huangshi715 on Feb 15, 2024 7:04:01 GMT -5
A few weeks ago, I did a keynote at a conference in Copenhagen where I talked about the 10 biggest A/B testing mistakes I’ve made in my career. Preparing for that presentation was a fascinating process as it forced me to scrutinize all the mistakes I’ve made and narrow them down to a top 10 list. I was surprised that one of my biggest mistakes was that I used to do too much testing. Sound strange coming from a guy who has been known as “the split test junkie”? Well, read the rest of the article and it will make a lot more sense… Let’s start with a real-world success story I recently ran an A/B test for a client where we experimented with a radically different version of one of their most important landing pages.
The treatment won the test and went on to earn the client an extra five Kuwait Email List figures within the first two weeks of replacing the original landing page. How did we get these results? It wasn’t by running a large number of experiments or testing a bunch of different ideas. We got the results by putting a hell of a lot of time into preparing the landing page treatment. In fact, of the time I spent working on this project, only a fraction went into actual testing. We spent the majority of time collecting data, getting insight and building optimization hypotheses.
I did customer interviews, conducted online surveys, dug into analytics, analyzed funnels, ran heat maps, reviewed session recordings and interviewed the guys in customer service and sales. Then I took all that data and turned it into actionable insights that I could use to build a solid, informed hypothesis on how to optimize the landing page so more potential customers would buy. Only then – with all that insight in place – did we move on to working on the copy, content and design for the landing page itself. Give me six hours to optimize a landing page and I’ll spend the first four getting insight Abraham Lincoln once said, “Give me six hours to chop down a tree, and I’ll spend the first four sharpening the ax.
The treatment won the test and went on to earn the client an extra five Kuwait Email List figures within the first two weeks of replacing the original landing page. How did we get these results? It wasn’t by running a large number of experiments or testing a bunch of different ideas. We got the results by putting a hell of a lot of time into preparing the landing page treatment. In fact, of the time I spent working on this project, only a fraction went into actual testing. We spent the majority of time collecting data, getting insight and building optimization hypotheses.
I did customer interviews, conducted online surveys, dug into analytics, analyzed funnels, ran heat maps, reviewed session recordings and interviewed the guys in customer service and sales. Then I took all that data and turned it into actionable insights that I could use to build a solid, informed hypothesis on how to optimize the landing page so more potential customers would buy. Only then – with all that insight in place – did we move on to working on the copy, content and design for the landing page itself. Give me six hours to optimize a landing page and I’ll spend the first four getting insight Abraham Lincoln once said, “Give me six hours to chop down a tree, and I’ll spend the first four sharpening the ax.