Split Testing Landing Pages : The What

This will be the first installment of a mini-series of articles on split testing your landing pages. I’ll kind of throw these posts in over time around other posts, so don’t expect them all in a row.

Ok, so we all know how optimization is key, blah blah blah. You’ve heard it a million times before…A/B split testing landing pages is critical for success and optimizing profitability. Rather than babble philosophically about the Why, the practical and useful information is the What. So What exactly should you be split testing? A lot of the things I’ll go over may be obvious to some, but maybe this post will provoke some creative thinking in other areas you can split test on your own pages.

Topics are always easier to explain when examples are provided, so I’m going to use one of my old landing pages to explain split testing. I’m going to use one of my many dating landing pages that targeted “chat room” type keywords (this used to be a huge niche before all advertisers decided the quality was dreadful).

Click here to see the page.

And quickly before the haters come in,

Back to the show, here’s a smaller screenshot of the page with the areas I’m going to talk about :

Note: you can already see that this page is slightly different than the one in the link. This I just grabbed from the source PSD and shows the original button that was green instead of purple. The purple button won the split test battle.

Note #2: This landing page isn’t perfect by any means, I designed it myself and it never really got out of the “prelim” stages because I ended up just running a Mate1 host&post instead.

Whoa, for being a pretty simple and straightforward landing page, you can see pretty much everything on the page is split testable. Proper optimization goes a bit deeper than just A/B testing the color of a button. Now I’m going to break down each part, and how it can be split tested.

#1 – The Header

So you can see we went with a header that’s going to blend with the Zoosk affiliate offer. They use that same blue, and you can see I pulled their logo to increase the relevance and connection between the landing page and the actual offer. On the right hand side is what we want to split test here, that header text. This very well may be the first thing users read when they hit the page (heat maps would show), so we want it to be extremely relevant to the ad they just clicked. You can also see I was geo-targeting the headline so it would read whatever state they’re in. Here are a few different variations I would split test :

1. 5,934 Singles Chatting in New York (control)
2. Nothing (leave the area blank)
3. 5,934 New York Profiles to Browse
4. 5,934 Singles Chatting Now
5. 5,934 Singes in Chat Rooms
6. continue to chat rooms >> (make clickable link)

You can pretty much come up with a million and one variations and ways to word it, which is the beauty of optimization. No matter how profitable or amazing your page is, there’s always a way to squeeze just a little bit more out of it.

 

#2 – Main Headline

If the header headline isn’t the first thing a visitor will read, the page headline probably is what they looked at instead. So just like with #1, we want this to be relevant to the ad they clicked. Here are some variations of this :

Those are just a few examples. The actual text isn’t the only thing you should be testing – colors are huge and can drastically affect the performance of your page.

 

#3 – The Button

The big button very may well be the most important part of this landing page. Therefore, it’s something you want to pay extra attention to when split testing. When split testing a large call to action button, keep in mind the many variations you can make to it :

  • text
  • font
  • color
  • styling (gradient, drop shadow, glow, etc)
  • arrow
  • animation
  • hover-over

Here are a few examples using the button on the Zoosk page :

For the examples I just kind of did a bunch of random sampling. When you’re actually going to split test all of these elements, for the most part you should do them individually, and in order of prominence. For these buttons, I’d split test in this order :

1) Color
2) Text
3) Arrow
4) Font
5) Styling

 

#4 – Text Section

With simple landing pages like these it’s generally wise to at least have some content there for people to read. A lot of the visitors are just going to disregard this section and just click the big button when they see it, but not everyone will do this. Rather than spend an hour creating a bunch of different examples, I’ll just type some different possible variations out.

1) Remove the section. Maybe it will convert better with just a button and image? You never know until you test it.

2) Change the bullet points. You can change the order, change the bullet image, or highlight different points.

3) Replace it with a “New Members” section. Find a couple snazzy mug shots, give them a name and short bio, geo-target the location, and have a small “Chat Now” button to the right of each member.

4) Replace it with a “Current Stats” text box. List things like the number of photos uploaded, number of videos, number of chatrooms, number of active chatters, etc.

 

#5 – The Small Button

Variations for this button will be pretty similar to the large button, so no use in going too far in-depth about it. When you’re split testing and change the big button from say green to purple, don’t always change the little button color with the big button. A green big button and red little button may have a higher CTR than a green big button and green little button. Multi-variate testing is key.

 

#6 – Photo Button

Couple elements we can split test here: the color, text, and font. Some possible text alternates :

  • Chat Now!
  • View Profile!
  • Send Message!
  • Free Chatting!
  • FREE!

Something like the font is an element that probably is pretty useless to split test in this scenario. Chances are the font type really isn’t going to make any difference on the overall CTR of the page, but once again you never know until you test and confirm that. When setting up your first multi-variate experiment though, only test the most important elements (the main headline, call to action, picture, and content).

 

#7 – Picture

The picture is a pretty big element on the page. I’d actually say this is the first thing your visitor is going to see, so split testing it is extremely important. This is a dating offer we’re talking about here, so guys are going to want to see someone sexy. You can see that the actual example webpage and screen shot have different pictures, those were a couple that I tested.

The actual picture isn’t the only thing we can split test here either, we can test different photo frames as well. Here’s a couple examples :

 

#8 – Photo Text

The frame and picture aren’t the only things we can test here, you can test different texts out like :

  • Add Friend!
  • Send Message >>
  • Chat Now >>
  • Chat this user >>

Again, this is a smaller detail, but it might be something that catches their attention.

 

#9 – Facebook Connect

Footers are always nice to kind of close a page off and complete it. In addition to having a regular lead-gen page, Zoosk also has a Facebook install that pays out. So I took the footer space and figured I’d use it for that on the chance that a person may see Facebook as an option they want to explore.

As far as split testing this section goes, maybe you could build out that area more or draw more attention to it. It’s not something I split tested though just because it’s really the least important part of the page.

 

Extra Notes & Things to Talk About

The examples above are all just for that specific and simple landing page. While most simple landing pages have very common elements (big button, image, content area, header), there may me more or less things to test on your page. This post was meant to get the creative juices flowing in your brain and maybe help you realize that split testing goes beyond just changing a few colors around. Here are some other things that you may want to split test on your own pages :

  • The background. In our example we wanted to blend with the offer, so there wasn’t much we could do with the background. In many other instances, the background is something that can and should be manipulated. Try different flat colors, gradients, patterns, and dynamic images to see which color theme converts best.
  • The domain and logo. This is something that may make a difference to your visitors. A site may perform better with a .org domain instead of .com, because the certain audience you’re targeting may be expecting something really credible, like an organization.
  • Page arrangement. Should the content be on the left side and button be on the right side? Or should the content be on the left side and button on the right side? All things to think about and test.
  • Complexity. This is something we briefly talked about earlier. Build your page exactly how you want it and save it. Then, at the bottom add in a horizontal divider and underneath that add more content. Pop in a graph, another image, and another content paragraph. Your visitors may want more to read on your landing page before moving on. On the flipside, sometimes brutal simplicity is what works. Test it.

Again, a lot of these things are obvious and this is not the end-all guide for split testing. But, you may find that taking a little bit of a deeper look into your own pages can yield some surprising results. We may test out all 9 elements on the Zoosk page and find out one thing that made a big difference in CTR was the Photo Text (#8). The photo button, photo frame, heading text, and content box may make no difference. But it was worth testing everything out to find that one small element made a big difference. I’m not talking out of my ass here either, you’d be surprised at how much just wording a line of text here or there can make a difference. A 1% increase in CTR over hundreds of thousands of impressions can make a big difference in your long-term profit.

The next article in this mini-series will focus on The How. We know a bunch of things to split test, but how do we best do it?


30 Comments

  1. Georgie
    January 16, 2011

    Before advertisers ‘decided’ quality was crap? Or before they realised?

  2. January 16, 2011

    Wow! Great information about split testing.. I always split test but definitely not this much, seems like I’m leaving money on the table!

    -Dino

  3. January 16, 2011

    Dating niche is hot on POF great LP design & tips!

  4. January 16, 2011

    @Georgie,

    Before they decided it was crap. The day zero on a lot of older dating offers used to be great with chat traffic. I was getting higher payouts than any other affiliate got. Then it went bad and we got cut…true story.

  5. Raphael
    January 16, 2011

    Nice post Thanks!

    dang, 9 variables…

    then i guess it will take some serious time to get it fully optimized huh.

  6. January 16, 2011

    It’s really not that hard. I’ll go into detail about it in the next article.

  7. rgirdin83
    January 16, 2011

    Dude,
    Yoour testing way too many things and you’ll need a bajillion visitors before your multivariate tests are statistically relevant.

    Stick to testing the things that have been proven to have the most impact on conversion:
    1. Headline
    2. Main picture (not the border of the pic, but the actual person in the image. Also test face only shots vs. ful body)
    2. Subhead
    4.Button text

    You really don’t need to test more than that to start, That will give you better data faster on areas that will have more of an impact.

  8. January 16, 2011

    @rgirdin83 I know that and I’ll explain it more in the next article. It’s important to first only test the critical areas on the page, and as time goes on optimize finer and finer. You’re right, you would need a ton of money to get accurate data testing 9+ variables all with multi-variates at the same time.

  9. Rhawn
    January 17, 2011

    Good stuff, looking forward to the rest of the series.

    NICK THROLSON SWOOPS IN WITH THE GOLDEN NUGGET. POW.

  10. January 17, 2011

    Awesome stuff… looking forward to your next article. Keep up the good work Paul ;)

  11. January 17, 2011

    Breath of fresh air Paul… You design most of these graphical elements yourself or do you have someone crank these out?

  12. January 17, 2011

    Great article Paul! I hope that you will get a bit more in detail on what you test first with what kind of budget. That would be an interesting read since we’ve limited PPC budgets ;-)

    Great to see one of your old landing pages finally! I think a lot of us can learn from this.

  13. January 17, 2011

    @local affiliate Landing pages are simple enough to where I can design most of them myself. For more important and larger projects, I have designers.

    @Frank Yep, the next post in the series will clear things up a lot about how to split test so many elements effectively.

  14. Mike
    January 17, 2011

    Is there a technique you use to split test the landing pages? Is there some sort of script or program? Or do you just make changes and compare daily CTR’s.

  15. January 17, 2011

    great writeup,

    Depending on your traffic source/targets, you could test more spots.

    I tend to test more on PPV traffic as it comes quicker and cheaper. You could then port the winning combinations over to ppc/social/etc and do more testing.

    p.s. I see a gwo post coming up soon :)

  16. January 17, 2011

    Awesomely comprehensive post! I’ll be referring my clients and designers to this post for sure.

  17. January 18, 2011

    Good to see you back – solid intro post for split testing.

    I’ll definitely send a few affiliates to this post – lots of people really need help on this topic. I’ll be interested to see the next one as well.

    Welcome back!

  18. January 18, 2011

    Does this work when even using other networks?

  19. Yogi
    January 20, 2011

    This post is very comprehensive and really “spill the beans” on LP optimization.
    Good stuff, gave me a big benefit…

    I’m not an IM newbie, but LP creation is kind of new to me.
    This may sounds like a really basic question, but
    how do you design all the variables of a landing page?

    do you use an editor like Photoshop or something similar?

    Thanks in advance.

  20. January 20, 2011

    Very Interesting Post! Thank You For This Post!

  21. marco
    January 20, 2011

    This is will help a ton of people, including me.

  22. January 20, 2011

    Split testing within wordpress there are several plugins you can use to split test wordpress themes.

  23. January 22, 2011

    Split testing really improves how the website performance

  24. gdubs
    January 22, 2011

    “Dating niche is hot on POF”
    “INSIGHT”…throlson has it

  25. February 1, 2011

    Well, very well people know you and your website. I think because of your already existing establishment and well known brand, person that you can have even some dating.
    Thanks for the post, well written , analysed and informative.
    Also how you are working around the pop-up blocker? Won’t it prevent many users ( especially new visitors)?
    If ido this on my site which does not have traffic like you. I’ll be toast! ;-)
    http://joinmillionairesociety.blogpost.com

  26. February 9, 2011

    very good post! one of the best that i read about landing pages..

  27. February 16, 2011

    Dude this was a killer article. I’ve been tabing back and forth making changes to my site while reading it.

    Cant wait for the next part~

  28. February 18, 2011

    Great post,

    Your blog will help me to increase PPV traffic on my site. I really need the help on this topic. You put really very helpful information here. Keep blogging, I will look for your next post.

    Thanks

  29. March 20, 2011

    What a great post, I even had a little tear in my eye, the guy is so happy now you may have changed his life just by caring. Okay back to the reason the article was written this really shows an out of the box type message and evolkes the mind to think differently split testing is the only way to go. Thanks for posting

  30. tke71709
    April 28, 2011

    Any news on the next article in this series?

Leave a Comment