nn

ernests embutnieks blog

Recent Posts

AB-TEST-RESULTS

Button or Banner – Google Optimize A/B Test

If you’re reading my blog, then you know my blog is optimized with Google Adsense and with a couple of Affiliate Program, one of my biggest partners are Brave, at least that’s the only one that brings in a small amount of revenue or as I call it – pizza money.

If you’ re only starting with digital marketing or just learning how to start a blog, you might find it useful later down the road.

There’s one specific landing page where I compare Google Chrome vs Brave Browser. I had a link at the bottom if you like you can Download this browser here. Some time ago I checked my statistic and I figured I should try to improve the CTR(click-through-rate) to that link. I figured if the link would be more appealing and visual people would click on it more. So I decided to change the link to a button and see how it would increase my conversion rate. Now I feel dumb that I didn’t do A/B Test on that, but just changed it straight away, could have made a small case study out of it.

Results:

increase-your-conversion-rate

Now sure, there was also a traffic increase during March, but the conversion rate increased by 121.49%. Not so bad is it? Imagine if it was a real business that earns let’s say $5000 a month and just by changing one small details your earnings increase to $12 000+

Now, people that work in Marketing knows that this is a no brainer, if you have a clear CTA(call-to-action) button your CTR will increase dramatically, but I just like to measure it anyway. So since I didn’t do an A/B test on that, I figured I can do another one.

I will switch the button at the bottom with a banner and let’s see what happens.

A/B Test – Button or Banner

  • Variant A – The Button or the original. So this is the button that I’ve been using before this test: Download-button
  • Variant B – Custom Banner. I found this banner on the internet on a couple of websites so I don’t know the original source and I hope the creator won’t mind.
    download-brave-banner-test
  • Variant C – Brave’s Official Banner. Which can be found on Brave’s official website under assets for creators.
    Brave-official-banner-test

That’s the three variants I’m going to test out. This is how it should look when you have finished your variants:

 

Hypothesis

My hypothesis is simple – the banner needs to be engaging, clear and of course noticeable. Just by adding a simple download button it already increased the conversion rate dramatically, we saw that already above. I believe if we’ll have a bigger banner with clear CTA text our conversion rate will increase even more.

My belief is that Variant B will outperform the rest of the variants. It’s engaging with a clear CTA button. Definitely a lot more noticeable than variant A. I don’t believe that Variant C will perform that great. It has a great design. From all of the variants I like it the most, but the CTA text is just…bad. It’s not clear, it’s confusing. By clicking on it I’m not sure what’s the next step.

Summary:

By creating a more noticeable banner with clear CTA text we will increase the conversion rate.

Choosing an Objective

My main goal is to figure out which of the three variants will have the highest conversion rate. I can’t track it with 100% accuracy, because the event doesn’t happen on my website, but what can I track is how many people actually click on the banner or the button, so that will be my conversion.

As you know Brave pays $5 for each user that downloads the web browser and uses it at least for 30 days. So if you have a question why should I do this kind of test, then the answer is simple – I’m trying to increase my revenue, by changing one simple thing on my website.

Your website or your business might be completely different from mine, so you would just have to create a different structure & objective. Maybe your goal is to increase the average duration time on your website or increase the conversion rate on your email signups or social share or anything else that you could think of and want to improve, regardless all of these things can be optimized and tested.

Now is it hard? Not particularly, but if you would like me to create on how to work with Google Optimize, leave a comment below.

I launched the test on 31. of March.

Now for some reason, the experiment didn’t work on all sessions so it took some while to get some traffic inside to make some conclusions. If your website is receiving a high amount of traffic you shouldn’t struggle with this. You will need a lot of traffic to prove your hypothesis is false or correct.

A/B Test Results

It’s been almost 3-months. In April I accidentally screwed up the code of the website which stopped Google Analytics for a couple of days, so I lost a little data, but regardless, here are the results of the experiment:

ab-test-results-google-optimize

My hypothesis was that Variant B (Custom Banner) will outperform the rest of them since it’s the biggest banner with clear CTA text, although I believe the design still can be improved, especially the font it still had the highest conversion rate.

Even though it has the highest conversion rate we need to make sure that the statistics are significant enough.

Variant A vs Variant B

AB-Test-Results-Winner

Source

So even though the first impression is that Variant B is the winner, but it only has statistically 59% change that it has a higher conversion rate than Variant A.

Variant A vs Variant C

ab-test-results-ac

Variant A outperformed Variant C. In this specific calculator we can’t create a test with three variants. Even though it seems like a no brainer, but statistically speaking there’s only 90% chance that Variant A is better than Variant C in the long term.

If you use a different calculator, please leave a comment below with your suggestion.

Summary

I can’t say with certainty that Variant B (Custom Banner) was the winner, meaning I can’t say that my hypothesis was correct with 100% certainty. Welcome to the statistics. In this case, I will go with my hunch and just choose Variant B on my site. 59% is enough. For the next A/B test, I will gather more statistic so it’s more accurate and I can make the change with certainty.


Thanks for reading, if you have any comment drop a line below. If you’ve had any A/B tests on your site that improved your results I would love to hear them.

Don’t forget to subscribe to the email list and push notification.

My website is Brave Verified, if you want to tip me with BAT, you’re more then welcome.

Sometimes Marketing is More Art Than Marketing. A lot of People Don't Get That.

Leave a Reply