Are You Forgetting One of the Most Important Factors In Your Keyword Research?

  • November 17, 2011 by Guest Writer
  • 32 Comments

Note from Caleb: This is a guest post from Tom Ewer of Leaving Work Behind.

There are many mistakes you can make when conducting keyword research. You must be disciplined in your methods in order to avoid the potential hazards. Fortunately, there is a great deal of advice on the Internet regarding keyword research. If you read up on the subject and carry out your research in a considered and objective manner, you will have a good chance of avoiding disaster.

However, there is one important metric ignored by many keyword researchers that is just as important as traffic estimates when deciding upon the viability of any given keyword.

What is this metric? Google organic search click through rate.

First Of All – Conventional Wisdom

Many of you are likely familiar with Market Samurai. I use it and find it extremely valuable. There is however certain data produced by the software that I completely ignore. Search Engine Optimization Traffic (SEOT) is one such metric that I completely disregard when carrying out keyword research.

Why?

Market Samurai’s SEOT calculation is based upon the infamous AOL data leak of 2006. In short, they calculate SEOT by multiplying the estimated number of searches by 42%. The problem is that the data from the AOL leak is almost completely useless.

There are many reasons for this. For one, the data is 5 years old. Web users are a very different breed nowadays. It is also based upon usage of the AOL search engine – which is nothing like present-day Google. The data set is also aggregate-based – i.e. it assumes that each user only clicks on one result. I could go on.

The AOL data is seemingly the most widely-used when estimating search engine conversion rates, which is rather concerning. Please, disregard conventional wisdom.

Available Data

So what study should you look to? Well, there is certainly plenty of data available. There is just one problem – none of the studies agree with each other. To give you an idea of the scale of the problem, take a look at the information below. It is taken from six data sets (including the AOL data), collected between 2004 and 2011.

Data sources: SEOmoz | Enquiro | BrandSoftech.com | Optify | Cornell University

The graph is a graphical representation of the percentage of visitors that click on each position in Google according to each study. The table reveals the percentages.

The table below that shows the percentage point swing from the lowest percentage data point to highest percentage data point, per ranking position, across all studies. So for instance, the Slingshot study predicts the lowest click through rate for the top spot in Google (18.20%), and the Cornell study predicts the highest click through rate (56.36%). The swing in percentage points between those two numbers is a huge 38.16%.

What does this mean for you? Well, let’s say that you are targeting #1 spot in Google for an exact match keyword that is searched for 15,000 times per month. If you used the Cornell study, you would expect nearly 8,454 visitors per month from that one keyword. If you used the Slingshot study, you would expect just 2,730 – 32% of the visitors that the Cornell study predicts.

Take that one step further – let’s say you do some calculations, and decide that each unique visitor to your site is worth, on average, $0.05. Depending upon which study you reference, your monthly income projection for the above hypothetical keyword will be anywhere from $136.50 to $422.70. That is a big difference!

Multiple Variables

Now let’s really throw a spanner in the works.

Take a look at these screenshots:

I’ll take your organic search results, and I’ll raise you social media, photos, You Tube, news feeds, local listings, maps, and of course paid placement. Then we’ll throw location-based results tailoring, and custom results based upon your own personal search history into the mix.

How on earth can you make an accurate judgment as to what your conversion rate will be with so many variables in play? The fact is, you can’t. There are some things that you can do though.

Consider The Taxonomy Of Web Searches

The following information is derived from the study, “A Taxonomy Of Web Search” by Andrei Broder, who is the vice president of search advertising at Yahoo. He produced this report whilst he was with AltaVista. The numbers presented in the report are rather interesting for our purposes.

According to Broder’s survey, 24.53% of searchers were looking for a particular brand. Your site may be completely overlooked by the majority of such searchers, if they are in fact looking for a brand name website that happens to contain certain keywords that you are ranking for.

Only 8.16% of searchers were in ‘buying mode’. Say if you are running an online tennis store and are targeting rather ambiguous keywords, such as ‘tennis’. Your click through rate may be extremely low for that keyword, even if you rank #1 in Google, as less than 10% of searchers typing in that keyword are likely to be in a buying mood. Of course, this percentage should increase with more commercial keywords, such as ‘tennis racquet reviews’.

If you find a keyword that seems to have great traffic, low competition and good commerciality, pause for thought – will the searchers be attracted to your site? Is it a good fit?

On Site SEO

I find this is something that is often overlooked or undervalued – your site’s meta title and description. I cannot understate the importance of these two key meta tags.

Put yourself in the shoes of someone who comes across a list of sites on Google. Now, the order of the sites is definitely a key factor in determining where they will click, but searchers are unlikely to click even on the #1 result, if the title and description does not match what they are looking for.

This is one area in which you can have a tangible impact on your search engine conversion rate. Make sure that your site’s title and description are tailored to your understanding of the searchers’ intent.

Get Experienced

The more keywords you rank for with your sites, the more data you will collect, which in turn will begin to provide you with a fairly good idea of the kind of conversion rates you can expect. I am not saying that you should ignore the above reports or any other available data for that matter – you should digest it and use it as part of your estimates. But ultimately, stay conservative so that you do not get any nasty shocks down the line.

Although this article is filled with data and studies, the key point is very straightforward. First, always carefully consider what your search engine conversion rate may be, and always err on the side of conservatism when finalizing your estimates. And second, ensure that your site is optimized for the audience you are targeting, otherwise despite high rankings, you may never receive a great number of visitors.

What experiences have you had with organic search click through rates? Have you experienced particularly high or low click through rates based upon the projected traffic, and if so, what has the experience taught you about converting organic search engine traffic? 

Tom Ewer is the founder of Leaving Work Behind, a blog that aims to put you on the path to self-employment and working on your own terms. Find Tom on Twitter.


Think Traffic is now The Sparkline. Click here to check it out.

Or View The Archives

Danielle Lynn November 17, 2011 at 6:43 am

Hi Tom,

Thank you for your detailed and thoughtful post about keyword research! I’m no SEO expert, but I do dig up my own keywords from time to time. I’ve wondered before about the factors you’ve mentioned — especially after observing some of my own surfing habits.

Like you pointed out, I notice that I tend to skip sites that don’t have the right meta tags, or don’t match what I’m looking for.

Personally, I’m guilty of not putting much thought into my own meta tags. Guess I know what my personal side project of the day is going to be!

Great post!

Tom Ewer November 17, 2011 at 8:54 am

Hey Danielle,

You are more than welcome :-) I’m glad the post has given you a reminder to address a factor that is pretty damned important when it comes to SEO.

Tom

Cristina Ansbjerg November 17, 2011 at 7:10 am

Amazing in-depth post here Tom. Very well researched.
I recommend everybody Tom’s ebook about keyword research. You can find it in his blog.

Tom Ewer November 17, 2011 at 8:55 am

Hello Cristina, fancy seeing you here! ;-) Thank you for the recommendation – that means a lot :-)

Conni November 17, 2011 at 7:17 am

Wow, Tom, great post! I should really implement at least the basic SEO for my blog and posts etc., so great reminder and great advice!!!

And like Cristina, I can totally recommend Tom’s Keyword Research Ebook!

Tom Ewer November 17, 2011 at 8:55 am

Conni – thank you for the kind words :) I’m glad I’ve got you two on my side ;)

Greg Taylor November 17, 2011 at 7:18 am

Nice post. Great mention of onsite SEO strategy. If your article, search or SEM keywords aren’t congruent with your brand’s web keyword strategy you’re missing out on the full benefit of an optimized page or post.

Thinking big picture, not short term, pays off in spades.

Tom Ewer November 17, 2011 at 10:38 am

Anyone who manages to use the word “congruent” has something useful to say, in my opinion :)

Seriously though, the congruence you speak of is damned important. Couldn’t agree with you more.

Daniel Himel November 17, 2011 at 7:48 am

This post is epic. Corbett, thanks for taking the time to put this together.

Corbett Barr November 17, 2011 at 9:22 am

Sure Dan, and thanks to Tom Ewer for writing it for us!

Dave Tong November 17, 2011 at 7:52 am

What an epic post…

This post pretty much answers quite a lot of forum questions like “my site’s ranking on first page for a KW with 5-digit exacts but I’m not making money!”…

Thanks a million!

DAve

Tom Ewer November 17, 2011 at 8:56 am

No problem Dave – I figured this out the hard way – I would love for other people not to have to.

Sergej November 17, 2011 at 8:24 am

Great post. This information is really new for me. I was always looking at SEOT as accurate number.

I’m just curious now, why guys from Samurai team don’t implement or include other statistics.

Thanks you

Tom Ewer November 17, 2011 at 10:37 am

Market Samurai can either stick a completely inaccurate number in, or get rid of the SEOT number altogether. There is no “intermediary” option.

There is no magic percentage for your click through rate. As explained in the article, there are too many factors at play.

I’m not sure why Market Samurai include SEOT. They must have a reason. Maybe you should ask them! :)

Gregory Ciotti November 17, 2011 at 8:25 am

You had my attention at Halle Berry ;) .

Seriously Tom your keyword content is top notch, it’s always fresh too and pretty much offers me a new perspective every time I read it.

Keep up the great work bud.

Tom Ewer November 17, 2011 at 8:58 am

Thanks Greg :) all of this great feedback is really heartening – the post took a long time to put together and was borne out of some pretty heartbreaking mistakes!

I have no idea why Halle Berry popped into my head when I was looking for a good example of photo and news inclusions on the Google SERP… ;)

Pat November 17, 2011 at 11:39 am

Experience is definitely the best way to collect the best data. Each niche and each keyword will be treated by Google and clicked thorough differently. In my experience, many of the sites I’ve built do have the 42%ish click through rate, but plenty others do not.

Great article Tom!

Tom Ewer November 17, 2011 at 12:53 pm

Thanks Pat :)

veppa December 7, 2011 at 4:28 am

I also get 39% on related keyword positioned #3 and 51% on niche keyword position #1. on google. So I think market samurai numbers depend on competition. If competition low then there is no much directly targeted content from competitors then you can get higher than 42% CTR.
I agree that high CTR is due to title and descriptions tags.

Franck November 17, 2011 at 11:48 am

Sorry to say, but i didn’t like the post

The way our interpret data sets is frivolous (samples size?, pondered average, standard deviation?, pears and apples mixed together?), the supposed problem of SEOT is pointless, and worst of all the mencioned Broder’s brief (the right way of doing things) is from 2000, when Internet was at a prehistoric stage.

* No ones knows SERP CTR’s
* As a business owner you need something to take decision
* Choose SEOT or choose any other metrics, and stick to it. At least you are taking decision in a coherent manner
* Keyword Research is all about making a decision at a given time. There is NO absolute numbers or GOOD / BAD. You take of picture of a niche and take your decision, That’s it

Forget the numbers, it’s all about RELEVANCE. It’s exactly what Google is aiming for its users, relevant answers for the daily searches its users.

Tom Ewer November 17, 2011 at 1:04 pm

Hey Franck,

No need to apologize :) I welcome debate.

Unfortunately I don’t understand what you mean by your first paragraph, so I can’t respond to that.

Beyond that, I think you are saying that SERP CTR’s are ultimately very difficult (or impossible) to estimate to within a reliable degree of accuracy. I agree (and make that clear in the article), so I’m not sure we have an argument there :)

You say to choose SEOT or choose any other metric, it doesn’t matter which, as long as you are consistent. I disagree with such advice strongly, for the very reasons explained in the article. If I am consistently overestimating the CTR, I will be setting myself up for failure (especially if I am making a capital investment for which I expect a specific return). Consistently relying on an unreliable piece of data (which in this case, is Market Samurai’s SEOT) is a very bad move.

And I *really* don’t think you can say “forget the numbers” when it comes to keyword research. Keyword research is ALL about the numbers. Would you spend 6 months trying to rank for a keyword that attracts 20 searches a month? Of course not. Numbers are an integral an inescapable part of keyword research.

As for relevance being a key factor when it comes to SEO in general, I couldn’t agree with you any more :)

I really appreciate you giving us your thoughts.

All the best,

Tom

Shamelle@BetterBloggingWays November 17, 2011 at 9:56 pm

Understanding how difficult it is to rank in the top 10 of Google for your chosen keyword is probably the most important skill you can acquire when it comes to building niche websites.

Unfortunately, it is also the most difficult skill to learn :-(

Tom Ewer November 18, 2011 at 1:43 am

That’s usually the way it goes Shamelle! :)

Rose November 19, 2011 at 8:28 am

That slingshot survey sounds about right. I think if you are on the first page, you shouldn’t expect more than 15%, especially if there is a lot of adwords above you, and the adwords box is a particularly pale yellow (so you can barely see it).

Anodized Aluminum Cookware HQ November 21, 2011 at 6:46 pm

Thanks for the information. I really liked the explanation of the 42%. I always wondered if it was reliable or not. What interests me is the fact that the information is just coming to light now. I suppose the information was out there but not available. I am trying to get a keyword ranked for a niche site and this information has helped me understand that I need to rethink my keyword searches when I am on google adwords. Anyways, thanks a bunch for the info.

Jeffrey Trull November 22, 2011 at 7:10 am

Great post, Tom! I think this assumption for the #1 position and the click through rate is something a lot of us that pay attention to SEO assume is factual. Your research here dispels that. I think this highlights the importance of the title and meta description tags even moreso knowing that these play a huge role in getting people to click on your search engine result.

Fernando @ careers blog November 28, 2011 at 1:41 pm

Awesome post, Tom

Specially the SEOT data in Market Samurai, I used to swear by those numbers -until 2 minutes ago-.

I guess I’m back at looking at search volume and competition to choose from which keywords to target.

Alok November 29, 2011 at 3:31 am

Absolutely valuable post. Keyword research is a vital and the most primary step in internet marketing. We just can’t ignore this, because it is the crucial element of on-page optimization. You have explained the keyword research in an analytic way which is new to many online marketers. Thanks for sharing a valuable piece of information.

Tom Ewer November 17, 2011 at 10:39 am

It was my pleasure Corbett – thanks for the opportunity :) and thank you Daniel for the compliment!

Comments on this entry are closed.

Sites That Link to This Post