Keyword Research and Competitive Analysis

Tools

Concept

Creating a Comprehensive Keyword List

Using SEMrush to Research and Improve Keywords

Swiss Made Marketing

Using SpyFu

Creating an “All Terms List”

Competitive Analysis

Using Swiss Made Marketing For Competitive Analysis

Using Link Research Tools for Competitive Analysis

The Nuts and Bolts of Competitor Analysis

Competitive Analysis For AdWords

Creating a Mind Map

Conclusion

Keywords are the foundation for any PPC, SEO or Outreach campaign.  The keyword research competition scene has changed drastically in the last few years as Google’s algorithms change to favor authoritative brands and become infinitely more sophisticated with their capacity to understand context.   When working on SEO keyword research competition, looking at keyword volume and long tail opportunities is by no means enough in the current landscape.  Understanding how to perform competitive analysis is a fundamental aspect of keyword research competition.  What’s the point of choosing keywords with high volume, unless your site has the right metrics to compete?

In this article, we’ll walk you through the process of keyword research competition from the point of view of SEO.  We take into consideration not just the process of finding keywords, but also qualifying them based on the relevance and competitiveness.  We’ll teach you the different metrics to consider when analyzing keywords, including the best tools to use and a step by step process.

Tools

Here are the tools you’re going to need:

Concept

When brainstorming for keywords, where should you start? For most industries, unless you already have a list of tried-and-tested keywords from Adwords, your first step is Persona research.  You need to understand your industry, and audience, incredibly well and figure out what they might be typing into the search engines to find relevant products or services.  These terms will give you your jumping board – a place to start the process of keyword research competition.

Before you go much further, you’ll need to understand the theoretical construct around keyword research competition.  You can’t simply choose a couple of very broad, untargeted terms, nor can your keyword research competition just be a list of words pertaining to the EXACT meaning of what you offer.  Think in keywords in terms of BUCKETS.  Each bucket represents a concept, and within each bucket you can hold a number of specific terms related to that bucket.  To illustrate the concept, let’s use the term Weight Training as our Bucket term.

keyword research competition

Now we’re going to think of Head Terms that are related to our Bucket Term. Head terms are short, 2-3 word phrases that are contextually relevant to your main term.  What’s important to do here is to think laterally instead of vertically. We are not so much as trying to get more in depth with our Head Terms but instead expanding the reach of the Bucket Term.

keyword research competition

As you can see, Head Terms aren’t variations on our Head Term, but topics closely related to our Bucket Term.

Next come Long Tail Keywords and Body Keywords. Body Keywords are 3-4 word terms that have high volume and are closely related to your “Head” or “Bucket” terms.  Long Tail terms are 4+ word terms that have low volume and are semantically related.  The key here is that you may only have a small number of body terms that comprise a large volume of searches each, whereas you may have thousands of long tail terms, each with only 1-2 searches / month, that together create massive volume.   Long Tail terms is where you achieve DEPTH.   These groupings need to be narrowly focused and as closely related to their head term as can be.

For example a selection of Long Tail and Body Keywords for the Head Term “Calorie Intake” would be:

keyword research competition

The difference between Body Terms and Long Tail Terms is how narrow their targeting is. So a term like “calorie calculator” could be classified as a Body Term because there are variations on it that could narrow its focus, like “medical calorie calculator” or “youth calorie calculator”. However, “calorie intake for muscle building” is much narrower in its targeting. These terms are usually comprised of more words and there are very few areas, if any, where the term could become more specific.

What Body Terms lack in focus they make up for in reach and vice versa for Long Tail, so they both should be included in your marketing and SEO efforts.

It is okay for a head term to be a keyword in its own group. For PPC marketers, you can think of the Bucket > Head > Body > Tail structure as the Account > Campaign > Ad Group > Keyword structure.

This concept of broad to narrow groupings and aggregating our keywords by how they are related is crucial to understand before getting on with the practicalities and the tools for keyword research competition and competitive analysis.

Creating a Comprehensive Keyword List

The first stop that we’re going to make is with Google’s Keyword Planner Tool. This is where we are going to begin the process of compiling our list of potential keywords so we can begin conceptualizing and brainstorming.

We are going to select Search for new keyword and ad group ideas.

keyword research competition

Google will let you enter your landing page, a product or service (bucket or head term), or product category. You can input a landing page or product category if you would like, but we’re going to focus on the the keywords. So we’re going to input our bucket term which will be “google penalty removal”, a service provided by Gryffin.

keyword research competition

When we click on Get Ideas we get a graph that shows the average monthly searches for the keyword. As well as “Ad group” and “Keyword” suggestions and their competition and search volume.

keyword research competition

When judging relevancy you want to make sure that someone who is interested or could be potentially interested in your service might use that term in Google. You should also look at the Avg. monthly searches and competition metrics.  When looking at competition data, remember that this is specific to ADWORDS, not SEO.  So a keyword may have a very high bid price, but have low competition in Google organic.

We’re going to now start creating our plan.

Under the Ad group ideas you want to find ad groups -head terms- that are related to your service.

keyword research competition

Click through to the relevant ad groups.  If you see keywords that are relevant and you’d like to add, then click on the arrow keyword research competition next to the relevant keywords.

When you’re done searching for keywords from 1 ad group, then move on to the next ad group.  However, instead of adding the Ad groups to your plan, look to the right under your plan:

keyword research competition

and add your own ad group with the keyword research competition icon. Then type in the name of the ad group.  If you don’t add your own ad group, Google will automatically choose the first ad group that you selected, and you’ll lose the closely related targeting available.  As you evaluate keywords, add all of the keyword that seem relevant.   Don’t worry about quantity yet, quality is what you’re aiming for when adding keywords to your plan.

keyword research competition

Once you have gone through the ad groups and added your keywords. Your plan should look something like this:

keyword research competition

Now depending on how broad your service or products is you can have fewer or more ad groups with fewer or more keywords. But the ad groups should all be narrow and specific. Don’t be afraid to add more ad groups, the more specific the better.

The “Ranking Dropped” ad group is a good example of thinking laterally. When someone is hit with a Google penalty, what are they going to experience? A drop in their page’s rankings, so the audience represented by these keywords are looking for google penalty removal services before they even know that they need Google penalty removal services.  You’re reaching them at the top of the funnel based on their knowledge and ability to search about your products or services.

After you have finished your plan you are going to export your keywords and ad groups by clicking on the keyword research competition icon.

keyword research competition

Here you will be able to export your plan to an CSV file. If you are working with AdWords, you can include a bid in your plan which will give you traffic estimates, and export to an AdWords Editor CSV which will allow for import into AdWords. Here we’re just going to download an excel CSV.

Now if your prefer to work in Excel that is okay. I will be using Google Sheets throughout this piece.

So your data will look like this:

keyword research competition

You can remove the Currency, Segmentation, Suggested bid, and Impr. Share columns, they will not be necessary.

And now we have our initial keyword list:

keyword research competition

Next we will begin the process of researching these keywords in more depth.

Using SEMrush to Research and Improve Keywords

SEMrush is a very useful tool for finding keywords and examining competition, which is what we’re going to do now. So in your spreadsheet we’re going to add another sheet titled Competitors.

keyword research competition

Now we’re going to take our most relevant keywords from our keyword list and put them into Google. In this case I’ll use “google penalty service”

keyword research competition

Now I’m going to take the URL’s in green from the high ranking pages and paste them into my competitors list. So it will look something like this:

keyword research competition

Now I’m going to take one of my competitors from this list and put their URL into SEMrush.

keyword research competition

SEMrush will now give us valuable information about our competitors including traffic, if they have ads running, which countries they’re ranking for, and -what we are looking for- the keywords they are ranking for. For some sites that only rank for a few keywords, SEMrush won’t give that much info because there isn’t that much to report.

Now we’re going to click on keyword research competition.

This will give us all of the keywords that this website is ranking for according to SEMrush. This keyword list is extremely lucrative as you’re gaining direct insights into the keywords that competitors are targeting.

keyword research competition

It is important to look through this list and see if there are keywords that relate to your head terms. You can do this by looking through the entire list, or by adding a filter. If we click on keyword research competition we’ll get this:

keyword research competition

So now I’m going to filter the keyword results so I only get keywords that contain “penalty” in them.

keyword research competition

So now we see that this site is ranking on three of the keywords that we are going after. So I’m going to hit keyword research competition then choose CSV.

keyword research competition

Just like we did with the last report. I’m going to copy and paste the information into Google Sheets.

keyword research competition

All of this information is going into a new sheet labeled SEMrush. It’s okay to only have the keywords in the sheet, but if you think the other metrics will be useful then you can go ahead and leave them.

SEMrush also gives you keyword suggestions based on keywords as opposed to a website. So if in the same search bar we input a keyword instead of a website, in this case “google penalty” we’ll get:

keyword research competition

Now SEMrush will give us a Phrase match report and a Related keywords report. They will also give us search volume for the keywords. We can export either report if they have a lot of related keywords.

SEMrush will also give us domains that are ranking organically with the keyword and domains that are advertising on with the keyword.

keyword research competition

So it might be useful to repeat the process we did with our competitors with the domains in these lists to get more keyword ideas and information about the competition.

Now that we have gotten a few more keyword ideas from SEMrush, my list looks like this now.

keyword research competition

Now we’re going to expand and improve this list with a new, particularly powerful, tool.

Swiss Made Marketing

Swiss Made Marketing aka SECockpit is a uniquely powerful tool for not only getting ideas for new keywords, but also information about keywords already in our list.

After logging into the tool, we’ll be greeted by this page:

keyword research competition

The process of using this tool will be similar to SEMrush. What we’re going to do is click on this keyword research competition icon to create a new keyword search.  I’m going to use “google penalty removal” for this keyword search:

keyword research competition

After we hit keyword research competition the tool will start generating a keywords report, this usually takes 1-3 minutes. Once the report is finished, it will look like this.

keyword research competition

Before we start gathering the keywords, we’re going to want to make a container to keep them all in. We can do that by adding a new folder by clicking this keyword research competition icon. This will create a folder where we can drop all of our keywords in. I’m going to name my folder after the keyword that I used to generate the list, “google penalty removal”

keyword research competition

You can choose a unique icon for the folder if you want. Once you’ve named your folder you can hit keyword research competition to create your folder. Now you should see your folder in the left column of the page.

keyword research competition

Now that we have our container for the keywords, we can start dropping some in there. If we take a look back at our generated keyword list, we should arrange by monthly searches descending. keyword research competition This ensures that we get the most related and the most searched for terms first.

keyword research competition

Now you can go through the list and simply drag and drop your potential keywords into your newly created folder.

Once you have completed your dragging and dropping, your folder should now be filled with new keywords.

keyword research competition

You can repeat this process with new keyword searches, and you can add new folders if need be. Once you are done adding keywords to your folder, we’re going to export our list by clicking keyword research competition and choosing your prefered format. Once again, this information is going to go into its own sheet labeled “Swiss Made Marketing”, or “SMM” for short.  You can remove any columns that won’t assist you.

keyword research competition

Using SpyFu

The next tool we are going to use is SpyFu.

keyword research competition

SpyFu works really well at giving you competitive keywords and allows you to search for both keywords based on a head term or website. We’re first going to click on  keyword research competition.

This allows us to search for keywords based on one keyword (or several) and give us more information about traffic and cost if we’re going to be advertising on these keywords. I used “google penalty removal” for my smart search and found this:

keyword research competition

Now we’re going to want to sort by Global Daily Searches by clicking on keyword research competition. Make sure that the numbers are descending. Now we can look through this list and check mark the boxes next to new potential keywords.

keyword research competition

Once you have selected all your keywords, you can click on keyword research competition to export your keyword list to excel. Make sure to choose Export Selected to CSV so you don’t export the entire list SpyFu gave us, only the ones we selected.

Create a new sheet labeled “SpyFu in your spreadsheet application and place your keywords inside.

keyword research competition

You can also take a name from your competitor’s list and search for keywords on SpyFu that way. Simply take your competitor’s site and put it into the search bar.

keyword research competition

As you can see, we get a similar list of keywords and we can follow the same process of selecting and exporting to add these keywords to our research.

You can do this with as many keywords and competitors you like, it will be easy enough to remove duplicates later.

Creating an “All Terms List”

Now that we have sheets of possible keywords from all of our different tools, we can consolidate them into one master list of keywords. We’re going to create a new sheet labeled “All Terms”

keyword research competition

Next we’re going to into our other keyword lists and add a new column labeled “Status”  here we’re going to plug in a simple “Yes” or “No” as we go through our keyword lists one last time and exclude any that are not related, or have too poor of metrics for your objective.

keyword research competition

Don’t worry about catching duplicates here just yet. We’ll take care of this later. Once you have filled out the “Status” column you are going to copy the “Yes” keywords and move them into your “All Terms” sheet; just the keyword will suffice.

Tip: To make your copy and pasting simpler, you can right click on the “Status” column once its filled and choose “Sort Sheet A → Z” Make sure that the heading row is frozen before doing this though.

So your keyword lists should look like this after you have judged their statuses:

keyword research competition

and your “All Terms” sheet should look like this:

keyword research competition

Now we can take care of duplicates by right clicking on the first column and sorting by “A → Z”. This way any duplicates will be shown right next to each other and we can easily remove one of them, or more if there are multiple copies of the same keyword.  Google Drive has a free add-on tool called “Remove Duplicates” that you can use.

Once we have removed duplicates, we are left with a clean list of potential keywords that we can do competitive research on.

keyword research competition

Competitive Analysis

Now that we have our keyword list, it’s time to research how much competition these keywords have. We’re going to do this with Swiss Made Marketing and Link Research Tools.

Before we cover how to use these tools, let’s take a look at the important metrics to consider when looking at competitors’ metrics:

  • Power Trust – LRT’s version of Pagerank
  • Page Rank – Google’s value on link power. This is a deprecated metric and will soon become obsolete.
  • MozRank – Moz’s version of page rank
  • Page Authority – Moz’s version of the authority of the page.
  • Domain Authority – Moz’s version of the authority of the domain.
  • Juice Backlinks – The number of unique links going to the page
  • Linked Domains – The number of links to the domain
  • Total Backlinks – The total number of backlinks to the page
  • Internal Site Links – The number links in the domain that point to pages in the same domain
  • BLdom: Backlinks to the domain
  • DomPop: Referring domains from unique C classes
  • Link Velocity (LV) – How quickly the site is building links (Data is aggregated by past months)
  • LVT: Link Velocity Trends
  • SWR: Sitewide Ratio for links
  • DLR (Deeplinks) – Ratio of how many links go to internal pages as opposed to the homepage.
  • DomCreated – When the domain was created
  • ACrank – Majestic rank (similar to MozRank)
  • Glidx – Number of pages indexed in Google
  • TitleRank-home – the position of the linking homepage’s title in Google’s search results
  • Alexa – Amount of traffic to the site.
  • DomPopEdu: Links from .edu domains
  • DomPopGov: Links from .gov domains

These metrics are important in establishing whether or not you will be able to compete with other domains for the keywords you’re targeting.

Using Swiss Made Marketing For Competitive Analysis

Swiss Made Marketing has some great tools for competitive analysis. Now we’re going to click on this keyword research competition icon to generate information about our keywords. When prompted with this:

keyword research competition

Just simply copy and paste our keyword masterlist inside and click on keyword research competition,

keyword research competition

It will take the tool a few moments to get the information, but once it does we’ll see this:

keyword research competition

Now we can export this with all the metrics we need, but before we do that let’s click on one of the keywords:

keyword research competition

Here we’re going to get a handy analysis summary of our keyword. This displays the top ten ranking pages in Google for the keyword. We can observe their mozRank, Page Authority, Domain Authority, Juice Backlinks, Linked Domains etc. for the pages.

Red means that that page has a high ranking for the metric, and therefore will be extremely difficult to compete with.

Orange means that it’s possible to overtake the page, but it will be difficult.

Green means that overtaking the page in this area is possible.

You’re going to want to observe this page to get a better understanding of your competition for your keywords.

If we go back to our keyword list, we can export it just like we did our keywords originally.

You can navigate between windows via the tabs at the top of the page.

keyword research competition

Now that we are back to looking at our keywords

keyword research competition

Let’s hit  keyword research competition and download a spreadsheet of this information. Now we can copy this sheet and put it into our spreadsheet under a new sheet labeled “SMM Top Keywords”:

keyword research competition

Here we’re going to examine the metrics above for our keywords and determine which ones have available “spots.” Spots meaning pages in Google’s top ranks where you have a shot of overtaking due to your better metrics.

We’re going to now create a new sheet labeled “Target terms” and in there we are going to put the keywords with potential spots for our page.

keyword research competition

I also included an “Observations” column that has information on how to rank in these positions.

For reference, here are all the metrics that are provided by Swiss Made Marketing and what they refer to.

  • Words: Number of words in your keyword
  • Monthly Searches: Number of times the keyword is searched for in a month.
  • Rank 1-10: How difficult it will be to rank 1-10
  • Visitors 1-3: The number of visitors you can expect if you rank 1-3
  • Rank 1-3: How difficult it will be to rank in the 1-3 positions
  • Visitors 4-7: How many visitors you can expect in the 4-7 positions
  • Rank 8-10: How difficult it will be to rank in the 8-10 positions
  • Visitors 8-10: How many visitors you can expect in the 8-10 positions.
  • MR Avg: The average Moz rank for the pages ranking for the keyword.
  • MR Max: The page Moz rank of the page with the highest page rank.
  • PA Avg: The average page authority of the pages ranking for the keyword
  • PA Max: The page authority of the page with the highest page authority
  • DA Avg: The average domain authority of the pages ranking for the keyword
  • DA Max: The domain authority of the page with highest domain authority.
  • JL Avg: The average number of links from unique referring domains on the pages ranking for the key word.
  • JL Max: The number of links from unique referring domains from the page with the most juice links.
  • Domains: available domains that can be used for the exact match for the keyword. If domain is (n-e-t for the keyword Gryffin Media, then the available domain is gryffin-media.net)
  • CPC: Estimated Cost Per Click
  • Competition: The level of competition for the keyword

Link Research Tools is a very useful website for competitive analysis with keywords. Once you Sign in you’re going to want to click on Competitive Keyword Analyzer.

keyword research competition

Just like in Swiss Made Marketing, we’re going to paste our keywords from our master list in here. The only problem is that Link Research Tools only allows you to paste 10 keywords at a time. Which only means you’re going to have to run multiple reports if your keyword list exceeds 10.

keyword research competition

After we click on keyword research competition it will take a few moments for our report to generate. Your report should be the newest created here:

keyword research competition

In our report, we’ll be able to see a heatmap based on our keywords.

keyword research competition

But for more specific information, you’re going to want to scroll down under Details.

keyword research competition

Here we will find the top ten ranking pages for Google for the keyword. We’re going to export this list by clicking on keyword research competition. Once again, you’re going to take the information, copy and paste into our own spreadsheet under a new sheet labeled “LRT Top Keywords”

keyword research competition

Now we’re going to follow the same process that we did with the Swiss Made Marketing Sheet and pick out pages that we can do better than and put their keyword into our “Target Terms” sheet.

The Nuts and Bolts of Competitor Analysis

All the work you’ve done to this point is in preparation of having the necessary data in front of you for analysis.

Now that you have a list of keywords with the top sites ranking, you need to delve into all of the statistics to see which keyword has what we call “Open Spots.”  Generally, every keyword will have at least 3 “immovable” spots – these are sites like wikipedia and news publications that no amount of optimization will dislodge.

Your work will entail how many other “Open Spots” each keyword has, and whether the metrics of the pages ranking for those spots are attainable.  Of course you’ll need to have run all of the metrics for your own site and pages so you can use those as a point of comparison.

Then you can look at as much data as you can to make your choices.

To start, you can immediately exclude pages with brand names or big authority, like Wikipedia, Best Buy, Nokia, etc.. These pages aren’t going anywhere so long as the brand is popular.

Next look at DomPop – the number of unique links to that domain.  If your site has 10 DomPop and the average of the competitor sites is  1000, it will take you a LONG time to have enough authority to compete for this term.

Next, look at other link metrics like PageAuthority, DomainAuthority, and LVT.  You want to look for “Outliers” – pages that have significantly smaller metrics than others.  For example, if the average Domain Authority is 50 but one of the pages ranking in the top 10 has a Domain Authority of 20, then this presents an opportunity.  If you then find that the LVT for that site is a very high, positive number you’ll know the reason they’re ranking well with fewer links is because they’re rapidly gaining links.

Don’t forget to look at other factors such as links from .gov and .edu domains, as sometimes these can help give a site higher Trust.  In addition, the average age of the domains can help you understand whether your brand new domain can break in.

Other factors such as pages indexed in Google and social shares can also help you understand the whole story of each page ranking in the SERP’s for the keyword you’re analyzing.

You can also go to Swiss Made Marketing’s and Link Research Tools’ heat map summaries as you go through these lists to help you quickly/easily pick out opportunities.

Once this process is finished, your “Target Terms” sheet should look like this:

keyword research competition

Now here is our final keyword list. Now we need to simply organize the list appropriately.

Competitive Analysis For AdWords

This step is only required if your keywords are going to be used for an AdWords campaign. Here we’re going to go back into Google keyword research competition Planner, but this time we’re going to click on this:

keyword research competition

Then we’re going to paste our keywords into the box and hit

Here you’ll be able to set up the bid you are planning to use with these keywords. Then you will get a detailed estimate of how much traffic you are going to get and the Avg Pos. of your ads. The Avg Pos. metric will be an indicator of how competitive the keyword is. The lower the position, the more competition there will be based on your bid.

keyword research competition

This information is useful and can save you time and money when setting up your AdWords campaigns.

Creating a Mind Map

The best way to organize these keywords is via mind map or word cloud. We will be using Mindmeister.

keyword research competition

We start out our map with the brand or company that’s providing the service. Now we’re going to start thinking about the concept again, Head > Body > Long Tail, structure. So the branch we just created is for “Google Penalty.”

keyword research competition

Now I’m going to go into my keyword list and pull out all the different topics we find in our key word. The Body Terms or Ad Groups. After going through my keyword list my map looks like this:

These are broad topics but specific topics. This grouping might change as you expand your keywords. You might find that some groups will need to be split, and that’s normal. Now that we have our groups. We’re going to go into our keyword list and put our keywords into these groups accordingly.

keyword research competition

As you can see, all the keywords are included based on whether they share words with the group. Some keywords match the group name identically but that’s completely acceptable. With a map like this it will be very easy to conceptualize and edit your keyword lists.

Conclusion

Keywords are the foundation for any SEO improvement and PPC campaign, unfortunately that knowledge of that fact isn’t a secret so it is important to pair your keyword research competition with competitive analysis.

As with most things, the best way to learn is to get your hands in and do it yourself, so give it a try and let us know if you need help!