How to do an SEO Audit

 

 

How to do an SEO Audit

In 2017, SpyFu’s SEO ranks plummeted, and we didn’t know why.  We decided it was time to get down and dirty with a massive SEO Audit.  This video will go over the tools and techniques we used to improve our SEO ranks.

If you’re unfamiliar with MySpyFu here’s a link to the MySpyFu video curriculum *cross-linking ftw!*

Video Transcript:

My name is Patrick and I’m from SpyFu.com.  Today, I’m going to talk about how we recently did an SEO audit for our site.  Things we learned, mistakes we made, and hopefully give you some pointers if you’re ever wanting to do it for your site.

And I think you should.  Because it’s working.

This is a different type of video because we used a variety of tools and software for this, including but not limited to SpyFu.  Good tutorials on how to use some of these 3rd party sites already exist, so I’ll just touch on what they are and how we used them in this video.

Part One – SEO Calamity:

Why did we decide to do this massive SEO overhaul?  It’s simple. Our organic ranks started tanking, we started losing a lot of traffic and revenue.  

Not going to lie, we kind of freaked out.  We wanted to find the culprit, and what we found was a bug on our end dealing with

 

Robots.txt

 

Robots.txt basically informs google, and other web crawlers, to index pages on your site, so people can find them.  It also allows you to tell google what sections of your site you DON’T want them to index.  

We accidentally set a bug in motion that basically told Google not to index any of our main site.

This bug, which was the programming equivalent of turning off the wrong lightswitch, told google to deindex hundreds of thousands of pieces of content pretty much over night.

This lasted for 5 days without us realizing it.  We found the problem and reverted it back, but the damage was done.  We went from 500,000 pages indexed to 100,000.  

We got some of that traffic back. But not all of it, and it was a crushing blow, and an important lesson.

 

Lesson One – Stay Vigilant

Our SEO has been fairly consistent over the last decade without really applying best practices to all of our content.  When you take that approach for a long time, things start to fall through the cracks, including a small bug that tanked our rankings. We thought that what we were doing was good enough, and that was our first mistake – one we didn’t want to make again.

We started by implementing a tool called

 

Robotto.org

 

It’ll monitor your robots.txt for changes and alert you. It’s free, and it’s rad.

But we didn’t stop at fixing the robots.txt, we instead decided that it was time to strike from all angles.  Do everything we could to get our SEO in good shape.  

The first thing we tackled because is was glaring low hanging fruit was alt text

 

Alt Text

 

We started by adding alt text to every image, on the main site.  Alt text is a short, literal, description of an image.  

We use WordPress for our resources site.  In your wordpress account you can easily go into Media, Library, Click on an image, and add the alt text right there.

And once all of that was done, we decided to analyze every piece of content on resources.spyfu.

We started with a sitemap, that we got from a WordPress plugin called

 

Yoast

 

With Yoast it’s very easy to save an XML file of your entire sitemap.  

We fed this XML file into Screaming Frog.  Screaming Frog returns a ton of data including Article Titles, H1s, the specific URL of the content, and even if there’s any existing canonicalization.  

We exported the Screaming Frog CSV and fed THAT into URL Profiler.  

This tool gave us new information including free backlink stats for each page, and a publish date.  It feels strange that the other tools didn’t provide that, but a publish date is HUGE when dealing with aging content.

After we had this snowball of data we put the entire thing into a google doc.  

 

Google Docs

 

In general we are big fans of google docs and google sheets.  It’s free, and allows multiple people to work on one document or spreadsheet at the same time.  Collaboration was key for a project of this size.  

For each piece of content there was a row.  For each data point for that content – a column.

To be honest, we didn’t use most of the data.  We used a handful of key columns and added some more as the project went on.  But we didn’t delete any data either; since we were doing the audit anyway, we wanted to be thorough.  

With our spreadsheet all set up, it was time to make sure that our content was optimized for SEO.

Okay, “optimized” might not be the right word.  

 

Lesson Two – Perfection is a Trap

Our goal in this wasn’t to be perfect, but we wanted to be ‘better than yesterday’ which is a work ethic of ours.  If your goal is to be perfect, you run the risk of shooting your productivity in the foot, because you’re focusing on something that might be unattainable.  But if you are simply ‘better than yesterday’ you can improve your site as a whole, in less time.

This approach was pivotal to this project.  We had to sift through almost 700 pages, with 50 data points each.  Being perfect wasn’t gonna happen.

After this massive document was created we knew that we were going to have a problem.  In google sheets, anything you change in the document will be changed for everyone.  If you add something, delete something, or even sort, that’s how the document will be for everyone.  

We found a solution called

 

Filter Views

 

Which allow you to set up views for individual users – like sorting and filtering.  If you add or delete data, it works like normal, meaning it’ll change for everyone.  But individual filtering and sorting  was an absolute game changer for us.  

I needed to focus on the content that had more than zero videos on it.  Sidra needed to filter those out so she could focus exclusively on all the articles she’s written.  

After we realized this, we added a few more columns to help us organize this list.

I think this is a good time to jump to

 

Part 2 – Prep Work

Prep work.

Doing prep work is miserable, but it was easily the most important part of this process, and so a big chunk of this video is going to be about… prep work.  If you want to skip past this part, simply fast forward to **time stamp**

Because honestly, THAT’S WHAT WE WANTED TO DO!!

We wanted to jump in, article by article, and just fix it.  Do the work, and fix it one at a time.  This might make sense for a smaller project, but we were looking at hundreds of posts that needed analyzing and improving.  Doing it post by post would’ve been more enjoyable, but taken three times as long.

The best way to make it manageable, as well as to bring other people in to help with the project, was to prep it iteratively.

 

For us that meant adding a column that:

  1. Identified, in simple terms, what “product” the post was about.  Was it about SpyFu Kombat, SpyFu Recon, General Internet Marketing Insights?  We specialize in a variety of things, and when we write content we generally like to focus on one “product” so people can find it easier. Having this extremely simple column helped us sort the content into content groups.
    1. When you sorted by both product and date, you could easily identify what was old and needed to be canonicalized or 301 redirected to something newer and more relevant.
    2. Since multiple people were working on adding these labels, they weren’t written exactly the same every time, which screwed up the sort (which was the whole point). Not a big deal, just had to go back a couple of times and standardize them as much as we could.  
  2. We also added a few very simple columns including –
    1. The initials of the person who would fix the content
    2. A description – which eventually just became notes that we wanted to make about the content
    3. And a column to X off when the work was done.
  3. We then identified “what to do” in a column.  Which mostly boiled down to if we should 301, Canonicalize, or do an SEO Review of the content.

These labels were extremely important to the fate of the content, and the eventual SEO map of our site.

301 is the harshest. It means if someone clicks on an article, or URL, it will be redirected to another page.  This redirect means that it’s impossible for anyone to see the original content ever again.  You’re basically telling people, AND Google, “this content is worthless and outdated, I don’t want you to see it, I want you to see this instead.”

Canonical is a much softer approach.  People can still get to the article if they click it, but you’re letting google know that there is something better.  Canonicalizing tells google, “this content is still somewhat relevant, but we have something better.” And google will give the link juice to that better piece of content.  We did this with most of our stuff, and so far it seems to have been the right move.

SEO Review was the label we used to identify content that was still relevant and good.  We just needed to make sure it was SEO friendly, and if it wasn’t up to snuff (which most of our content wasn’t) we needed to go in there and fix it.

 

We’ve been asked, “How did you determine what needed to be done with each article?” and I really wish I had a formula for you.  But honestly, that part was mostly gut.  Sidra and I have been the primary content creators for almost a decade.  So if my gut said that a product video was currently irrelevant, I’d label it to be canonicalized or 301’d.

If you have an SEO team, or a third party you want to do all of this, I’d recommend you identify the date of your last site redesign, and have them canonicalize everything before that date.  If the link is broken, then 301 it.  

 

Lesson 3 – Be patient, stay the course

Before I dig into the tools we used – I want to say, this didn’t happen in a day.  We went through each piece of content several times, over the course of weeks to identify and label these simple rows one by one.  Again, during this time we didn’t actually do the work of reviewing or redirecting, even though we wanted to.

When sticking to this “assembly line” approach, we were eventually able to fill out the necessary information, so we’d be extremely efficient once we got into the actual work.  As well as delegate some of the more basic tasks to people who weren’t intimately familiar with the content.

This turned out to be the correct approach.  When it came time to actually start the work, we were able to bring Zac on to do things like 301, redirects, and cross linking, on his own time.   

 

Part 3 – The tools we used

I mentioned earlier that we use WordPress for resources.spyfu.com.  

Using the Yoast plugin for WordPress felt like an SEO cheat sheet.  It somewhat gamified SEO for each article.  Asking what keywords we wanted to focus on and then giving us pointers on how to have that article rank better for those keywords.  

It’s free for most features, and crazy affordable for the premiere version.  We probably couldn’t have done this as efficiently without Yoast, and I can’t recommend it enough.

If you’re not on WordPress, but know of a similar tool for other sites, please put it in the comments below.  Help other SEO noobs like me

After we started working with the articles themselves we realized that we needed to add a couple more columns to the sheet.

 

Part 2(again) – Ah crap more Prep Work

  1. Cross-link – which let us know what content we should interlink to within our own site.  Google loves these cross connections, especially when they’re relevant.  So we would add a link to the existing text in the content, or add a sentence or two to get those links in there.
    1. You want to make sure that the words you’re adding a link to are relevant to the content it’s linking to.  Don’t just flood your content with random links attached to random words, Google does not like that.
    2. At this point, external links weren’t our priority, only URLs within SpyFu.
    3. For every piece of content we double checked that it also linked back to the product page of our site.
        1. This is a definite regret of mine.  When our URL structure changed a few years ago, I never went back to my old content and linked my tutorial videos (which were still being watched) to our current product pages.  That simple upkeep could have vastly improved our SEO, which probably would’ve lead to an increase in revenue.
          1. #regrets
  2. Focus Keywords – This is the keyword that we told Yoast (and in turn google) that we wanted to focus on.  

 

Fortunately, finding and tracking the best keywords in an industry is our specialty.  So let’s finally talk about SpyFu!!!!

 

Part 4 – Using SpyFu for SpyFu

We used the same techniques we recommend to you!  

First we went into SpyFu Kombat and entered in our 3 biggest competitors.  We then looked at their “core starting keywords”, which are the keywords that all three of them share.  From there we could cherry pick the best and most relevant keywords and add them  to a MySpyFu project.

MySpyFu automatically starts tracking these keywords, and how their ranks change for our domain.  You can do this with your domain as well!

If you’re unsure what MySpyFu is, I’ll include a link to the tutorial videos in the description below.  

**CROSS LINKING FOR THE WINNNNN!!!**

The joy is that you can go to any research page on SpyFu searching for keywords, and add them to this project.  SpyFu will let you know if you’ve already added them, so there won’t be any duplicates.

This MySpyFu project now includes these keywords. We then separated those keywords into different groups, so we could track them easier when the SEO audit was done.  

The hope was, that after all of this work, we’d see improvements on these groups of keywords.  We knew that it shoullllld, but weren’t sure that it would.    

SEO is scary like that.  So we started watching the weekly tracking feature like a hawk, praying that eventually our ranks would increase.

I hate to spoil the ending, but they did increase.  I probably wouldn’t be making this video if nothing happened, but so far, it’s looking like this crazy audit has been worth it.  

Setting up this project before we did the actual work was a smart move too.  It gave us a benchmark of what our ranks were before we started.  From that point we could really see how much this SEO overhaul moved the needle.  

After we had the keywords ready to go, we exported them and put them into another Google sheet for shareability and ease of use.  

Finally, we got around to the actual work.  Here’s how it went down.

 

Part 5 – The Actual Work

We set up a variety of filter views, in the Google doc.  For me, I filtered out everything that didn’t have a video, and that were listed as canonical or 301.  Meaning I only got pages with videos, that were ready for an SEO Review.  I then sorted this list by product, this was helpful because it made cross-linking easy.  Here are the 3 pieces of relevant content about Backlinks, have them crosslink to each other.  

I put my initials on all of those videos, so if I ever needed a different view, I could simply filter by my initials.

This is why that prep work was important.  Thanks to all the labeling, I finally had a list of content that was tailor made for my eyes. 

 

Let’s do this step by step

  1. I’d open the URL from the Google doc.  
  2. I’d browse the content, maybe watch the video, and then search the Keyword document for a keyword that I think would fit.
  3. I was already logged into my WordPress account and so I’d just click edit up at the top.
  4. Add in the cross links, unless Zac had already done it.  
  5. Add the focus keyword into Yoast, and then mostly followed Yoast’s instructions!
    1. Adding the focus keyword to the first paragraph of content
    2. Sometimes changing the SEO title if it looked sloppy
    3. Almost always changing the snippet, occasionally just copying and pasting the first sentence or two from the article itself – because it already contained the focused SEO keyword.
    4. And then clicking update it until it said the SEO was “Green”,
      1. or at least yellow.

 

Lesson 4 – Take the easy road

If the content was too complicated, for example if there were 20 pieces of content for one subject, or the video was kind of old but mayyyyybe not too old. I’d skip it.  Sometimes I’d leave a note about why I skipped it.

We weren’t sure if this was the best way of doing things at first, but now we highly recommend this approach.  

It’s too easy to get lost in indecision; thinking too long or hard about one article will just halt progress.  If you allow yourself to skip anything too complicated you can maintain your momentum of work.  

Of course, this meant that I had to come back to it eventually.  But by that time I was more familiar with the content, and I sometimes I knew exactly what to do on the second pass.  Sometimes I was still stumped and so I’d skip it again.

Unfortunately by the very end, the only content I had left was the really complicated stuff.  The stuff I skipped over several times.  It did take a lot longer to fix up those articles, but it was easier because I knew I was on the home stretch.  

I’ve mentioned a couple of times before, but we also brought our co-worker Zac into the project.  While Sidra and I were reconstructing the SEO Review content, we had a boat load of 301s and Canonicals to take care of.  

These didn’t need things like focused keywords, or expansive cross-linking.  So Zac used his filter view and knocked them out one by one.  

 

How is this done?

 

You can canonicalize things right in Yoast, under the advanced settings.  Simply paste the URL of the article you think is more relevant!

In WordPress the 301 redirects are found under tools, and then Redirection.  Scroll to the bottom, enter in the URL you don’t want anyone to see anymore.  And then enter the URL of where you want them to go instead.  

Doing all of this we were able to work on this thing collaboratively and autonomously.  Occasionally we had to discuss what to do with one tricky piece of content or another, but for the most part we just churned away at it.

Do the work, refresh the filter view, repeat until everything has that X marking it as complete.  And that was it!

 

Part 6 – What comes next?

Now it’s about tracking those keywords, and upkeep.  Because honestly, this sucked, and I never want to do it again.  

Making sure every new piece of content is SEO friendly before we post it is going to be a standard SpyFu practice from now on.  

One big regret I have is that a lot of the URLs for these pages weren’t SEO friendly.  You don’t really want to change the URL of existing content; but moving forward I’m definitely going to pay much closer attention to it.  Either structuring the URL to better explain the content, or include the focus keyword – hopefully both.

 

But here’s the thing.  It’s working.  

Our overall SEO numbers are slowly improving.  The ranks on our focused keyword groups are increasing across the board.  We weren’t sure if that would be the result and it’s legitimately thrilling and heartwarming.  

And the joy of SEO is knowing that work will have a positive impact FOREVER.  Months of work for years of payoff.  

On a more personal note, we learned a lot.  Both on the struggles you guys have to face on a day to day basis, as well as how we can improve SpyFu to make it better for you.  

 

I know some of you out there are screaming at your monitor that there was a better way go about this.  If you are that person, please leave a comment below with any tips or tools that make this process better or more bearable.  

Thanks for sticking around for this long video!  I hope you learned something from our experience, and can apply the lessons we learned to improve the SEO of your site.

As always, thank you for watching.

 

  • Donald Penix

    Very informative! Thank you for making and sharing this video.

    • AgentSidra

      There was such a strong reaction to the original webinar, that sharing it in a tighter video guide made sense. Thanks for checking in, Donald!

  • Tim Hill

    We don’t use WordPress because we feel it it a little too closed of a platform. It doesn’t make sense that the WordPress/Yoast combo is the only option to do what they both do together. We us Joomla and a few others depending on the needs of the client. What exactly does Yoast do that other components can’t?

    • AgentSidra

      That’s fair, Tim. I hope this didn’t come off as an endorsement where you should shift gears to WP/Yoast. It’s more of a suggestion for those who might be looking for a platform and are open to ideas. We’re just sharing our own experience. There are certainly limitations like what you described, but we do like some of the built-ins like readability score and the SEO tools (like meta description tool, easy re-directs, easy canonicalization tools, etc.).

      Can you tell us a little more about what people might expect with Joomla? We’re always open to learning a better way to do something.

      • Tim Hill

        Not at all. My comment wasn’t directed at that part of the article as much as the fact that WP seems to have grown unstoppable legs and everyone’s nephew acts as if they are a website developing genius because they know how to change a preconfigured template. I guess my point is lets start having discussions about other platform SEO components to let newbs know they shouldn’t use WP as a crutch or be afraid of learning HTML and CSS.

        • AgentSidra

          Ah, got it! And yes, that’s good insight. Don’t let the easier way keep you from growing into more flexible platforms.

          Thanks for elaborating, Tim!

  • maryfrancis

    love it.. I never used Yoast because I didn’t want to leave my on-page SEO up to a plugin but now that I see how you use it I can give it another try.

    • AgentSidra

      Great point. It might be better if we describe it as a guide. Yoast isn’t doing any work for us other than giving us an objective guide. It’s a lot like parking cameras on a car. They help you see when you’re in good shape vs having to turn the wheel, but they don’t parallel park for you.

  • leegrabow

    Love this! Too often, we get focused on off-site link building or making sure we’re writing the next blog, when we have tons of solid content in need of a refresh. Or, we’ve cannibalized our blogs in the SERPs, not bothering to add canonical URLs or make basic improvements to UX (What good is driving traffic to your site, if the page isn’t engaging to users?). This audit gets back to basics, and I love that you’re not afraid to be transparent about your own experience with SEO mishaps, testing, and ultimately, winning.

    • AgentSidra

      You nailed it! We were treating our content as individual pieces instead of part of a whole system. Thanks for helping us see that others are in the same boat. Good for you for investing in your site improvements. I hope it pays off well for you!

  • https://www.newmanwebsolutions.com/ Lori Newman

    This is awesome! We use Yoast ourselves (love it) and went through a similar situation where our site started to tank – it was due to a couple of reasons – don’t want to throw anyone under the bus but hosting issues and CDNs played a huge role for us. Sure makes you go in panic mode when you see all your hard work start to plummet. Definitely going to check out the robotto tool – looks like it’s worth having. Thanks for sharing!

  • http://www.pets24.co Rona Ben David Elbaz

    That’s great! Thank you for sharing! If possible, I’d love to see the spreadsheet’s columns you’ve been filling out. Thanks again!

  • pinchvalve

    Excellent advice, as a manager, this provides a few great bullet points that I will hang on my wall to keep the team on track!