The Evolution of SEO [Video Presentation]

Posted by RobOusbey

Last November, Moz VP Product, Rob Ousbey, gave a presentation at Web Con 2020 on the evolution of SEO, and we're sharing it with you today! Rob draws on his years of research experience in the industry to discuss how SEO has changed, and what that means for your strategies. 

Editor's Note: Rob mentions a promo in the video that has since expired, but you can still get a free month of Moz Pro + free walkthrough here! 

Video Transcription

Hello, everyone. Thank you for that introduction. I very much appreciate it, and it's wonderful to be with all of you here today. I'm Rob Ousbey from Moz.

Real quick, I was going to share my screen here and say that my gift to you for coming to the session today is this link. This won't just get you a free month of Moz Pro, but everybody who signs up can get a free walkthrough with an SEO expert to help you get started. I'll put this link up again at the end of the session. But if you're interested in SEO or using a tool suite to help you, then Moz might be the toolset that can help.

Also, if you want to learn more about SEO, come join me on Twitter. I am @RobOusbey, and it would be wonderful to chat to you over there.

One reason I put my bio up here is because I've not been at Moz for all that long. I just started about a year ago. Before that, I was at Distilled, which is an international digital marketing agency, and I ran the Seattle office there for over a decade. I mention that because I want to share with you today examples of what I discovered when I was doing my client work. I want to share the research that my team members did when we were in your shoes.

A troubling story

So I wanted to kick off with an experience that stuck in my mind. Like I say, I've been doing this professionally for about 12 or 13 years, and back when I started, SEO was certainly more straightforward, if not getting easier.

People like my friend Rand Fishkin, the founder of Moz, used to do correlation studies that would discover what factors seem to correlate with rankings, and we'd publish these kinds of reports. This was the top ranking factors for 2005. And back then, they were broadly split between factors that assessed whether a page was relevant for a particular term and those that asked whether a site was authoritative. A lot of that relevance came from the use of keywords on a page, and the authority was judged by the number of links to the site. So we would help companies by doing good SEO. We'd put keywords on a page and build a bunch of links.

And I want to tell you a story about one of our clients. This is from just a couple of years ago, but it definitely stuck in my head. We were doing a lot of content creation for this client. We created some really informative pages and some really fun pages that would go viral and take over the Internet, and all of this earned them a lot of links. And this was the result of our efforts — a consistent, steady growth in the number of domains linking to that site. We had an incredible impact for them.

And here's the graph of how many keywords they had when they ranked on the first page. This is fantastic. They ranked for a lot of keywords. And finally, here's the graph of organic traffic to the site. Amazing.

But if you looked a little closer, you notice something that is a bit troubling. We never stopped acquiring links. In fact, a lot of the content we produced is so evergreen that even content built two or three years ago is still gathering new links every single week. But the number of keywords we have ranking in the top 10 went up and up and then stopped growing. And not surprisingly, the same trend is there in organic search traffic as well. What appears to have happened here is that we got strong enough to get on the front page with these keywords, to be a player in the industry, but after that, just building more links to the site didn't help it rank for more keywords and it didn't help it get any more search traffic.

SEO fundamentals

It seems like all the SEO fundamentals that we've learned about, keywords and links and technical SEO still apply and they're still necessary to help you become a player in a particular industry. But after that, there are other factors that you need to focus on.

Now this evolution of SEO into new factors has been an accelerating process. My colleague at Moz, Dr. Pete Meyers has been tracking and collecting a lot of data about this. Last year, Google made close to 4,000 improvements to their results, and that's the result of running something like 45,000 different experiments.

Pete has also been tracking how much the search results change every day. Blue is really stable results. Orange is a lot of changes. And so if you felt like your rankings for your site are getting more volatile than ever, you're not wrong. When we hit 2017, we saw more changes to the results every day than we ever had before.

Now the way that Google's algorithms used to be updated was by a bunch of people in a room making decisions. In fact, it was this bunch of people in this room. They decided what factors to dial up or down to create the best results.

Google's goal: portal to the Internet

But what does this mean? What does it mean to make the best results? Well, we should think about what Google's real goal is. They want to be your portal to the Internet. They want your web experience to begin with a Google search, and you'll continue to do that if they make you satisfied with the results you see and the pages you click on. If they send you to the perfect web page for your query, that's a satisfying experience that reflects well on Google. If they send you to page that's a bad experience, it reflects poorly on them.

So it's interesting to ask, "How would Google avoid doing that, and what would be a bad user experience?" Well, there are some obvious things, like if you arrive on a page that installs malware or a virus on your computer, or you arrive at a product page where everything is out of stock, or you go to a website that's really slow or full of adverts. These are the pages Google does not want to include in their results.

And they've always been good at measuring these things pretty directly. More than 10 years ago they were testing how fast sites are and then using that to inform their rankings. If they spot malware or viruses on a site, they'll temporarily remove it from the search results.

But they also tried more opinion-based measures. For a while, they were running surveys to ask people: Are you satisfied with these results? This was how they knew if their algorithm was working to get people what they wanted, to give them a good experience. 

But the Google way of doing this is to try and do it at massive scale and hopefully to do it in the background, where users don't have to answer a survey pop-up like this. And doing this in the background, doing it at huge scale has been more and more possible, firstly because of how much data Google has.

Click through rates

So I want to take a look at some of the kinds of things they might be looking at. Here's an example of something they may want to do. Let's consider the average click-through rate for every ranking position in the search results. Imagine that Google knows that 30% of people click on the first result and 22% click on number two and 5% click on number six and so on. They have a good understanding of these averages. But then for a particular keyword, let's say they notice number six is getting 12% of the clicks. Something is going on there. What is happening? Well, whatever the reason why this is, Google could be better satisfying its users if that result was higher up in the rankings. Whoever is ranking at number six is what people want. Maybe they should rank higher.

"Pogo sticking"

Here's another example. This is what we call pogo sticking. A user does a search and then clicks on a result, and then after a couple seconds looking at the page, they realize they don't like it, so they click the back button and they select a different result. But let's say they don't like that one either, so they click back and they select a third result, and now they stay here and they use that site. Imagine a lot of people did the same thing. Well, if we were Google, when we saw this happening, it would be a pretty strong indicator that the third result is what's actually satisfying users. That's actually a good result for this query, and it probably deserves to be ranking much higher up.

User satisfaction: refinement

There's even an extension of this where users pogo stick around the SERPs, and then they decide they can't find anything to do with what they wanted. So they refine their search. They try typing something else, and then they find what they want on a different query. If too many people are not satisfied by any of the results on the first page, it's probably a sign to make a pretty serious change to that SERP or to nudge people to do this other query instead.

Google's evolution with Machine Learning

And doing this kind of huge analysis on a massive scale is something that was made much easier with the advent of machine learning. Now for a long time the folks in charge of the search results at Google were very reluctant to incorporate any machine learning into their work. It was something they did not want to do. But then Google appointed a new head of search, and they chose someone who had spent their career at Google promoting machine learning and its opportunities. So now they've moved towards doing that. In fact, Wired magazine described Google as remaking themselves as a "machine learning first" company.

What we're seeing now

So this is where I want to move from my conjecture about what they could do into giving some examples and evidence of all of this for you. And I want to talk about two particular modern ranking factors that we have evidence for and that if you're doing SEO or digital marketing or working on a website you can start considering today.

User signals

Firstly, I talked about the way that users interact with the results, what are they clicking on, how are they engaging with pages they find. So let's dive into that.

A lot of this research comes from my former colleague, Tom Capper. We worked at Distilled together, but he's also a Moz Associate, and a lot of this has been published on the Moz Blog.

User engagement

Let's imagine you start on Google. You type in your query, and here's the results. Here's page one of results. Here's page two of results. Not going to worry much about what happens after that because no one tends to click through further than page two.

Now let's think about how much data Google has about the way people interact with those search results. On the front page, they see lots going on. There are lots of clicks. They can see patterns. They can see trends. They can see what people spend time on or what they pogo stick back from. On the second page and beyond, there's very little user engagement happening. No one is going there, so there's not many clicks and not much data that Google can use.

So when we look at what factors seem to correlate with rankings, here's what we see. On page two, there is some correlation between the number of links a site has and where it ranks. That's kind of what we expected. That's what SEOs have been preaching for the last decade or more. But when we get to the bottom of page one, there's a weaker correlation with links. And at the top of page 1, there's almost no correlation between the number of links you have and the position you rank in. 

Now we do see that the folks on page one have more links than the sites on page two. You do need the SEO basics to get you ranking on the first page in the first place. We talk about this as the consideration set. Google will consider you for the first page of results if you have good enough SEO and if you have enough links.

But what we can take away from this is that when all that user data exists, when Google know where you're clicking, how people are engaging with sites, they will use those user metrics as a ranking factor. And then in situations where there isn't much user data, the rankings might be more determined by link metrics, and that's why deeper in the results we see links being a more highly correlated factor.

In a similar way, we can look at the whole keyword space, from the very popular head terms in green to the long tail terms in red that are very rarely searched for. Head terms have a lot of people searching for them, so Google has a lot of user data to make an assessment about where people are clicking. For long tail terms, they might only get a couple of searches every month, they just don't have that much data. 

And again, what we see is that the popular, competitive terms, where there's lots of searching happening, Google seems to be giving better rankings to sites with better engagement. For long tail terms, where they don't have that data, the rankings are more based on link strength. And there have been plenty of studies that bear this out.

Larry Kim found a relationship between high click-through rates and better rankings. Brian Dean found a relationship between more engagement with a page and better rankings. And Searchmetrics found that time on site correlated with rankings better than any on-page factor.

Contemporary SEO

And even though Google keeps a tight lid on this, they won't admit to exactly what they're doing, and they don't describe their algorithms in detail, there are occasionally insights that we get to see. 

A couple of years ago, journalists from CNBC had the chance to sit in on a Google meeting where they were discussing changes to the algorithm. One interesting part of this article was when Googlers talked about the things they were optimizing for when they were designing a new feature on the results page. They were looking at this new type of result they'd added, and they were testing how many people clicked on it but then bounced back to the results, which they considered a bad sign. So this idea of pogo sticking came up once again.

If that was something that they were monitoring in the SERPs, we should be able to see examples of it. We should be able to see the sites where people pogo stick don't do so well in SEO, which is why I'm always interested when I find a page that has, for whatever reason, it has a bad experience.

User metrics as a ranking factor

So here's a site that lists movie trivia for any movie you might be interested in. It's so full of ads and pop-ups that you can barely see any of the content on the page. It's completely overrun with adverts. So if my hypothesis was correct, we'd see this site losing search visibility, and in fact that's exactly what happened to them. Since their peak in 2014, the search visibility for the site has gone down and down and down.

Here's another example. This is a weird search. It's for a particular chemical that you buy if you were making face creams and lotions and that kind of thing. So let's have a look at some of the results here. I think this first result is the manufacturer's page with information about the chemical. The second is an industrial chemical research site. It has all the data sheets, all the safety sheets on it. The third is a site where you can buy the chemical itself.

And then here's another result from a marketplace site. I've blurred out their name because I don't want to be unfair to them. But when you click through on the result, this is what you get, an immediate blocker. It's asking you to either log in or register, and there's no way I want to complete this form. I'm going to hit the back button right away. Google had listed nine other pages that I'm going to look at before I even consider handing over all my data and creating an account here. 

Now if my theory is right, as soon as they put this registration wall up, visitors would have started bouncing. Google would have noticed, and their search visibility would have suffered. 

And that's exactly what we see. This was a fast-growing startup, getting lots of press coverage, earning lots of links. But their search traffic responded very poorly and very quickly once that registration wall was in place. The bottom graph is organic traffic, and it just drops precipitously.

Here's my final example of this, Forbes. It's a 100-year-old publishing brand. They've been online for over 20 years. And when you land on a page, this is the kind of thing you see for an article. Now I don't begrudge advertising on a page. They need to make some money. And there's only one banner ad here. I was actually pleasantly surprised by that.

But I'm baffled by their decision to include a video documentary in the corner about a totally different topic. Like I came to read this article and you gave me this unrelated video. 

And then suddenly this slides into view to make absolutely sure that I didn't miss the other ad that it had in the sidebar. And then the video, that I didn't want any way about an unrelated topic, starts playing a pre-roll ad. Meanwhile their browser alert thing pops up, and then the video — about the unrelated topic that I didn't want in the first place — starts playing. So I'm trying to read and I scroll away from all this clutter on the page. But then the video — about an unrelated topic that I didn't want in the first place — pins itself down here and follows me down the page. What is going on? And then there's more sidebar ads for good measure.

And I want to say that if my theory is right, people will be bouncing away from Forbes. People will avoid clicking on Forbes in the first place, and they will be losing search traffic. But I also know that they are a powerhouse. So let's have a look at what the data said. 

I grabbed their link profile, and people will not stop linking to Forbes. They're earning links from 700 new domains every single day. This is unstoppable. But here's their organic search visibility. Forbes is down 35% year-on-year. I think this is pretty validating.

At this point, I'm confident saying that Google has too much data about how people engage with the search results and with websites for you to ignore this. If your site is a bad experience, why would Google let you in the top results to begin with and why would they keep you there?

What can you do?

So what can you do about this? Where can you start? Well, you can go to Google Search Console and take a look through the click-through rates for your pages when they appear in search. And in your analytics package, GA or whatever else, you can see the bounce rate for visitors landing on your pages, particularly those coming from search. So look for themes, look for trends. Find out if there are pages or sections of your site that people don't like clicking on when they appear in the results. Find out if there are pages that when people land on them, they bounce right away. Either of those are bad signs and it could be letting you down in the results.

You can also qualitatively take a critical look at your site or get a third party or someone else to do this. Think about the experience that people have when they arrive. Are there too many adverts? Is there a frustrating registration wall? These things can hurt you, and they might need a closer look.

Brand signals

Okay, so we talked about those user signals. But the other area I want to look at is what I talk about as brand signals. Brand can apply to a company or a person. And when I think about the idea being a brand, I think about how well-known the company is and how well-liked they are. These are some questions that signal you have a strong brand, that people have heard of you, people are looking for you, people would recommend you. 

And this second one sounds like something SEOs know how to research. When we say people are looking for you, it sounds like we're just talking about search volume. How many times every month are people typing your brand name into Google?

Again, my colleague, Tom Capper did some research about this that's published on the Moz Blog. He looked at this problem and said, "Okay. Well, then let's see if the number of people searching for a brand has any correlation to how well they rank." And then there's a load of math and a long story that led to this conclusion, that branded search volume did correlate with rankings. This is in blue. In fact, it correlated more strongly with rankings than Domain Authority does, so that's the measure that shows you the link strength of a website. 

So think about this. We've worried about links for two decades, but actually something around brand strength and maybe branded search volume seems to correlate better.

For data geeks, here's a way of using the R-squared calculation to answer the question, "How much does this explain the rankings?" Again, what you need to know here is that branded search volume explained more of the rankings than anything else.

So we've been preaching about this for a while, and then literally two days ago I saw this tweet. A team in the UK was asking about controversial SEO opinions. And the SEO manager for Ticketmaster came out and said this. He believes that when Google sees people searching for your brand name alongside a query, they start ranking you higher for the non-branded terms. And I don't think this is controversial. And in fact, one of the replies to this was from Rand Fishkin, the founder of Moz. He also now believes that the brand signals are more powerful than what links and keywords can do.

What can you do? 

So what can you do about this? Well, first you have to realize that any investment you make in brand building, whether that's through PR activities or through like traditional advertising, is good business to do anyway. But it now has twice the value because of its impact on SEO, because those activities will get people looking for you, following you, sharing your brand. If you work for a billion-dollar company, you should make sure that your SEO and PR teams are well-connected and well-aligned and talking together. If you don't work for a billion-dollar company, I've got two small, interesting examples for you.

Example: AdaFruit

First I want to call out this site, AdaFruit.com. They sell electronic components. There are many, many sites on the web that sell similar products. Not only do they have great product pages with good quality images and helpful descriptions, but I can also look at a product like this and then I can click through to get ideas for things I can build with it. This is some LED lights that you can chain together. And here's an idea for a paper craft glowing crystal you can build with them. Here's the wiring diagram I'd need for that project plus some code I can use to make it more interactive. It's only an $8 product, but I know that this site will make it easy for me to get started and to get value from making this purchase.

They go even further and have a pretty impressive AdaFruit channel on YouTube. They've got 350,000 subscribers. Here's the videos, for instance, that they publish every week walking you through all the new products that they've recently added to the site. 

The CEO does a hands-on demo telling you about everything they have in stock. And then they have other collections of videos, like their women in hardware series that reaches an audience that's been typically underserved in this space.

AdaFruit made a significant investment in content for their own channels, and it paid off with some brand authority, but brand trust and brand engagement as well.

Example: Investor Junkie

But I want to show you one other example here from arguably a much less exciting industry and someone who couldn't invest so much in content. This is InvestorJunkie.com, a site that does reviews of financial services and products. And when I was working at the agency, we worked with this site and specifically with its founder, Larry. Larry was an expert in personal finance and particularly in personal investments. And this was his solo project. He blogged on the site and used his expertise. But as the site grew, he hired some contractors as well as our agency, and they created a lot of great content for the site, which really helped with SEO. But to make a significant impact on brand strength, we had to get the word out in front of loads of people who didn't already know about him.

So we took Larry's expertise and we offered him as a guest to podcasts, a lot of podcasts, and they loved having him on as a guest. Suddenly Larry was able to provide his expertise to huge new audiences, and he was able to get the Investor Junkie brand and their message in front of lots of people who had never heard of the site before.

But better still, this had a compounding effect, because people who are interested in these topics typically don't just subscribe to one of these podcasts. They subscribe to a bunch of them. And so if they hear about Larry and Investor Junkie once, they might never think about it again. But if he shows up in their feed two or three or four times over the course of a few months, they'll start to form a new association with the brand, maybe trusting him more, maybe seeking out the site.

And as an aside, there's one other thing I love about podcasts, which is that if you're creating a blog post, that can take hours and hours of work. If you're creating a conference presentation, it can take days or weeks of work. If you're a guest on a 30-minute podcast, it literally takes you about 30 minutes. You log on, you talk to a host, and then your part of the work is done.

So this can get you in front of a new audience. It gets people looking for you, which Google will notice. But it has even more SEO value as well, because every podcast typically has a page like this with show notes. It's a page that Google can index, a page that Google can understand. And Google can see the signals of trust. It can see your brand being mentioned. It can see the links back to your site as well. I obviously can't speak highly enough of podcasts for PR, for brand awareness, and even for SEO.

Did this help Larry and the Investor Junkie team? Yeah. This obviously wasn't the extent of their SEO strategy. But everything they did contributed to them getting great rankings for a variety of competitive terms, and it helped them rank up against much bigger sites with much bigger teams and much bigger budgets. And that story actually came to an end just about two years ago, because the site was finally acquired for $6 million, which is not bad for a solo founder who was just busy building his own brand.

In summary

All right. I'll wrap up with some of these thoughts. Google has been evolving. They've now been able to collect so much more data about the way people interact with the search results and other pages, and they're now using machine learning to process all of that so they can better assess: Are we giving people a good user experience? Are the sites that we're ranking the ones that satisfy people's queries? The game of SEO has changed.

Now when you're starting out, all the basics still apply. Come to Moz, read the Beginner's Guide, do great technical SEO, do great keyword research, do great link building. Those are still necessary to be considered to become a player in your industry to help get you near the first page for any terms you want to target.

But when you're trying to move up the front page, when you're trying to establish yourself much further and become a much bigger brand, we're not seeing a lot of correlation between things like links and getting into the very top rankings for any particular term. Instead, think about the good game that Google is playing. They want to make sure that when someone clicks on a result, they stay there. They don't want to see this pogo sticking. They don't want to see the link and the title that people want to click on sitting down at number six. So target their KPIs. Think about how you can help Google by making sure that your results are the ones people want to click on. Make sure that when people click on your results, that's the page that they stay on.

But ultimately, you will never lose out if you improve your brand authority and engagement with your content. These are just good things to do for business. A stronger brand, content, and a website that people want to spend time on is hugely important and pays dividends. But now it's all doubly important because it also has this massive impact on your SEO.

Video transcription by Speechpad. 

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!