Reporting on Ranking Changes with STAT’s Google Data Studio Connectors

Posted by ipfister

Since Moz released the new Google Data Studio Connectors for STAT, you might be wondering how to best implement them for your reporting strategy. My colleagues at Path Interactive and I absolutely love how granular you can get with your reports in STAT, and finally having the ability to cleanly and effectively pull those reports into Data Studio (the tool we use for our own reporting) is a godsend.

While the Historical Keyword Rankings connector reports on rank over time, it may not be as obvious how to report on rank change over time. In this post, I'll give you step-by-step guidance on how to report on rank change — as well as a couple other filtering and reporting tips — while using the connectors within Google Data Studio.

If you aren't a STAT user yet but you want to know how it might fit into your SEO toolkit, you can take a tour of the product. Click on the button below to set one up!

Learn More About STAT

Connecting your data source

Before you begin, you need to identify a few things to set up the connector: your STAT Keyword API Key, the Project ID, and your Site ID. If you don’t already know how to identify these via the STAT API, you can head over to STAT’s documentation here to learn more. After you’ve identified these, it’s time to connect your data source. 

We’re going to be doing something a little out of the ordinary here, but stay with me — you’ll see why in just a second!

For this step, we'll be connecting two instances of the same source. Because our goal is to compare rank change over time, we’ll use the same source twice to identify those deltas.

When setting up your connector, be sure to name the source something that you’ll easily recognize:

In my case, I usually go with something simple such as “[client name] STAT Keyword Connector.” When this is complete, repeat the step above, but name it something different, e.g. “[client name] STAT Keyword Connector 2.”

Finally, make sure the metrics you plan on comparing have unique names for each connector. To do so, go into your data source. Click on the metric’s name so that you can rename it, and then rename it something unique. For this case we'll be doing it for “Google Base Rank,” since we're comparing ranks, but it can also be done for “Google Rank,” if we wanted to compare that. Again, I like to just keep it simple: for the first data source call it “Google Base Rank 1,” and then for the second data source call it “Google Base Rank 2.” When all is said and done, it should look something like this:

Building your table and blending data

Now we’ll start to get a bit more technical. Blending the data of the two connectors lets you compare two instances of rankings against each other. Your final result will produce a table showing the ranks of two given dates, as well as their rank change. The five-step process will look like this:

1. Blend data of keyword connectors one and two
2. Add in your common metrics for the two sources (keyword at the minimum, but you can also add in location, device, market, and search volume)
3. Add in the metric you'd like reported (Google base rank and/or Google rank)
4. Set date range
5. Apply “No Null” filter

1. Blend data of keyword connectors one and two

The first step here is to blend the two connectors so that you can compare two instances of ranks against each other. 

First, you need to create a new report, or go into a report that's already set up. Next, select your data source. Here you'll select the first instance of the source that you set up earlier (if you’re starting on a fresh report, it'll ask you to add a data source immediately). Once selected, click on “Blend Data” underneath the data source on the right hand side of Google Data Studio, seen here:

This will bring you to the Blend Data source tool. From here you select to add another data source, being your second instance of the connector.

2. Add in your common metrics

Once you've chosen to blend both connectors, you need to set your metrics. Towards the top, you’ll see “Join keys.” This is in reference to what's going to be the same for both instances, so here at the minimum, you want to include “keyword.” Feel free to play around here with adding different metrics.

Note: We'll go over this later, but if you plan on having different graphs filtered by a certain tag or location, make sure to add these in here.

3. Add in the type of rank you want reported

After setting your metrics under “Join keys,” now select the metrics that will be unique for each date. Depending on what you want to compare, under “Metrics” you'll pick “Google Base Rank,” “Google Rank,” or both. You may also include “Date” here too if you'd like. Once done, click on “SUM” next to the metric name, and change this to “MIN.” You'll see why in just a moment.

At this point, your blended data should look something like this:

4. Set date range

Now you need to set the two date ranges you're comparing to each other. 

To do this, under the first connection, set your first date: Under “Date Range,” click on “Custom,” then click on the field to select your date. Here you might see that there's an option for two dates, but for this solution, we're using the same date for each connector.

In the end, it'll be something like “Connector 1” selected for the “start date” and “end date” as the first of the month, and for “Connector 2,” the “start date” and “end date” will be the last of the month. This is essentially pulling in the rank for the first instance as well as the second instance, so you can compare the two.

5. Set “No Null” filter

The last step in setting up your blended data is creating a “No Null” filter. When the keyword connector reports on ranks that your site is not ranking for, it will return as "null." To avoid flooding your data with fluff, you need to create a filter removing instances of "null."

First, click on “Add A Filter” below where you selected the date range. Next, towards the bottom, click on “Create A Filter.” Set the parameters of the filter as “Exclude” > “Google Base Rank 1 (2)” > “Is Null.” Be sure to name the filter something identifiable such as “No Null.” It should look like this:

Applying rank change to your report

Now you can create a new field that will report on the rank change by making a calculated field to find the difference of the two ranks. 

Under dimensions, select “Add Dimension,” and click on “Create Field.” You can name it “Rank Change,” but to create the field, start typing “Google Base Rank,” and you’ll see your instances from each connector come up. To make the calculated field, select your “Google Base Rank 1” and subtract it from “Google Base Rank 2,” so it should look something like this:

Hit apply, and now your rank change should be calculated!

There is also an additional way to get the same result, but with a few drawbacks, such as not being able to name the header, as well as not being able to filter or sort your rank change. The benefit to this approach is that it's easier to set up initially, as you don’t actually need to blend the data. However, not setting up the blended data will also forfeit having the initial rank visible. When in your edit view, set a custom date range that you're reporting on under “Default date range.” Here, you can then set a comparison date: if looking back a month, you can set this to the first. If you go with this option, it should look like this:

Head into the "Style" tab, where you can change the comparison to “Show Absolute Change” under “Metrics.” You can also change the colors of your positive and negative arrows to more accurately represent the movement (you can see from above that the “negative” change is a green arrow, this defaults to red).

Using filters

Applying filters to your data set can be extremely beneficial to making sense of your data! Using filters with the connector can help you segment out rankings for a particular location, or create charts that show rankings for a specific keyword group that you’ve set up using keyword tags. 

Take a look at this report I set up as an example. Within STAT, I created keyword tags to target locations determined by what zip code they were. Then, I was able to create a filter for each chart targeting that keyword tag:

Setting filters up is extremely simple. First, go into edit mode. Next, scroll down the side until you find “Filter.” Then under Filter > Table Filter, click on “Add a Filter.” This will bring you to the filter picker. Toward the bottom, click on “Create a Filter.” Here you can set the parameters for the filter you'd like to show.

Some of my other favorites include filtering to only show the top few pages (filters out non-relevant and high ranks), using the keyword tag filter like I showed before, and also filtering by location. But you don’t have to stop there! Adding in the additional dimensions available to you in the connector, you can use the filter to show things such as desktop vs. mobile or how your keyword ranking performance does in different markets.

Blending your Google Analytics, Google Search Console, and STAT data

One of my favorite uses for the connectors is the ability to blend the data with your Google Analytics and Google Search Console data. By blending this data together, you’re able to directly tie keyword rankings with different metrics, such as clicks or goal completions.

You’re probably a pro at blended data at this point, but just for reference, the data blended should look like this:

A few things to note: it’s important what order you put the connectors in. I’ve found that adding the STAT connector first works best (i.e. if you put Google Analytics first, you’ll get a report with the infamous "not found" keyword). Additionally, to pull in Search Console data in order to match with your other connectors, using “Query” will have the same effect as “Keyword.”

The result would look something like this, but feel free to edit the design how you wish!

Now you can go even further with this and match up URLs, but this will require some RegEx. 

You'll rename the “Google URL” field in STAT and “Landing Page” field in Google Search Console in order to match the URL structure convection within Google Analytics by taking out the domain portion of the URL. To do this, go into your data source for each STAT connector and Google Search Console, and click “Add A Field” in the top right.

Next, enter to following RegEx for the STAT connector:

REGEXP_REPLACE(Google URL, ".*[\\.]com", "")

And for Google Search console:

REGEXP_REPLACE(Landing Page, ".*[\\.]com", "")

Remember to name them something to differentiate from the default field. I use “Landing Page (no domain).”

When building a report, use these new fields for consistency across the URL structure so that, when you select them when blending data, they'll match. 

Use this method in the same way as above to get the desired results of pulling in data from across all three connectors to match up with each other! In the end you should be able to find what keyword ranks for what URL, as well as have many sessions or clicks that are brought in as well as goal completions, or any other combination.

Well there you have it! Hope this was helpful to you. If you have any other questions you can comment below or find me on Twitter @ianpfister. Happy reporting!

---

To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.

Take the Survey

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Baltimore Trump Tweet Causes Backlash from Archbishop

 Most Reverend William Lori, Archbishop of Baltimore, has entered the ongoing debate regarding the state of the region.

The discourse started last week after President Trump commented on the living conditions in Baltimore despite their receiving federal aid.

How Big Is the Gender Gap Between Men and Women in SEO?

Posted by NicoleDeLeon

To anyone working in SEO, it’s fairly evident that this is a male-dominated industry. Although there are powerful women SEOs in the field (like Moz CEO Sarah Bird, for example), if you glance at a conference speaker lineup or peruse the bylines on search-related blogs, you’ll see that those who identify as female are few and far between. A recent list of the 140 most influential SEOs featured 104 men and just 36 women. 

So how big is the gender gap? And how does it translate to tangible things like pay and job titles? To find out, we mined the data from our State of SEO 2020 survey, which featured 652 SEOs in 51 countries. Here are some of the things we learned.

But first, a mea culpa. If SEOs who identify as women have an uphill climb in this industry, there’s no doubt that female-identifying SEOs of color have a hill that is steeper still. I deeply regret not asking demographic questions on race and ethnicity which would have allowed us to analyze the disparate impacts that bias plays on BIPOC women SEOs. It was a missed opportunity. That said, we are currently running a survey on BIPOC in SEO that aims to cover those issues and more as we continue to take an introspective view of our industry.

Men outnumber women by more than 2 to 1 in SEO

Of the 652 SEOs who participated in the study, 191 identified as women (29.3%) and 446 identified as men (68.4%). Additionally, one identified as non-binary and 14 preferred not to say. Data was collected on a SurveyMonkey form. We reached out to our own database, purchased lists of SEOs around the world, and promoted the survey on social channels for respondents. We offered no compensation or reward for participating. Non-binary, persons who chose not to identify a gender by choosing “preferred not to say”, and SEOs from the African continent were underrepresented mostly due to the outreach database itself. Finally, respondents selecting non-binary and “preferred not to say” were not calculated in the men/women percentages. 

A voluntary survey is not a scientific sampling, but those percentages mesh with previous studies by Moz that found those who identified as women made up 22.7% of SEOs in 2012, 28.2% in 2013, and 30.1% in 2015. In all four studies, men outnumbered women by more than 2 to 1. 

Importantly, the new results suggest the gap hasn’t narrowed over the past five years.

This was not a surprise to many female-identifying SEOs who participated in the study.

“I started out in the SEO industry about 10 years ago. Compared to that, I do see more women at conferences, on online platforms, and in the day-to-day work with clients,” one said. However, she added that she hasn’t seen much progress in the last 5 years. “It’s like we are kind of stuck. I suspect it’s at least partly a visibility issue: Men have been there forever, building their reputation and expertise. It is hard to keep up with that if you had a late start.”

We interviewed more than a dozen female-identifying SEOs, most of whom asked not to be named. Although a few had supportive bosses, clients, colleagues, and mentors along the way, many shared experiences of being passed over for promotions, having to fight to be heard in meetings and, in some cases, being paid less than men for the same work.

“I think you can sum up the SEO industry by looking at speaker panels of all the major conferences. There is no equality. Are you a white male and 50+? You must be an expert! Are you a woman, 40, who’s been doing this since 2004? Oh, honey, go sit down. We have an expert already,” said one woman. She said she spent 13 years at a website development company being “constantly overlooked” before moving to a digital marketing agency where she is respected and valued.

The gender gap is widest in Latin America

Global internet usage has boomed over the past two decades, with more than 59% of the world’s population now online. Although internet penetration is highest in Europe and North America, more than three-quarters of global users live elsewhere. These growing markets are served by robust communities of SEO professionals on every continent.

Our study reached SEOs in 51 countries, which we grouped into 11 large regions. Participation was highest in the U.S. with 269 SEOs responding, but we also surveyed 113 SEOs in Western Europe, 85 in the U.K., 43 in the Eastern Europe/Balkans region, 39 in Australia and New Zealand, 30 in Asia, 21 in Canada, 18 in Scandinavia, 16 in the Middle East, 12 in Central and South America, and 6 in Africa.

We found that the gender gap is most pronounced in Latin America (83.3% who identified as men to 16.7% who identified as women) and Australia/New Zealand (82.1% who identified as men to 17.9% who identified as women). 

The gender gap is least significant in Africa (although with an admittedly very small sample size due to the small number of African SEOs in our database) and Canada (52.6% who identified as men to 47.4% who identified as women). Under Prime Minister Justin Trudeau, a self-professed feminist who appointed a gender-balanced cabinet, Canada has made gender equality a priority, but progress has been uneven at times. 

When it comes to gender diversity in SEO, the U.S., Asia, and the U.K. all trail behind Europe, Scandinavia, and the Middle East.

Female-identifying SEOs are more likely to freelance and specialize in content

Generally, the three most common career environments for SEOs are serving as an in-house expert at a single company, working in an agency setting, or operating independently as a consultant or freelancer. Each path has its own pros and cons. We found some interesting gender differences in where SEOs are working. 

Male-identifying and female-identifying SEOs are equally likely to work in-house, with about 40% of both genders working inside a single business. And as we discuss below, both genders reported being satisfied with the working conditions and level of support they received in their roles.

Among those who practice their craft externally, men are slightly more likely to work in agencies than women (49.7% vs. 42.5%). 

The biggest gap was among freelancers. Female-identifying SEOs are almost twice as likely to be contractors or freelancers as those who identify as men (17.7% vs. 10.6%). However, it’s unclear if female-identifying SEOs are heading out on their own because they don’t feel they can get a fair shake working for others, or if they're drawn to the freedom and flexibility of freelance work.

Full-time freelancing has grown steadily across the economic landscape in recent years. It also tends to draw more women than men. Part of the appeal may be flexibility around childcare, but control over income was also a factor for some of the SEOs we interviewed. 

“I think a lot of women choose to do freelance because they want to be paid what they deserve, frankly,” said one 25-year-old female SEO in East Anglia, U.K.

However, another woman who works as an in-house SEO said, “When I got my start in marketing, most of the jobs offered to me were contractor roles, and it wasn’t clear how to become full time. It wasn’t by choice; it was what was available to me.”

Many female-identifying SEOs said it was hard for them to get hired or promoted, even with stellar track records. 

“I’ve seen loudmouth, no-record, no-proof men be hired. It’s absolutely aggravating. At my old company, I was skipped by two men who had half the knowledge for supervisor positions. Each man left within months to different companies to the next title,” said a 41-year-old female SEO in Minnesota. She subsequently changed companies and found a much more welcoming environment.

In addition to career paths, there are noteworthy differences in the areas of the industry that male-identifying and female-identifying SEOs are most likely to specialize in. Most SEOs consider themselves generalists, but among those who profess a specialty, women are twice as likely to be content experts (17.6% to 7.7%).

On the other hand, male-identifying SEOs are nearly twice as likely to be technical experts (21.5% to 12.6%). It’s unclear if this is a result of choice, fallout from the gender gap in STEM occupations generally, or if those who identify as women feel unwelcome among tech SEOs.

Among the female-identifying SEOs we interviewed, several said they think early gender stereotyping played a role, from the toys little boys and girls are given to what each gender is encouraged to pursue as a career.

“It’s similar to why women are not often involved in engineering jobs. Technical roles are historically associated with developer training, and women are more likely to transition from the marketing side than the programming side,” one said.

Several women also said technical SEO, in particular, is a “boys club.” 

“I participate in online forums for general Tech SEO and Women in Tech SEO, and the vibes are much different,” one woman said. “The male-dominated general forums are competitive. The female groups are more supportive, but again, we are trying to bring along and encourage women in the field.”

Another tech SEO who worked at an agency and in-house before going out on her own as a contractor said the culture can be intimidating: “I find that men are quicker to hop on and attack people about technical knowledge than women.”

Female-identifying SEOs generally charge less than men for their services

To find out more about the dollars and cents of SEO, we asked the agency and contract SEOs who participated in our study about their pricing models. In all, 261 SEOs were willing to share how they price their services and how much they charge. 

The three most common pricing models are monthly retainers, per-project pricing, and hourly rates. Although there was a wide range of rates among male-identifyng and female-identifying SEOs, the medians were consistently lower for those who identified as women.  

Among agency and contract SEOs, men are more likely to price their services with monthly retainers (59.1% of men vs. 39.4% of women). Women are more likely to charge per project (31.8% of women vs. 18.2% of men). About a quarter of both groups use hourly pricing.

But before we talk about prices...

Before we get into the details of how much male- and female-identifying SEOs earn, it’s important to note that we didn’t ask who actually set the prices. Depending on the size of an agency, SEOs who work there may have very little control over the pricing structure. 

The agency’s rates might be standard, or they might vary depending on who does the work. One can assume that freelancers choose their own rates, although they might be responding to signals about what the market will bear and what clients are willing to pay. 

Some studies have suggested that a variety of psychosocial factors lead female-identifying freelancers to charge less than their male counterparts. For instance, a Hewlett-Packard study identified a confidence gap in which women tended not to apply for a promotion unless they met all the qualifications, but men would go for it if they met 60 percent of the job requirements. 

Conventional wisdom holds that women are more cooperative and men are more competitive. Whether or not that’s true, men initiate negotiations more readily than women and tend to ask for higher compensation.

In a future study, we will certainly ask who determined the service pricing. For now, we can only report what male-identifying and female-identifying SEOs told us they charge.

Retainers for those who identify as male are 28.6% higher than for those identifying as female

Our respondents included 138 agency and contract SEOs who use monthly retainers as their primary pricing model. These retainers ranged from less than $250 a month to more than $25,000 a month, but overall they were higher for men. At the midpoint of the ranges on our survey, those identifying as male charge a median retainer of $2,250 a month while those identifying as female charge a median of $1,750.

When we looked at agency SEOs and freelancers separately, the median for freelancers was much lower, but it was the same for both genders: $750 a month. However, the sample size was quite small. There were only 19 freelancers in the study who primarily use retainers. Among the 119 agency SEOs who use retainer pricing, the median retainer was $2,250 for those identifying as male and $1,750 for those identifying as female. 

Project prices for men are 66.7% higher than for women

Our respondents included 54 agency and contract SEOs who typically charge on a per-project basis. The scope and cost of projects varied greatly, from less than $250 to more than $100,000. But the data showed that overall, men charge more per project — a median of $5,000 vs. $3,000 for female-identifying SEOs. 

We decided to dig deeper and found an interesting exception when we looked at agency SEOs and freelancers separately. 

The price gap was more than three times as wide among those who work in agencies. Our sample included 36 agency SEOs who use per-project pricing. Male-identifying SEOs reported that their agencies charge a median of $8,750 per project while those who identify as women said their agencies charge a median project fee of $2,250. 

The reverse was true among independent SEOs. The sample size was small, so we’re not sure what to make of it, but among the 18 freelance or contract SEOs we polled who charge by the project, women had the edge. Female-identifying freelancers charge a median fee of $3,750 per project to $1,750 for male freelancers. 

One contractor in her 50s hypothesized: “I think women may be more detail-oriented and spend more time with their project. Maybe men may charge less because they have more clients?”

Median hourly rates for male-identifying SEOs are 16.8% higher than for female-identifying SEOs

Our respondents included 57 agency and contract SEOs who typically bill by the hour. Among this group, the median rate is $125 for male-identifying SEOs vs. $107 for female-identifying SEOs. In this case, the difference is largely attributable to more women working as freelancers. The median rate for men and women SEOs at agencies was $125 an hour, and the median rate for both who work as contract or freelance SEOs was $88 an hour. 

Many of the female-identifying SEOs we interviewed said women tend to undervalue themselves and need to be more assertive in negotiating prices.

“I think confidence and not being scared to charge what you’re worth comes into play for the higher rates,” said digital marketing and content specialist Kristine Strange.

Both men and women feel equally supported as in-house SEOs

Some good news for in-house SEOs: When asked about working conditions, frustrations, and pain points, both men and women had very similar responses. Both reported strong levels of interdepartmental cooperation and support for SEO priorities.

Female-identifying SEOs are slightly more satisfied than male-idneitfying SEOs with in-house SEO resources

The resources available to in-house SEOs are largely dependent on the size and fiscal health of the company that employs them. 

Among in-house SEOs, women are as likely as men to work for enterprise-level companies. We found that 27.1% of male-identifying in-house SEOs and 24.8% of female-identifying in-house SEOs work for companies with more than 250 employees. And 72.9% of male-identifying and 75.2% of female-identifying SEOs work for companies with 250 or fewer employees. 

In-house SEOs across the board rated engineering support as their biggest challenge. Female-identifying SEOs were generally more satisfied than their male peers with the expertise of their teams and their staffing levels. They were equally satisfied with other elements of their SEO programs.

Conclusion

Although there are some very prominent and talented female-identifying SEOs, they are still underrepresented. And when they do enter the field, they are often compensated at lower rates than men. There is no single solution to broadening the talent pool, but we have a few thoughts.

Welcoming industry: The overwhelming number of  women who spoke to us about these findings wished to remain anonymous. We can only assume that means female-identifying SEOs do not feel safe openly discussing issues of gender within an SEO workplace. Silence only serves to bolster the status quo. We must foster an industry culture that does not punish the whistleblower but instead seeks to listen, understand, grow, and improve opportunities for all its members.

Training and mentoring: More than in many other industries, there isn’t one clear path to becoming an SEO. The STEM fields are one training ground, but many other SEOs learn the craft from mentors. To achieve more diversity, which is good for the industry and outcomes, it’s important for girls and those who identify as girls to be supported and welcomed into STEM classes during their student years. 

As an industry, we need to take the job of mentorship seriously. Experienced SEOs can do more to mentor young talent, particularly those who identify as women. Agencies can do more to recruit and hire people with different backgrounds.

Several women whom we interviewed mentioned the importance of mentors and allies:

"I sit in countless calls where I say something and until my CTO repeats what I say, some clients don’t hear me. My CTO is so supportive and wonderful, and he will literally say, 'She’s right when she says, ‘Blah.’ She’s got 20 years under her belt… .' Then their light turns on."

"I’m good at learning complex software and doing complex technical tasks but wasn’t encouraged in this until my recent job — and even then, it wasn’t until I got a female manager that I was recognized for this ability and assigned those kinds of tasks on a regular basis."

"I spent the first two years double- and triple-checking all my work, backing everything with links from male experts in the industry. One day the CTO told me I didn’t need to do that. He trusted me. I found myself in the bathroom in tears. It took me a long time to stop sending links. (Sometimes I still send links, but only if I think he needs to read them to keep up with me!)"

Transparency about pay and pricing: The taboo about discussing fees and compensation keeps inequities hidden. It’s time to shatter that norm. Independent SEOs should run their pricing plans by mentors of all genders for perspective. Agencies should be sure that skill and experience, not gender, is the driving factor in pay and pricing. 

Don’t undersell yourself: If negotiation doesn’t come naturally to you, spend extra time preparing proposals. Research your competitors and talk with mentors. Focus on the value you’re adding. Be sure to factor in your skill level and experience as it grows. Don’t fall into the confidence gap trap. Even if you don’t tick all of the boxes, if you have most of the qualifications, forge ahead to apply or submit a proposal.

I want to acknowledge the important role that several female-identifying SEOs played in the making of this article. First, I have the privilege of working with some amazing women every day in my SEO agency. Thanks to Cindy Glover, without whom I could not have produced this study. I also want to thank Areej AbuAli whose work in creating the Women in Tech SEO community has been an invaluable resource to the SEO industry and in particular, SEOs who identify as women. Women in Tech SEO not only helps to amplify the voices of those identifying as women within the community, but also helps connect them to each other.

If you wish to explore your own possible implicit bias around issues of gender and career, check out Harvard’s Gender-Career implicit bias test.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Let's Make Money: 4 Tactics for Agencies Looking to Succeed – Best of Whiteboard Friday

Posted by rjonesx.

We spend a lot of time discussing SEO tactics, but in a constantly changing industry and especially in times of uncertainty, the strategies agencies should employ in order to see success deserve more attention. In this popular (and still relevant) Whiteboard Friday, Russ Jones discusses four essential success tactics that'll ultimately increase your bottom line. 

Russ also delved into the topic of profitability in his MozCon Virtual presentation this year. To watch his and our other amazing speaker presentations, you can purchase access to the 2020 video bundle here.  

Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Howdy, Moz fans. I am Russ Jones, and I can't tell you how excited I am for my first Whiteboard Friday. I am Principal Search Scientist here at Moz. But before coming to Moz, for the 10 years prior to that, I was the Chief Technology Officer of a small SEO agency back in North Carolina. So I have a strong passion for agencies and consultants who are on the ground doing the work, helping websites rank better and helping build businesses.

So what I wanted to do today was spend a little bit of time talking about the lessons that I learned at an agency that admittedly I only learned through trial and error. But before we even go further, I just wanted to thank the folks at Hive Digital who I learned so much from, Jeff and Jake and Malcolm and Ryan, because the team effort over time is what ended up building an agency. Any agency that succeeds knows that that's part of it. So we'll start with that thank-you.

But what I really want to get into is that we spend a lot of time talking about SEO tactics, but not really about how to succeed in an industry that changes rapidly, in which there's almost no certification, and where it can be difficult to explain to customers exactly how they're going to be successful with what you offer. So what I'm going to do is break down four really important rules that I learned over the course of that 10 years. We're going to go through each one of them as quickly as possible, but at the same time, hopefully you'll walk away with some good ideas. Some of these are ones that it might at first feel a little bit awkward, but just follow me.

1. Raise prices

The first rule, number one in Let's Make Money is raise your prices. Now, I remember quite clearly two years in to my job at Hive Digital — it was called Virante then — and we were talking about raising prices. We were just looking at our customers, saying to ourselves, "There's no way they can afford it." But then luckily we had the foresight that there was more to raising prices than just charging your customers more.

How it benefits old customers

The first thing that just hit us automatically was... "Well, with our old customers, we can just discount them. It's not that bad. We're in the same place as we always were." But then it occurred to us, "Wait, wait, wait. If we discount our customers, then we're actually increasing our perceived value." Our existing customers now think, "Hey, they're actually selling something better that's more expensive, but I'm getting a deal," and by offering them that deal because of their loyalty, you engender more loyalty. So it can actually be good for old customers.

How it benefits new customers

Now, for new customers, once again, same sort of situation. You've increased the perceived value. So your customers who come to you think, "Oh, this company is professional. This company is willing to invest. This company is interested in providing the highest quality of services." In reality, because you've raised prices, you can. You can spend more time and money on each customer and actually do a better job. The third part is, "What's the worst that could happen?" If they say no, you offer them the discount. You're back where you started. You're in the same position that you were before.

How it benefits your workers

Now, here's where it really matters — your employees, your workers. If you are offering bottom line prices, you can't offer them raises, you can't offer them training, you can't hire them help, or you can't get better workers. But if you do, if you raise prices, the whole ecosystem that is your agency will do better.

How it improves your resources

Finally, and most importantly, which we'll talk a little bit more later, is that you can finally tool up. You can get the resources and capital that you need to actually succeed. I drew this kind of out.

If we have a graph of quality of services that you offer and the price that you sell at, most agencies think that they're offering great quality at a little price, but the reality is you're probably down here. You're probably under-selling your services and, because of that, you can't offer the best that you can.

You should be up here. You should be offering higher quality, your experts who spend time all day studying this, and raising prices allows you to do that.

2. Schedule

Now, raising prices is only part one. The second thing is discipline, and I am really horrible about this. The reality is that I'm the kind of guy who looks for the latest and greatest and just jumps into it, but schedule matters. As hard as it is to admit it, I learned this from the CPC folks because they know that they have to stay on top of it every day of the week.

Well, here's something that we kind of came up with as I was leaving the company, and that was to set all of our customers as much as possible into a schedule.

• Annually: we would handle keywords and competitors doing complete analysis.
• Semi-annually: Twice a year, we would do content analysis. What should you be writing about? What's changed in your industry? What are different keywords that you might be able to target now given additional resources?
• Quarterly: You need to be looking at links. It's just a big enough issue that you've got to look at it every couple of months, a complete link analysis.
• Monthly: You should be looking at your crawls. Moz will do that every week for you, but you should give your customers an idea, over the course of a month, what's changed.
• Weekly: You should be doing rankings

But there are three things that, when you do all of these types of analysis, you need to keep in mind. Each one of them is a...

• Report
• Hours for consulting
• Phone call

This might seem like a little bit of overkill. But of course, if one of these comes back and nothing changed, you don't need to do the phone call, but each one of these represents additional money in your pocket and importantly better service for your customers.

It might seem hard to believe that when you go to a customer and you tell them, "Look, nothing's changed," that you're actually giving them value, but the truth is that if you go to the dentist and he tells you, you don't have a cavity, that's good news. You shouldn't say to yourself at the end of the day, "Why'd I go to the dentist in the first place?" You should say, "I'm so glad I went to the dentist." By that same positive outlook, you should be selling to your customers over and over and over again, hoping to give them the clarity they need to succeed.

3. Tool up!

So number three, you're going to see this a lot in my videos because I just love SEO tools, but you've got to tool up. Once you've raised prices and you're making more money with your customers, you actually can. Tools are superpowers. Tools allow you to do things that humans just can't do. Like I can't figure out the link graph on my own. I need tools to do it. But tools can do so much more than just auditing existing clients. For example, they can give you...

Better leads:

You can use tools to find opportunities.Take for example the tools within Moz and you want to find other car dealerships in the area that are really good and have an opportunity to rank, but aren't doing as well as they should be in SERPs. You want to do this because you've already serviced successfully a different car dealership. Well, tools like Moz can do that. You don't just have to use Moz to help your clients. You can use them to help yourself.

Better pre-audits:

Nobody walks into a sales call blind. You know who the website is. So you just start with a great pre-audit.

Faster workflows:

Which means you make more money quicker. If you can do your keyword analysis annually in half the time because you have the right tool for it, then you're going to make far more money and be able to serve more customers.

Bulk pricing:

This one is just mind-blowingly simple. It's bulk pricing. Every tool out there, the more you buy from them, the lower the price is. I remember at my old company sitting down at one point and recognizing that every customer that came in the door would need to spend about $1,000 on individual accounts to match what they were getting through us by being able to take advantage of the bulk discounts that we were getting as an agency by buying these seats on behalf of all of our customers.

So tell your clients when you're talking to them on the phone, in the pitch be like, "Look, we use Moz, Majestic, Ahrefs, SEMrush," list off all of the competitors. "We do Screaming Frog." Just name them all and say, "If you wanted to go out and just get the data yourself from these tools, it would cost you more than we're actually charging you." The tools can sell themselves. You are saving them money.

4. Just say NO

Now, the last section, real quickly, are the things you've just got to learn to say no to. One of them has a little nuance to it. There's going to be some bite back in the comments, I'm pretty sure, but I want to be careful with it.

No month-to-month contracts

The first thing to say no to is month-to-month contracts.

If a customer comes to you and they say, "Look, we want to do SEO, but we want to be able to cancel every 30 days." the reality is this. They're not interested in investing in SEO. They're interested in dabbling in SEO. They're interested in experimenting with SEO. Well, that's not going to succeed. It's only going to take one competitor or two who actually invest in it to beat them out, and when they beat them out, you're going to look bad and they're going to cancel their account with you. So sit down with them and explain to them that it is a long-term strategy and it's just not worth it to your company to bring on customers who aren't interested in investing in SEO. Say it politely, but just turn it away.

Don't turn anything away

Now, notice that my next thing is don't turn anything away. So here's something careful. Here's the nuance. It's really important to learn to fire clients who are bad for your business, where you're losing money on them or they're just impolite, but that doesn't mean you have to turn them away. You just need to turn them in the right direction. That right direction might be tools themselves. You can say, "Look, you don't really need our consulting hours. You should go use these tools." Or you can turn them to other fledgling businesses, friends you have in the industry who might be struggling at this time.

I'll tell you a quick example. We don't have much time, but many, many years ago, we had a client that came to us. At our old company, we had a couple of rules about who we would work with. We chose not to work in the adult industry. But at the time, I had a friend in the industry. He lived outside of the United States, and he had fallen on hard times. He literally had his business taken away from him via a series of just really unscrupulous events. I picked up the phone and gave him a call. I didn't turn away the customer. I turned them over to this individual.

That very next year, he had ended up landing a new job at the top of one of the largest gambling organizations in the world. Well, frankly, they weren't on our list of people we couldn't work with. We landed the largest contract in the history of our company at that time, and it set our company straight for an entire year. It was just because instead of turning away the client, we turned them to a different direction. So you've got to say no to turning away everybody. They are opportunities. They might not be your opportunity, but they're someone's.

No service creep

The last one is service creep. Oh, man, this one is hard. A customer comes up to you and they list off three things that you offer that they want, and then they say, "Oh, yeah, we need social media management." Somebody else comes up to you, three things you want to offer, and they say, "Oh yeah, we need you to write content," and that's not something you do. You've just got to not do that. You've got to learn to shave off services that you can't offer. Instead, turn them over to people who can do them and do them very well.

What you're going to end up doing in your conversation, your sales pitch is, "Look, I'm going to be honest with you. We are great at some things, but this isn't our cup of tea. We know someone who's really great at it." That honesty, that candidness is just going to give them such a better relationship with you, and it's going to build a stronger relationship with those other specialty companies who are going to send business your way. So it's really important to learn to say no to say no service creep.

Well, anyway, there's a lot that we went over there. I hope it wasn't too much too fast, but hopefully we can talk more about it in the comments. I look forward to seeing you there. Thanks.

Video transcription by Speechpad.com

---

Ready for more?

You'll uncover even more SEO goodness in the MozCon 2020 video bundle. At this year's special low price of $129, this is invaluable content you can access again and again throughout the year to inspire and ignite your SEO strategy:

• 21 full-length videos from some of the brightest minds in digital marketing
• Instant downloads and streaming to your computer, tablet, or mobile device
• Downloadable slide decks for presentations

Get my MozCon 2020 video bundle

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Creative Diversification — More Hooks and Less Risk for Link Building

Posted by GeorgeRoot

As digital PRs we can often get stuck with our "campaign goggles" on, especially in the ideation and production stage of a creative campaign.

By this I mean, you have a preconceived idea of where you'd like your campaign to be featured, what kind of headlines you want it to achieve, and how people should read your data and story.

As we all know, we can't control the outcomes of a campaign, but we can certainly push them in the right direction.

To give your link building campaigns the best chance in the outreach stage, you need to make sure there is enough creative diversification during the production process, especially for data-led pieces and surveys. This opens up your "journalist pool" and gives you a ton more people to outreach to with a potential interest in your piece.

What is creative diversification?

Creative diversification is how you minimize the amount of risk in your link building campaign by ensuring your idea has enough breadth during the production process. It doesn't matter what format you’re using for each campaign — you always need to confirm it’s diverse enough to stand up in a changing news landscape. You want to develop an idea that can naturally explore multiple angles and sectors in the outreach phase. This flexibility needs to be set up before production, by exploring the potential outcomes and headlines you’re going after before you have them.

Find related topics

In the production stage, we obviously need to focus on our fundamental topic. This is often the domain's main reason for being. It could be finance, travel, fashion — you get the picture.

Then you want to start branching out and overlaying topics: finance + students, travel + safety, fashion + Elon Musk, and so on. You’re attempting to grab subtopics of interest.

Every link builder will have a different approach to discovering these topics, but the simplest way to get started is to grab a piece of paper and start scribbling ideas by word association. Just write as much as you can and you’ll find there’s lots of closely-related topic areas your content could delve into. (Tools like BuzzSumo would be invaluable here, but if you’re after a free alternative, I have been enjoying playing with AlsoAsked.com lately for related topic inspiration. Nothing is going to beat existing news content, though.)

It's also crucial to think about topic relevance, because if you question a tenuous link between your domain and topic matter, you can be certain journalists will, too. Link relevance is a whole other conversation to be had, but as long as it aligns with your client’s goals and you’re happy with showing them the link/coverage in full, you can’t go far wrong.

As a team at Root, we scrutinize our data points and approaches a lot in the production phase of each campaign and we find that championing personal expertise and curiosity often leads to some interesting statistics. My own passion for veganism gave us a unique angle which proved fruitful when we went out with a third round of outreach for our recent COVID-19 spending campaign.

Take off your campaign goggles

If the idea for your new campaign was born from your mind, you’re emotionally and personally invested whether you like it or not. You’ll need to put these feelings aside to engage with as many potential angles as possible from the start.

When I say you need to take off your campaign goggles, you need to (preferably with a colleague) tear apart the campaign and think about where you can add further value. It's best to approach this objectively, so if you can tackle a colleague’s campaign and vice-versa, even better.

Some link builders will look at their angles and opportunities only once the content has been created and consider it an outreach decision. Success is definitely possible this way, but you’re stopping yourself from being as successful as you might have been had you thoroughly drilled into your content before and during the production process.

Highlight the key areas and approaches you'd like to tackle beforehand and you can feed this into your outreach strategy later on.

Make sector-specific data for journalists

When creating media lists and discovering relevant journalists, link builders can often be encouraged to rush through and ignore the content itself. If you know what they’re writing about, both on Twitter and in publications, you can begin to think about what data you could craft specifically for them.

In the campaign I mention in this blog, we focused on side-hustle data related to the fundamental topic of how people are earning their money during the pandemic, which was directly influenced by journalists.

The journalist who covered this specific topic in USA Today fortunately tweeted a lot about the stories he was working on, so it made it incredibly easy for us to tailor some content toward his interest and later offer him the type of unique data he wanted.

Aside from keeping tabs on Twitter, you can also find out what they’re interested in through Google Discover and Reddit to understand what’s being talked about and what is topical.

I know many digital PRs review key publications directly on a regular basis and have big Feedly feeds or watch insight roundups on YouTube instead. Either way, thinking about what a journalist will need in the next few weeks is imperative to early planning and ensuring your campaign is diverse enough from the get-go.

Diversify outreach with hash URLs

Another way you can make certain your content is diversified and prepared for a breadth of outreach is through the use of URL fragments or "hash URLs". In the case of our coronavirus spending research campaign, we used article hooks on the page to provide anchor links from the table of contents at the top which then allowed us to offer another layer of personalization.

The key findings or headlines section in a table of contents is an essential piece to any long-form data campaign and makes it incredibly easy for journalists and readers to find the most relevant statistics to them in literally seconds.

If you’ve never implemented this yourself, there’s a simpler way than hooks — you just need to know your HTML basics. (Please excuse me if I butcher this description as a non-dev!) Place id="#subject" within the heading tag, so it would look like: <h2 id="#subject">.

In the example below, a BBC journalist used the URL with "#vegetarian" when referencing our statistics about plant-based food usage. This came from the ID tag and meant the journalist could link directly to the bits of research that was relevant to their article.

On top of that, we could send journalists semi-personalized links in our outreach, too. It’s a win-win and is best practice for users and search engine crawlers to navigate your long-form content anyway.

This is a literal manifestation of your creative diversification process early on, as it’s now been produced and each hash URL is an extra asset pointing journalists to the most relevant data for them.

Creative diversification in action

The campaign I’ve mentioned in this piece was a lengthy, yet simple, survey campaign for a fin-tech client asking Americans about their spending habits during the pandemic. We secured a range of coverage, but the three biggest placements we landed (BBC, CNBC, and USA Today) all covered different angles and data points from each other, but from this one survey, and that wasn’t an accident.

In the production stage, we knew we needed to focus on the campaign fundamentals: spending during the pandemic. Our related topics led us to grocery store spending and another leap encouraged us to look at food choices (were American’s eating more veg during lockdown? Hmm). These topics were still closely related to our core focus (finances) and therefore useful for our outreach in terms of securing relevant and high quality links.

When it came to the outreach strategy, we prioritized landing placements tied directly to the campaign fundamentals, then the related topics fed into the consecutive rounds which we chose depending on the strength of the data we received from the survey.

If you’re thinking in the production process that there’s too much going on with too many angles, you may have just created multiple mini content campaigns for yourself.

We’ve found time and time again that the simpler stories and slimmer, more targeted outreach emails will land placements way more often than bloated emails trying to offer up far too much content at once.

That’s not to say that you should automatically split up larger pieces of content, but your outreach should be the final step in diversifying your piece. A data analysis research piece that taps into multiple sectors should simply highlight the most relevant information to the journalist in bite-size sections. We gave grocery spend data to retail business journalists, vegan food consumption data to food writers, and side-hustle data to those writing on the latest employment trends.

The next time you’re creating a content campaign, have your team (even if that’s just you) ruthlessly find new sectors, journalists, and angles to target, to ensure your next piece is as diverse as possible. Creative diversification = more hooks and less risk.

---

To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.

Take the Survey

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Fifteen Years Is a Long Time in SEO

Posted by willcritchlow

I’ve been in an introspective mood lately.

Earlier this year (15 years after starting Distilled in 2005), we spun out a new company called SearchPilot to focus on our SEO A/B testing and meta-CMS technology (previously known as Distilled ODN), and merged the consulting and conferences part of the business with Brainlabs.

I’m now CEO of SearchPilot (which is primarily owned by the shareholders of Distilled), and am also SEO Partner at Brainlabs, so… I’m sorry everyone, but I’m very much staying in the SEO industry.

As such, it feels a bit like the end of a chapter for me rather than the end of the book, but it has still had me looking back over what’s changed and what hasn’t over the last 15 years I’ve been in the industry.

I can’t lay claim to being one of the first generation of SEO experts, but having been building websites since around 1996 and having seen the growth of Google from the beginning, I feel like maybe I’m second generation, and maybe I have some interesting stories to share with those who are newer to the game.

I’ve racked my brain to try and remember what felt significant at the time, and also looked back over the big trends through my time in the industry, to put together what I think makes an interesting reading list that most people working on the web today would do well to know about.

The big eras of search

I joked at the beginning of a presentation I gave in 2018 that the big eras of search oscillated between directives from the search engines and search engines rapidly backing away from those directives when they saw what webmasters actually did:

While that slide was a bit tongue-in-cheek, I do think that there’s something to thinking about the eras like:

1. Build websites: Do you have a website? Would you like a website? It’s hard to believe now, but in the early days of the web, a lot of folks needed to be persuaded to get their business online at all.
2. Keywords: Basic information retrieval became adversarial information retrieval as webmasters realized that they could game the system with keyword stuffing, hidden text, and more.
3. Links: As the scale of the web grew beyond user-curated directories, link-based algorithms for search began to dominate.
4. Not those links: Link-based algorithms began to give way to adversarial link-based algorithms as webmasters swapped, bought, and manipulated links across the web graph.
5. Content for the long tail: Alongside this era, the length of the long tail began to be better-understood by both webmasters and by Google themselves — and it was in the interest of both parties to create massive amounts of (often obscure) content and get it indexed for when it was needed.
6. Not that content: Perhaps predictably (see the trend here?), the average quality of content returned in search results dropped dramatically, and so we see the first machine learning ranking factors in the form of attempts to assess “quality” (alongside relevance and website authority).
7. Machine learning: Arguably everything from that point onwards has been an adventure into machine learning and artificial intelligence, and has also taken place during the careers of most marketers working in SEO today. So, while I love writing about that stuff, I’ll return to it another day.

History of SEO: crucial moments

Although I’m sure that there are interesting stories to be told about the pre-Google era of SEO, I’m not the right person to tell them (if you have a great resource, please do drop it in the comments), so let’s start early in the Google journey:

Google’s foundational technology

Even if you’re coming into SEO in 2020, in a world of machine-learned ranking factors, I’d still recommend going back and reading the surprisingly accessible early academic work:

• The Anatomy of a Large-Scale Hypertextual Web Search Engine by Sergey Brin and Lawrence Page [PDF]
• Link Analysis in Web Information Retrieval [PDF]
• Reasonable surfer (and the updated version)

If you weren’t using the web back then, it’s probably hard to imagine what a step-change improvement Google’s PageRank-based algorithm was over the “state-of-the-art” at the time (and it’s hard to remember, even for those of us that were):

Google’s IPO

In more “things that are hard to remember clearly,” at the time of Google’s IPO in 2004, very few people expected Google to become one of the most profitable companies ever. In the early days, the founders had talked of their disdain for advertising, and had experimented with keyword-based adverts somewhat reluctantly. Because of this attitude, even within the company, most employees didn’t know what a rocket ship they were building.

From this era, I’d recommend reading the founders’ IPO letter (see this great article from Danny Sullivan — who’s ironically now @SearchLiaison at Google):

“Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating.”

“Because we do not charge merchants for inclusion in Froogle [now Google shopping], our users can browse product categories or conduct product searches with confidence that the results we provide are relevant and unbiased.” — S1 Filing

In addition, In the Plex is an enjoyable book published in 2011 by Steven Levy. It tells the story of what then-CEO Eric Schmidt called (around the time of the IPO) “the hiding strategy”:

“Those who knew the secret … were instructed quite firmly to keep their mouths shut about it.”

“What Google was hiding was how it had cracked the code to making money on the Internet.”

Luckily for Google, for users, and even for organic search marketers, it turned out that this wasn’t actually incompatible with their pure ideals from the pre-IPO days because, as Levy recounts, “in repeated tests, searchers were happier with pages with ads than those where they were suppressed”. Phew!

Index everything

In April 2003, Google acquired a company called Applied Semantics and set in motion a series of events that I think might be the most underrated part of Google’s history.

Applied Semantics technology was integrated with their own contextual ad technology to form what became AdSense. Although the revenue from AdSense has always been dwarfed by AdWords (now just “Google Ads”), its importance in the history of SEO is hard to understate.

By democratizing the monetization of content on the web and enabling everyone to get paid for producing obscure content, it funded the creation of absurd amounts of that content.

Most of this content would have never been seen if it weren’t for the existence of a search engine that excelled in its ability to deliver great results for long tail searches, even if those searches were incredibly infrequent or had never been seen before.

In this way, Google’s search engine (and search advertising business) formed a powerful flywheel with its AdSense business, enabling the funding of the content creation it needed to differentiate itself with the largest and most complete index of the web.

As with so many chapters in the story, though, it also created a monster in the form of low quality or even auto-generated content that would ultimately lead to PR crises and massive efforts to fix.

If you’re interested in the index everything era, you can read more of my thoughts about it in slide 47+ of From the Horse’s Mouth.

Web spam

The first forms of spam on the internet were various forms of messages, which hit the mainstream as email spam. During the early 2000s, Google started talking about the problem they’d ultimately term “web spam” (the earliest mention I’ve seen of link spam is in an Amit Singhal presentation from 2005 entitled Challenges in running a Commercial Web Search Engine [PDF]).

I suspect that even people who start in SEO today might’ve heard of Matt Cutts — the first head of webspam — as he’s still referenced often despite not having worked at Google since 2014. I enjoyed this 2015 presentation that talks about his career trajectory at Google.

Search quality era

Over time, as a result of the opposing nature of webmasters trying to make money versus Google (and others) trying to make the best search engine they could, pure web spam wasn’t the only quality problem Google was facing. The cat-and-mouse game of spotting manipulation — particularly of on-page content, external links, and anchor text) — would be a defining feature of the next decade-plus of search.

It was after Singhal’s presentation above that Eric Schmidt (then Google’s CEO) said, “Brands are the solution, not the problem… Brands are how you sort out the cesspool”.

Those who are newer to the industry will likely have experienced some Google updates (such as recent “core updates”) first-hand, and have quite likely heard of a few specific older updates. But “Vince”, which came after “Florida” (the first major confirmed Google update), and rolled out shortly after Schmidt’s pronouncements on brand, was a particularly notable one for favoring big brands. If you haven’t followed all the history, you can read up on key past updates here:

A real reputational threat

As I mentioned above in the AdSense section, there were strong incentives for webmasters to create tons of content, thus targeting the blossoming long tail of search. If you had a strong enough domain, Google would crawl and index immense numbers of pages, and for obscure enough queries, any matching content would potentially rank. This triggered the rapid growth of so-called “content farms” that mined keyword data from anywhere they could, and spun out low-quality keyword-matching content. At the same time, websites were succeeding by allowing large databases of content to get indexed even as very thin pages, or by allowing huge numbers of pages of user-generated content to get indexed.

This was a real reputational threat to Google, and broke out of the search and SEO echo chamber. It had become such a bugbear of communities like Hacker News and StackOverflow, that Matt Cutts submitted a personal update to the Hacker News community when Google launched an update targeted at fixing one specific symptom — namely that scraper websites were routinely outranking the original content they were copying.

Shortly afterwards, Google rolled out the update initially named the “farmer update”. After it launched, we learned it had been made possible because of a breakthrough by an engineer called Panda, hence it was called the “big Panda” update internally at Google, and since then the SEO community has mainly called it the Panda update.

Although we speculated that the internal working of the update was one of the first real uses of machine learning in the core of the organic search algorithm at Google, the features it was modelling were more easily understood as human-centric quality factors, and so we began recommending SEO-targeted changes to our clients based on the results of human quality surveys.

Everything goes mobile-first

I gave a presentation at SearchLove London in 2014 where I talked about the unbelievable growth and scale of mobile and about how late we were to realizing quite how seriously Google was taking this. I highlighted the surprise many felt hearing that Google was designing mobile first:

“Towards the end of last year we launched some pretty big design improvements for search on mobile and tablet devices. Today we’ve carried over several of those changes to the desktop experience.” — Jon Wiley (lead engineer for Google Search speaking on Google+, which means there’s nowhere to link to as a perfect reference for the quote but it’s referenced here as well as in my presentation).

This surprise came despite the fact that, by the time I gave this presentation in 2014, we knew that mobile search had begun to cannibalize desktop search (and we’d seen the first drop in desktop search volumes):

And it came even though people were starting to say that the first year of Google making the majority of its revenue on mobile was less than two years away:

Writing this in 2020, it feels as though we have fully internalized how big a deal mobile is, but it’s interesting to remember that it took a while for it to sink in.

Machine learning becomes the norm

Since the Panda update, machine learning was mentioned more and more in the official communications from Google about algorithm updates, and it was implicated in even more. We know that, historically, there had been resistance from some quarters (including from Singhal) towards using machine learning in the core algorithm due to the way it prevented human engineers from explaining the results. In 2015, Sundar Pichai took over as CEO, moved Singhal aside (though this may have been for other reasons), and installed AI / ML fans in key roles.

It goes full-circle

Back before the Florida update (in fact, until Google rolled out an update they called Fritz in the summer of 2003), search results used to shuffle regularly in a process nicknamed the Google Dance:

Most things have been moving more real-time ever since, but recent “Core Updates” appear to have brought back this kind of dynamic where changes happen on Google’s schedule rather than based on the timelines of website changes. I’ve speculated that this is because “core updates” are really Google retraining a massive deep learning model that is very customized to the shape of the web at the time. Whatever the cause, our experience working with a wide range of clients is consistent with the official line from Google that:

Broad core updates tend to happen every few months. Content that was impacted by one might not recover — assuming improvements have been made — until the next broad core update is released.

Tying recent trends and discoveries like this back to ancient history like the Google Dance is just one of the ways in which knowing the history of SEO is “useful”.

If you’re interested in all this

I hope this journey through my memories has been interesting. For those of you who also worked in the industry through these years, what did I miss? What are the really big milestones you remember? Drop them in the comments below or hit me up on Twitter.

If you liked this walk down memory lane, you might also like my presentation From the Horse’s Mouth, where I attempt to use official and unofficial Google statements to unpack what is really going on behind the scenes, and try to give some tips for doing the same yourself:



SearchLove San Diego 2018 | Will Critchlow | From the Horse’s Mouth: What We Can Learn from Google’s Own Words from Distilled

---

To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.

Take the Survey

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

Our Lady Of Grace Royal Blue Rosary

 Every purchase at Christian Catholic Shop helps support Catholic evangelization!

You Will Love This Beautiful Our Lady Of Grace Royal Blue Rosary with beautiful 6mm royal blue marble colored beads and 8mm capped faux pearl beads. Special Limited Edition! Made of High Quality Materials. Makes a beautiful gift. Order now while in stock!

blue rosary