I found an interesting post from PotPieGirl this morning that included a link to Google's 2011 Quality Raters Handbook. I'm not going to talk about the same things that she talks about, so I recommend clicking over there and reading her post.

Near the end, she writes:

...if we can each train ourselves to be Google Raters, we will be able to put out the content that Google DOES want to rank. No need to game them, we simply just give them the quality they are looking for -- which I believe is what Google wants, don't you?

Sounds good to me.

The document is 125 pages long, so as you might have guessed, I didn't read every word this morning. But I skimmed it all and read most of the meat of it. Here are a few parts I thought you might find useful.

The Rating Scale

In addition to some flags that operate independently of the rating scale (like spam and porn), raters rate search results on the following scale:

  • Vital
  • Useful
  • Relevant
  • Slightly Relevant
  • Off-Topic or Useless

Note that I said they rate search results, not pages. The difference is critical. A high-quality page may be rated "Useful" for one search query, but "Off-Topic or Useless" for another. (See the examples on pages 25 and 26). It makes perfect sense once you think of it. Quality raters are assessing how well a page fulfills the intent of the person who's searching.

Which brings up another important point.

Classification of User Intent

Beginning on page 9 is a discussion of three types of searcher intent:

  • Action queries: the user is looking for a place to do something (buy a product, download something, watch a video, etc.) Helpful pages enable the person to do that thing.
  • Information queries: the users wants to know something. "Helpful pages have high quality, authoritative, and comprehensive information about the query." (page 11)
  • Navigation queries: the user is trying to find a single, specific page - an official company homepage, a store locator page, a particular restaurant's menu, etc.

Some queries have more than one possible intent. Others have a single, dominant intent, and pages are rated relative to it. To be rated high, a page has to be relevant not only to the words in the query, but to the intent class of the query.

Reader Comment:
Deep Karmakar said:
This very article of yours have provided me valuable insights. How websites or pages are rated by quality raters, have been discussed thoroughly. I've even bookmarked this page so that I can go through this article any time I want. Thanks a lot !!!!
(join the conversation below)

"Relevant" vs. "Useful"

Page 21 explains the distinction between "relavant" and "useful":

Relevant pages should still "fit" the query, but they might be less comprehensive, less up-to-date, come from a less authoritative source, or cover only one important aspect of the query.

From this and other places in the document, it's clear that if a query is broad, the best pages are those that comprehensively cover the whole topic. If the query is narrow, the best pages are those that cover specifically what the query asked for. A top-notch page that covers one specific topic won't be rated as high as a more general page for a general query, even if it's an excellent source of information for part of what the searcher is looking for.

Also, the authority of the source is significant. An excellent article on diabetes hosted on a website with articles about every topic under the sun is going to be considered less authoritative than an article hosted on a the website of a credentialed medical establishment, and that's going to affect its rating.

Copied Content

Webmasters often worry about the so-called "duplicate content penalty" when considering copying content. Page 24 discusses the "Slightly Relevant" rating, including guidelines for when copied content do or don't suggest a "Slightly Relevant" rating:

Please note that not all pages with copied content are considered "low quality". The website www.answers.com contains content copied from Wikipedia.org and other dictionary and encyclopedia sites, but is not considered to be a low quality site because the content is well-organized and intended to be helpful for users. Similarly, there are pages on medical information sites that contain copied content. If the page is well-organized and appears to be designed to be helpful for users and not just to display ads for users to click on, it should be rated based on how helpful the content would be for users.

The 10 Pillars of Wealth: Mind-Sets of the World's Riches...
List Price: $12.99
Amazon Price: $10.46
Forever Young
List Price: $18.99
Amazon Price: $17.09

So for example, a proper Blog Riff would not be downgraded for copying a quote from the post it Riffs on, because it contains the Riffer's original thoughts and is designed to expand on an idea, not just mooch off someone else's work. It's harder to guess how simple curated content with no additional commentary would be rated. Would it get credit for selecting high-quality content, or would it be rated low because it's just a copy? After all, it's difficult to argue that the original shouldn't be rated higher, even though the curator is providing a valuable service to people who happen to land on their site first.

List Queries

Page 43 discusses queries whose intent is to find a list -- for example, "chicken recipes".

  • When the query seems to ask for a list that includes many, many possibilities, individual examples usually aren't as helpful as a list.
  • When the list of possibilities is short, then individual examples are helpful.
  • Sometimes, there are very famous or popular examples on the list. In these cases, the individual famous or popular examples are helpful, even if the list of possibilities is long.


Page 95 defines what is considered spam:

Webspam is the term for webpages that are designed by webmasters to trick search engines and draw users to their websites.

It goes on to explain that the spam flag and quality rating are independent. Even a page rated "Vital" may be flagged as spam if it uses deceptive techniques.

Affiliate Sites

Some people believe that Google hates affiliates. The raters guidelines suggest that they have a general suspicion of affiliates and makes it clear that they consider thin affiliates to be spammers. A thin affiliate is an affiliate who doesn't offer unique, useful information.

Page 45 links to a thin affiliate page and says:

This page is spam (see the Webspam Guidelines, Part 4 of the General Guidelines, for more information). Clicking the product links takes you to Amazon. Nothing can be purchased on the landing page.

And page 81 contains this note:

Major cosmopolitan cities are preferred targets for spammers, especially hotel affiliates. Such results should be flagged as Spam, even if they are related to the query and helpful to users. For example, a hotel affiliate page with a list of Chicago hotels may be assigned a rating Relevant, but also receive a Spam flag.

However, other page (...which I didn't note specifically) indicate that affiliate sites that offer comparisons of different products or merchants selling the same product or other unique, useful information that would be helpful to consumers, may be rated high.

Overall, it was an interesting read -- definitely worth skimming if you care about your search engine rankings.