Google says that an ad’s clickthrough rate (CTR) is the primary component of its Quality Score because (according to Nicholas Fox, Google’s Director of Product Management) “if lots of people ‘tell’ Google that an ad is ‘high quality’ by clicking on it, then Google believes it’s a high quality ad.”
But, on the AdWords Help page called ‘What is ‘Quality Score’ and how is it calculated?’, they give a whole laundry list of factors that they say help to determine QS, including: the historical clickthrough rate, your account’s history, the quality of the ad’s landing page, the relevance of the ad to the search query, and the catch-all category of “other relevance factors”. Brad Geddes in one of his BGTheory posts says: “There are over 100 factors that can affect quality score. However, not all will be triggered depending on the conditions involved.” (The second sentence is a reference to the fact that there are actually multiple Quality Scores, not just one, and that they are determined in different ways and for different uses. He provides a useful reference table.)
It’s no wonder that search marketers are often mystified to the point that some throw their hands in the air and resign themselves to thinking that Quality Score is just a fairly arbitrary measure of ‘how good Google thinks that your ad is’. Google doesn’t go out of its way to help to clear the air much, either. In ‘Introduction to the Google Ad Auction’ video on YouTube, Google’s chief economist Hal Varian spends the first few minutes explaining how their auction system works. The concept is familiar to anyone who has been in the SEM industry long enough to know the difference between ‘CTR’ and ‘CPC’ – For each ad, Google multiplies that advertiser’s bid by the ad’s Quality Score to determine a value called the ‘AdRank’:
Each ad is then listed on the results page in order from highest AdRank to lowest. In this sense, AdRank is sort of a ‘weighted bid’, with the Quality Score acting to make dollars from some advertisers worth more to Google than other dollars. In a later part of the video, Dr. Varian shows a pie chart that breaks down the Quality Score into its constituent parts. About 60% of the pie appears to be the ad’s CTR, about 30% its ‘relevance’, and the remaining 10% attributed to the ‘landing page quality’. Any reasonable person watching the video should be impelled to blurt out: “Wait a minute, Hal! What happened to those 97 other factors Brad Geddes mentioned? You only show 3!”
Perhaps more important than what Dr. Varian does mention are the things he does not talk about, like the existence of multiple Quality Scores. Discussing what constitutes a good-quality landing page, Hal mentions that the page should have “original content”, a “quick load time”, and be “easily navigable”. A naive interpretation of this would be to think that Google loads your landing page, counts how many seconds it takes to receive the HTML file, and then tabulates this time as part of your ad’s Quality Score, like some sort of text-based beauty contest, with the participants scoring some points for poise, a few more for looks, a couple for talent and so forth. In fact, most of those other 97 factors are likely what Craig Danuloff of ClickEquations calls ‘bozo filters,’ that is, pass/fail tests that determine whether or not your ad is allowed to participate in a given auction, but not used after that. As long as the HTML arrives in less than some fairly generous cutoff value (say, 30 seconds), then your ad gets a passing grade on that factor. Get passing grades on all those ~97 basic little tests and your ad is permitted to participate in the auction. So, even though the CTR wedge looks like its about 60% of the Quality Score pie, its role within the ad auction is probably much greater than that. (In fact, in his 2007 article “Position Auctions” analyzing the properties of in-auction metrics, Dr. Varian concerns himself with only one component of Quality Score: the clickthrough rate. Suddenly, our list of ‘over 100′ factors is reduced to just one!)
Another thing that Dr. Varian doesn’t mention is how Google stores the Quality Score for a given ad. On the whiteboard in his video, he shows the Quality Score as being numbers like ’1′, ’3′, and ’6′, but in fact, Quality Score is actually calculated and stored as a number between 0 and 1. (This sounds minor, but is actually very important.) Hal doesn’t mention that the value Google calculates for the Quality Score (the ‘actual Quality Score/s’) and the value that they report to you (the ‘reported Quality Score‘) are not the same thing. In our experience, most reported values of the Quality Score in most accounts are 7, with a few ads having 8, 9 or 10 and a few being 4, 5 or 6. This means that, if you want to maximize a given ad’s Quality Score, you will have a more-difficult time doing so by looking at the reported QS than just by looking at the ad’s CTR directly. It takes an enormous change in CTR to see the reported QS go from 6 to 7, or from 7 to 8, but even a tiny increase in CTR will help improve the actual 0-to-1 number stored in Google’s system. To maximize your Quality Score, then, just focus on maximizing your ad’s CTR.
He also doesn’t give us a list of things that (almost) certainly do not affect an ad’s Quality Score, like: your bid, your budget, and your keyword’s match types. (Quality Score is only calculated from exact-match queries for a given keyword. Thus, all the matchtypes for a given keyword have the same QS.) Some people raise bids, especially on new terms, thinking that Google might reward them with a higher QS, but examining the equation for AdRank above shows why this is counterproductive to Google: for any value of QS that is based on the bid, there is some other value (not based on the bid) that yields the same AdRank. Rather than include the bid in the calculation of the Quality Score, the search engine can just leave it to be a separate factor in the equation and get the same result. (Another reason that some account managers raise bids on new terms is to help increase their CTR, thinking that the CTR’s from higher positions will lead to a higher Quality Score, but that brings up another point Dr. Varian was silent about.)
One last thing to mention that Dr. Varian neglects is the fact that observations of your CTR are actually adjusted to account for their position. In other words, it’s not your absolute clickthrough rate that matters – it’s your CTR relative to other ads for that keyword. Each time your ad is shown, Google records the position and whether or not the ad got a click. This means that they not only know your ad’s observed CTR, but also the average CTR of all ads in all positions. So they can estimate how an ad would have done if it had been shown in, say, position 1 even it has never (or not recently) been in that position. Thus, every ad can be compared in a fair way to all of the other ads in the auction.
Google didn’t always take clickthrough rates into account when selling ad space. In the excellent article ‘Secret of Googlenomics‘ in the May 2009 issue of Wired magazine, Steven Levy reminds us that, way back around the turn of the century, Google used to sell ads by the impression. “Advertisers were … billed by the number of user views, or impressions, regardless of whether anyone clicked on the ad,” he wrote. The problem Google faced was: how to price those impressions. Too low meant giving away revenue they could have gotten for themselves. Too high meant lots of disappointed and angry advertisers. And even the advertisers themselves didn’t know what the impressions were worth. So, Google decided instead to sell their ad slots by a PPC auction system. Perhaps the most-obvious way to do this is to just rank ads by their bid (that is, AdRank = bid) and leave the advertisers to jockey for position amongst themselves. But the problem with this ranking method is that it encourages bad ads (that is, those with very low position-adjusted CTRs) to get bid into high position. Since advertisers only pay for clicks, not impressions, those who have low CTRs can afford to drive legitimate advertisers out of the top positions.
For Google’s finance department, the only equation important to Google is:
So, there are only two ways for Google to make more money: Get more people to search more often, and make more money from each search. Searchers were already taking care of the first part on their own. All Google needed to worry about was maximizing revenue per impression by placing the ad with the highest ‘expected revenue per impression’ at the top and all others in order below it. The bid is in ‘dollars per click’, so, if there is some factor that gets multiplied by the bid which has the dimensions of ‘clicks per impression’, then the AdRank has the dimensions of ‘dollars per impression’:
A more-general solution, rather than make the quality score be equal to the clickthrough rate, is to make the quality score be equal to CTR raised to some power (let’s call it ‘X’, though in academic literature it’s often called the ‘squashing parameter‘):
When X equals 0, then the AdRank equals the bid and the auction reverts back to the ‘rank by bid’ scheme. When X is 1, the auction becomes a ‘rank by revenue’ system and each ad gets sorted from highest expected revenue (to Google) per impression to lowest expected revenue per impression.
When X is very large (say, 3), then CTR becomes far more important than the bid and the system is then effectively ‘rank by relevance’ – Search engine users indicate which ads are the most relevant by clicking on them and those that get the most clicks then get the highest position. The bid in such a system plays no important role in ranking ads. (Since bids clearly do play an important role in PPC advertising, we can be certain that Google doesn’t use a value of X as high as this.)
So: Why is CTR the primary component of Quality Score? In short, because it has to be. The fact remains that Google is in the business of selling impressions, not clicks, regardless of what rules and changes they make behind-the-scenes in order to do that as best as they can. They can toss in some pass/fail filters to keep the Viagra-peddlers out of the top slots, and they can adjust the X value up or down (across-the-board, or, by keyword) to adjust the amount they listen to the bids relative to the amount they listen to the (searcher-generated) CTRs, but they can’t change the financial physics of selling ad space efficiently: Ads must be ranked by the bid times some quantity that depends largely on the CTR or else the auctioneer doesn’t maximize its own long-term revenue. (Lahaie and Pennock, in their article “Revenue Analysis of a Family of Ranking Rules for Keyword Auctions”, found that even the adjustments to X can’t be made without consequence for Google: the optimal value at any time depends on the correlation between the advertisers’ value-per-click and their CTRs, not on any major factor that Google controls.) Google has not chosen the formula for Quality Score arbitrarily; in fact, their hands are basically tied by the mathematics. Why is CTR the primary component of Quality Score? Because some function of CTR is all that the QS needs to be. There are three participants in the AdWords auction: the advertisers, the searchers, and the search engine. Only a system that gives each a voice (the bid, the CTR, and the squashing parameter, respectively) will satisfy them all in the long run.
The lesson for the search marketer is simple: Regardless of what Google does to X (or any other factor you can’t see or change anyhow), you only need to (1) make certain you pass the bozo filters and, when you do, (2) optimize your ads’ clickthrough rates relative to your competitors. No other question about Quality Score seems to matter very much in comparison.