Quality Score: Better with Age?

Posted on Wednesday, July 7th, 2010 by Print This Post Print This Post

Categories - Featured, SEM

In my last posting on the subject of Quality Score, I made the argument that, if we boiled QS down to what it really is, we’d have to call it a “bulk discount program.” Because click-through rate weighs so heavily in the Quality Score equation, the lower CPCs we see as a result of higher Quality Scores basically amount to Google telling advertisers, “This advertising real estate is more valuable to us if more people click on your ad; therefore the higher you can push your CTRs, the more of a discount we’ll give you on CPCs.” I stand by that position… all else being equal. But in the time since that posting, we’ve uncovered some strong evidence that there may be another variable in the equation that can under certain circumstances trump even the almighty CTR. That variable is none other than Father Time. It appears that sometimes the best thing you can do to improve Quality Score is absolutely nothing. Here’s a quick outline of what we’ve seen and done in recent weeks: 1)      Before we took over management of an account from one of our clients, the client had built out a number of geo-modified ad groups around important keywords. 2)      The fundamental structure of the “legacy” geo campaigns was sound. 3)      As part of our ongoing expansion and optimization of this account, we continued to add new geo-modified campaigns structured in the same way as the legacy versions, with ads that were also fundamentally the same, with only slight alterations aimed at raising CTR. 4)      After a few months, we were puzzled by the results, which looked like this: In other words, despite identical structure and setup with similar ad formats across all geo sets, AND despite CTR being more than 50% higher with the new set, Quality Score remained 50% higher for the legacy set. As confounding as this was – and as tempting as it is to throw our hands up and walk away any time we come across these kinds of seemingly arbitrary Quality Score issues – we dug deeper. As stated above, the structure of the ad groups was identical: each ad group in both the old and new sets was a single keyword concatenated with a large list of geographic names. Landing pages specific to the “thing+geo” combination were used for each individual keyword in both sets, and there was no difference in landing page design elements. In other words, the only thing we could think of that was different in any potentially notable way was the ad copy. So this is what we did: 1)      Analyzed the editorial differences between the ads running in the two sets. As mentioned above, they were fundamentally the same, but ultimately they were not exactly the same. These were the three key differences that we found in comparing the ads in the legacy ad groups to those in our newer ad groups:
  1. New ad groups used Keyword Insertion somewhat more often than old ad groups.
  2. New ads used customized Display URLs (e.g. Website.com/SpecialThing), while old ads used a single, generic Display URL (www.website.com) across the board with no exceptions.
  3. The client in question has coverage in a wide variety of verticals, and the way we expressed that in the newer ads that we’d written was something to the effect of “Find What You’re Looking For”; while the old ads expressed this by saying, “Find the details of [xyz] at website.com.”
2)      We isolated 3 different cross-sections of newer geo-modified ad groups, and applied one “legacy element” to each set. So in one cross-section we got rid of Keyword Insertion; in the second one we eliminated the customized Display URLs; in the third we changed the wording of the “find it here” message to match that of the legacy group.
  1. Results: CTR, Quality Score and CPCs took a nosedive across the board.
3)      Not giving up, we stretched a little bit further and hypothesized that maybe there’s something about the “alchemy” of those 3 distinguishing elements in combination that led to a better Quality Score. So we took the 3 test sets we’d been working with and applied all 3 differentiators to all 3 sets.
  1. Results: CTR, Quality Score and CPC took an even bigger nosedive.
4)      We reverted all of our test sets back to the ads that we had written, which pushed CTR back to its higher levels, and we’re basically crossing our fingers and hoping that the better CTRs will eventually translate to better Quality Scores and lower CPCs. We’ll see. To be fair, this wasn’t an entirely perfect test. As mentioned above, the client in question here covers a variety of verticals; and as we all know, different types of searchers in different categories respond differently to different types of messaging. However, the way we set up the tests should have accounted for that to a good degree. We tested a very diverse cross-section in each test set, and when comparing our core test metrics (CTR, QS, CPC), we only did so within each category individually. Still, in all cases, each set had degraded in QS and click metrics within a few weeks after making each round of changes. So what’s left to test? Time. The geo campaigns that were already there when we took over the account had been around for months and in most cases years, humming along unwatched and untouched, delivering efficient traffic at a solid CPC. Then we came in, expanded the geo campaigns to cover more keyword areas, wrote the same kinds of ads with slight variations that led to better CTRs… and in the process killed Quality Score. In short, aside from any sort of variables that we simply can’t perceive (“dog-whistle variables” I like to call them), it appears that the only major difference between the legacy geo campaigns and the new geo campaigns in this account is that the older ones are, well, older. I can’t explain why, but if there’s any validity at all to our testing methodology, it appears that the AdWords Quality-Score algorithm does indeed favor ad-keyword combinations that have been together for a long time. (Has Google, as a side-effect of its myriad revolutionary advancements in search technology, managed to quantify trust?) It’s a tough conclusion to communicate to a client (is there anything about Quality Score that isn’t?). You are asked, “What do you recommend we do this month to improve Quality Score in our campaigns?” Depending on the circumstances, it appears that your best answer may very well be, “Absolutely nothing.”

Tags | , ,

2 Responses to “Quality Score: Better with Age?”

  1. These are the most amusing dogs I ever seen! You ought to find out about it! Click the link: Lol! Funny Dogs

  2. hi, I uncovered some fine articles you may be liked to see. Please stop by: Click here

Trackbacks/Pingbacks


Leave a Reply

Follow Us on Twitter

Authors