For this case, I grouped the campaign traffic based on position. This grouping method would tell us how many impressions and clicks our campaign got at position 1-3, 3-5 and so on, and at what cost.

This table looks good at first glance, but what does it tell us? It tells us that positions 1-3 gave us the maximum traffic and cost more. But if we really want to compare the performance based on positions, then we need to look at CTR and CPC.

Let’s take a look:

According to this table, whenever ads in the campaign showed up at Position 5 or lower, I actually had a higher CTR and lower CPC. Strange, but true. According to this data, we should look only at showing our 5+, right? It’s still too early to make that decision.

Lets first look at percentage of traffic (clicks):

This makes things a little clearer. I spent 78% of my cost on ads appearing in positions 1-3 which gave me 64% of my traffic. This isn’t too efficient, but it’s still driving a large volume of traffic.  But traffic in Positions 3-5 also seem to be doing well in terms of CPC, so how do we decide what the ideal position is for this campaign?

That, in my opinion, would depend on the goal of the campaign. If the aim is drive more traffic, and in return awareness, then CTR is the metric we want to optimize.  The above analysis was done for a brand advertiser for a small sample size, but a similar analysis could be run for performance based campaigns, in which conversion data could provide further insight.

For advertisers running performance/conversion campaigns, there are several possible scenarios. For example, clicks from a higher position could result in more conversions per click, in which case a higher position would be positive. But if the conversion rate is found to be fairly uniform across positions, then it might make sense to go for the least CPC position. Of course, this is assuming that we have enough impressions at the lower position.

If you are a marketer who regularly sets bids based on position to maximize coverage and/or conversions, this type of analysis can provide further insights into which position is best for a specific campaign.  With this information you can start improving the efficiency of your campaign- rather than chasing the ‘holy’ position 1.



In search marketing, our mission is not to seek out new life and new civilizations, but rather the largest number of profitable dollars. For many online enterprises, this prime directive boils down to just:

That’s it. The simple truth is that the only ways for most companies to increase monthly profit are to increase the number of orders placed each month or the average profit per order (or both). The number of online orders per month is the number of searches times your impression share (IS) multiplied by the clickthrough rate (CTR) multiplied by your conversion rate (CR), so:

I call this The Drake Equation of Search Marketing. To some extent, it represents the actual mechanics of an online conversion. First, a search occurs. Some fraction (IS) of these result in an impression. Another fraction (CTR) of those impressions result in clicks, and some other fraction (CR) of those result in orders.

Every action we take as online marketers is intended to increase, directly or indirectly, one or more of these variables (without detrimentally impacting any of the others). But, of these factors, only the last two happen at your website. There, marketers apply landing page optimization (LPO), the purpose of which is typically considered: “to maximize the conversion rate (CR)”. However, we can see from the equation that there’s another facet of LPO whose purpose should be: “to maximize the average profit per order”. (Alternatively, one can simply combine these two factors, CR times ‘average profit per order’, to get the ‘average profit per click’, which is the overall metric by which the efficacy of various LPO efforts should be compared.)

The first three of the steps in this equation (searches, impressions, and clicks) occur on the search engine, where we do have some influence. Many people know that rank (a.k.a., position) strongly affects CTR, but so does the relative attractiveness of a listing.

PPC marketers experiment with the title, description text, and display URL in ad creatives to boost the CTR [Though again, they should really be maximizing CTR X 'profit per visit', which is 'profit per impression']. Similarly, SEOs influence organic results, since the title of a natural listing is often taken from the page’s title tag and the description from the meta tag or from text on the page. Online marketers often define the purpose of SEO to be something like “to get pages ranked as highly as possible for terms that are key to a site’s business”, but from the graphic above we can see that another facet to SEO is to make natural listings more attractive to searchers irrespective of any efforts to affect the listing’s position.

The primary factor which determines position in natural listings is ‘relevance’, which is partially found at the time of the query based on the page content’s quality and congruence with the search intent, but which is also partially determined by measures of the page’s ‘popularity’ which are calculated long prior to the query. We desire inbound links since each inbound link can be thought of, in some sense, as a vote by other website authors for our page, which boosts our PageRank. However, clicks at a search engine can also be viewed as votes for the relevance of our site, in this case, votes by the users of the search engine. Thus, click traffic becomes a self-reinforcing cycle, with highly relevant pages receiving more clicks (thus, a higher historical CTR) and therefore being treated as relevant in future searches.

The variables we probably influence the least are those at the beginning of the chain: the number of searches per month and our impression share. A massive off-line advertising campaign might drive additional searches for key terms, but as Frank Lee pointed out in ‘2012: We Weren’t Warned, Only Teased‘, even this is not guaranteed to produce traffic, let alone conversions. And, adding unique content to our site could increase our impression share, since it makes us more relevant to additional queries. But other than those, our best options are simply to promote good architecture to make our site easy to navigate (read: easy for search engines to spider) and to avoid hosting malware, or appear spammy or any of the multitude of reasons for search engines to switch our impressions off entirely.

This creates a problem for advertisers who have good-quality ads whenever poor-quality ads appear in the same set of results, especially in cases where those poor-quality ads are in higher positions. For example, I entered a search query into Google that, to the best of my knowledge, I had never used previously: [designer shoes].  Look closely at the ads that appeared on the right-hand side of the screen:

The first ad starts “Sale on Designer Shoes”. So far, so good, even though the text itself looks like it was written by someone who had chewing gum stuck on their keyboard. No big deal though, I’ll just skip that ad and go onto the next listing for “kate spade” shoes. Well, I’m male and katespade.com sounds like they probably sell women’s shoes, so I’ll skip that one as well. The next ad, however, looks a bit like a train wreck – no unique text beyond my own search query, a basically unparsable domain name, and something that looks like an attempt at Dynamic Keyword Insertion that, even if it had been done properly, would still have only inserted the words “designer shoes” one more time.

At this point, any reasonable person would simply stop looking at the ad listings (which is a real shame for ShopHousingWorks.com or Bloomingdales.com, which are further down on the page but might actually have what I’m looking for). This is what online marketers term The Pollution Effect: if some of the ads look really bad, searchers might not be willing to take the time to judge each ad on its own merits, since they might assume that ads which are placed below a bad ad are probably just as bad, if not worse.

Economists call this effect an ‘externality‘, which simply means that bad ads might impose a cost on other ads near them (in the form of a reduced click volume) by polluting the space with irrelevant or otherwise unappealing listings. On any given search the presence of one or more ads which are so bad that they cause searchers to cease examining the ads entirely (what I call “showstopper ads“) could therefore cause relevant ads to be unfairly disregarded.

“Well, so what?”, you might think. If 5% of ads are showstoppers, they should only reduce traffic by about 5% at most, right? Besides, Google has measures in place to weed out bad ads, so showstopper ads should never be a big problem. It turns out, neither of these seems to be the case.

I’ve estimated the effect of differing percentages of showstopper ads on the clickthrough rate (CTR) assuming that the ads are randomly placed, that searchers tend to examine higher-placed ads before lower-placed ads, and that encountering a showstopper ad causes the searcher to stop looking at any further ads. For each position, the ratio of the clickthrough rate at that position with some percentage of showstopper ads is compared to the CTR at that position when there are no showstopper ads.

Surprisingly, having just 5% of the ads be showstoppers likely causes a significant drop in traffic, with the ad in position #1 averaging a 5% drop, but with the ad in position #10 seeing its traffic drop by about 40%. And, the total number of clicks received by all of the ads combined drops by about 15%…all from just 5% of the listings being showstopper ads!

If 30% of the ads are showstoppers, the effects are even worse, with the ad in position #10 seeing its traffic drop by over 97% and the click traffic to all of the ads combined dropping by about 60%. Given these ratios, it’s understandable why search engines are so vigilant about weeding out irrelevant ads and those that violate style guidelines.

But recent changes to how Google serves broad-match ads in search results might make showstopper ads soon be much more common, rather than less so. I’ve noticed that Google has started carrying over ads from older searches into the ads for a new search, regardless of whether or not one is logged into a Google account at the time and also even if one has specifically turned off Google’s Web History feature and cleared the search history.

For example, I recently did a search for [buy authentic polish flag], getting ads from FlagsImporter.com, United-States-Flag.com and AmericanFlagsExpress.com in the results (among others). Immediately afterward, I did a search for [buy barometer], the results of which are shown below:

Look! There’s an AmericanFlagsExpress.com ad in there at the top of the right-hand sponsored links, and several other flag ads throughout the right-hand side. In total, this particular search shows five carryover ads (the most I’ve seen so far), representing 50% of the sponsored links on the page. As my calculations above showed, it probably doesn’t take too many low-relevance ads like these for click volume to suffer appreciably. Woe be unto you, BarometersPlus.com.

I’m certain that Google has their reasons for starting to show carryover ads, and no place more so than the internet can brilliance sometimes be indistinguishable from insanity, but frankly this practice does not seem to me to be in the users’ best interest. Performing similar searches on Bing.com doesn’t turn up carryover ads like this, so whatever’s going through Google’s head doesn’t seem to be happening elsewhere.

The lessons for marketers, I think, are threefold: First, on key terms, check to see that your ads do not rank below showstoppers or else you might be sacrificing good-quality traffic without even knowing it. Second, if Google persists in showing ads across search queries like this, know that the analysis of ad performance is going to get much more difficult, since your ad for flags might start showing in searches for barometers. (In fact, the Zappo’s leather jacket ad appeared in the search I did for “designer shoes” came just after doing a search for “leather jackets”). Unfortunately, Google’s Search Query reports might not be helpful in identifying cases where this occurs, since they include the catch-all category of ‘X other unique queries’ in the list of queries that prompted impressions. (I’ve seen reports that say ’1 other unique queries’. Why not just show me that 1 other unique query? Could it be because it was the equivalent of ‘buy barometer’ in a flag-seller’s account?) And, finally, recognize that if this practice of showing carryover ads becomes more common, then the top ad positions will become more valuable, since outranking showstopper ads might be the only way to avoid their toxic effect on clickthrough rates.  So, be prepared to pay more for placing your ads in top positions (which actually might be the reason why Google is showing carryover ads in the first place…)

Do you see carryover ads between unrelated search queries?  If so, please let me know.

Quality Score (QS) is a value reported by Google that is likely closely tied to an ad’s clickthrough rate (CTR).  I’ve described why this is so in a previous post of mine called ‘Why is Clickthrough Rate the Main Factor in Quality Score‘. In a sense then, Quality Score acts like a multiplier whereby Google treats dollars from some advertisers as worth more than dollars from other advertisers.  If your Quality Score is double mine, then I need to bid $2 for every $1 that you bid.  The actual cost-per-click (CPC) that an advertiser pays is simply the minimum amount they would need to have bid to beat the ad located below them:

So, if your AdRank (that is, your bid multiplied by your Quality Score) is 6.000 and I have a Quality Score of 2, I must bid $3.01 to beat you.  If I raise my Quality Score to 3, then I need to only bid $2.01 to beat you.  And if I manage, by testing lots of different ad copy, to find a version that gets a great clickthrough rate which results in a Quality Score of 6, then I need to only bid $1.01 to beat you.

Since each ad is only required to pay the minimum cost-per-click necessary to beat the AdRank of the ad located below it, the naïve perspective is that it is always beneficial to increase QS.  Optimization, in this view, is synonymous with maximization.

Unfortunately, it can easily be shown that it is not always in an advertiser’s best interest to have the highest Quality Score possible.  This is why it was disturbing to see a recent post by George Michie, consistently one of the most lucid voices on subjects related to pay-per-click advertising, saying the exact opposite recently at the Rimm-Kaufman Group’s blog.  Like most of Mr. Michie’s posts, this one is worth reading in its entirely, but to quote the passage most relevant to this discussion:

“…there is no complexity involved with QS strategy. You want the QS to be as high as possible always. That doesn’t vary by season, or by time of day, or by category. It doesn’t depend on stock positions, margin structures or return rates. Higher is better, and the mechanisms for making improvements are obvious.”

If only it were so simple.  But it can be made clear by example that higher Quality Scores are not always better.  Consider an advertiser who is faced with showing two different ads: one that’s generic enough to attract many clicks and another that’s specifically targeted so that it gets few clicks, but the clicks that it does get are from users who are likely to actually purchase the product.  (In advertising parlance, this is called ‘qualifying’ potential customers.)  The numbers below are fictitious, but realistic:

We can see that by showing the generic ad 1000 times, we attract 100 clicks, resulting in 15 conversions.  If each conversion brings in $100, then our revenue is $1,500.  If each click costs $5, then our profit is $1,000 (or $1.00 per impression), the clickthrough rate (CTR) for the ad is 10% and the Quality Score might be something like 10.  In contrast, the targeted ad might also be shown 1000 times, but get only 30 clicks, among whom are the same 15 people who would have clicked on the generic ad.  So, our revenue is still $1,500, but the cost (at $5 per click) is now only $150.  Thus, our profit is $1,350 (or $1.35 per impression).

Of course, because the clickthrough rate is now only 3%, the Quality Score is likely to be lower (let’s call it a value of 3, even though the Quality Score value that Google reports is probably not a direct, linear function of CTR).  But that doesn’t matter to the advertiser, because the profit is higher.  Mr. Michie said: “You want the QS to be as high as possible always”, but in fact the purpose of all economic activity is to maximize profit, not clickthrough rate nor Quality Score.  So, when performing tests of competing ad creatives, we should judge them by the differences in profit they generate per impression, irrespective of the Quality Score values that Google reports.

This article is a follow-up to ‘Optimal Bidding, Part 1. Behind the Scenes of ‘Google AdWords Bidding Tutorial’

There used to be this TV game show called “Card Sharks” where, in one portion, contestants were shown a playing card and then asked to guess whether the next card was going to be higher or lower in value.  (This wasn’t high-brow entertainment.)  The best strategy is obvious – when the card shown is low, guess that the next card will be higher, and when the card shown is high, guess that the next card will be lower.

In the new video ‘Google AdWords Bidding Tutorial‘, Google’s chief economist, Dr. Hal Varian, explains how advertisers can use Google’s Bid Simulator (GBS) to determine a bid that is “at or near the profit-maximizing level for each of your keywords.”  His method involves calculating the value per click, and then comparing that value to the GBS’s estimated incremental cost per click (ICC), as seen in this screenshot:

The specific example he gives is of an online retailer of digital cameras where each camera sells for $300 and costs the retailer $200 and where each click has a 5% chance of converting into a sale.  (In this case, since each click has a 5% chance of bringing in $100, the value per click is $5.00.)  Dr. Varian says, “Using data from Bid Simulator, or from your own experiments, we can see how many clicks we could have potentially received at different bids and how much those clicks would have cost.”  These are the first three columns in the screenshot from time 7:33 below.

Since each click brings the retailer $5 (before ad costs are considered), we can multiply the number of clicks by $5 to find the amount of money the retailer expects to make per week at each bid level (Revenue).  Subtracting the ad Cost from that amount tells us the net Profit per week.

The incremental cost per click (ICC) is simply the change in cost between any two bids divided by the change in clicks between those bids.  So, in Dr. Varian’s example, going from a bid of $3.50 to $4.00 costs an extra $97.29 (that is, $407.02 – $309.73) and brings in 21 more clicks (154 clicks – 133 clicks), for an ICC of $97.29 / 21 = $4.63.  However, each click brings the retailer $5.00, so it’s in this advertiser’s best interest to raise the bid and get more clicks at (or just above) a price of $4.63.  At time 5:52, Dr. Varian says, “Whenever your value per click is less than the incremental cost per click it will pay for you to lower your bid in order to reduce your cost.  Conversely, if your value per click is higher than your incremental cost per click, you should increase your bid.  You can see the ICC at our $4.00 bid is the closest value to our $5.00 value per click without going over.  So it’s going to bring in the highest profit.  Actually, in this example,” he says, “you probably want to bid a little bit more than $4.00.”  How much higher?  He doesn’t say.

I call this the ‘Card Sharks Approach to Bid Management’ and it might work well for people who have all the time and money in the world to ‘experiment’ with various bids, as Dr. Varian suggests, or for those who have the luxury to only know their best bid to the nearest $0.50 (or whichever other increment Google chooses to display) or who are willing to just guess some amount between the GBS’s tested values.  My question is: Since Dr. Varian shows that we have all the information we need to determine an optimal (or near-optimal) bid, why test different bids at all?  Why not just directly calculate (to the best extent possible) the single optimal bid and then just use that bid immediately, instead of nudging bids up and down until we hit some sort of observable sweet spot?

Personally, I think that showing AdWords’ users how to directly calculate a profit-maximizing bid is one of Dr. Varian’s ultimate goals and the simplified description he provides in this video is just a waypoint on that journey.  So, rather than wait for him to get to it on his own, I am going to describe for you a simple, visual means for determining your optimal bid from Google’s Bid Simulator.  Then, I’ll use some straightforward math to show you how to calculate the optimal bid (and CPA and ROI) for Dr. Varian’s example.

The screenshot from time 7:33 (Figure 2, above) contains some of the information found in Google’s Bid Simulator, but it also lacks a key component.  If you actually look at the Bid Simulator for a word in your account, you’ll notice that for high-traffic words, in addition to the columns of numbers, the dialog box also contains a graph that looks something like:

Each green point on this graph comes from a row on the spreadsheet – the point furthest to the right from the highest bid (the top row of numbers) and each next point to the left from the next row down.  (I’ve added the bid labels to each point for clarity, but they are not shown in Google’s actual Bid Simulator.)  Since it is obvious that at a bid of $0.00 the word will get 0 clicks at $0 cost, I have also added that point to the graph.

A recent blog post by Google which states that Hal Varian’s research indicates conversion rate (CR) does not vary much with position is very interesting, in part because this also implies that CRs do not change as a result of changing the bid. So, each word has a value per click (in effect, a rate at which an advertiser is willing to trade dollars for clicks) that does not depend on the bid, position, or number of clicks already obtained.  Therefore, we can draw a straight line on the ‘Cost vs. Clicks’ graph whose slope is the value per click and slide that line until it just barely touches the Bid Simulator’s estimates.  The point on the Bid Simulator’s estimates where the two lines meet (in this case, just over $4.00) is the optimal bid.

For bids lower than (that is, to the left of) this point, the advertiser should be willing to increase the bid because the slope of the ‘Cost vs. Clicks’ curve is less than the rate at which the advertiser is willing to trade dollars for clicks.  For bids higher than this level, the advertiser should be willing to forgo (too-expensive) clicks to save those dollars.  The point where the straight line crosses 0 clicks (in this case, about -$370) is the negative value of the expected Profit per week, which you can confirm in Figure 2 above.

None of this should be surprising to anyone who has read Dr. Varian’s article called ‘Position Auctions’ (International Journal of Industrial Organization, vol. 25, iss. 6, Dec 2007, p. 1163-1178) and all of my description from above is taken directly from that article.  It seems perfectly reasonable to me that Google might add this functionality to their Bid Simulator at some point.  (In fact, it surprises me that they haven’t done this already.) The advertiser could simply enter the ‘value per click’ in an input box, and the GBS could plot the line, find the optimal bid and determine the estimated profit per week in the blink of an eye.

However, if you know the relationships for ‘Clicks vs bid’ and ‘avg CPC vs bid’, it is also possible to just calculate the optimal bid directly on your own, without fiddling with the Bid Simulator and taking the Card Sharks Approach.  For Dr. Varian’s example, we can plot ‘Clicks vs bid’ and ‘avg CPC vs bid’ and find that, for this simple demonstration case, they are both basically straight lines:

That is, Clicks follows the line ‘m bid + b’, and avg CPC  follows the line ‘n bid + g’, where m, n, b and g are parameters that can be found by least-squares fitting (i.e., the ‘trendline’ feature in Microsoft Excel).  For this case, m is about 47.865, n about 0.676, b about -33.514, and g about -0.0357.  Oddly, the points corresponding to a bid of $4.50/click seem to have been adjusted upwards from a linear relationship, perhaps to make the results in Dr. Varian’s demonstration clearer.  (It’s mildly disturbing that he might have fiddled with the numbers, even for demonstration purposes, since it makes one wonder to what extent the estimates provided by the Bid Simulator itself might be manipulated.)

Nevertheless, our goal as advertisers is to maximize the amount of net profit made per week (after ad costs are considered).  Since net profit = Revenue – COGS – AdCost, our goal is simply to find the bid where d(net profit)/d(bid) = 0.

Revenue = Clicks x CR x RevPerConv

COGS = Clicks x CR x COGSPerConv

and

AdCost = Clicks x CPC

Thus:

We know from Dr. Varian’s research on conversion rates that CR is not a function of bid, and d(Clicks)/d(bid) = m, so if we say ProfitPerConv = RevPerConv – COGSPerConv, then the equation reduces to:

Multiplying the equations for Clicks and CPC together and differentiating with respect to the bid gives:

The first two terms, ProfitPerConv x CR, is simply the value per click (VPC), so:

(Note: this equation is only true for the specific ‘Clicks vs bid’ and ‘CPC vs bid’ relationships used in Dr. Varian’s example.) That is, for the specific example where Clicks and CPC are linear functions of the bid and have parameters equal to the values listed above, the optimal bid (which Dr. Varian called “a little bit more than $4.00”) is actually a little more than $4.07.  (If the Click and CPC relationships are assumed to be 2nd-order polynomial, rather than linear, the optimal bid turns out to be essentially the same, $4.08.)  So, there’s no need to use the Card Sharks Approach when you can just calculate the optimal value directly.

What is the CPA at which profitability is maximized?  If you recall, Dr. Varian calculated the ‘maximum profitable CPA’ in his video, but the ‘maximum profitable CPA’ is the CPA above which the advertiser’s profit is negative.  In other words, it is the CPA at which the expected profit is equal to zero.  When bidding, our target CPA is the ‘CPA of maximum profitability’, not the ‘maximum profitable CPA’.  We can see the difference between the two in the diagram below, which plots net profit vs. CPA.  Profit reaches a peak at a bid a little bit higher than $4.00 and declines from there until reaching 0 at a CPA of $100, when the bid is nearly $7.50 ($7.45, actually).  The ‘maximum profitable CPA’ therefore is $100, but the CPA of maximum profitability is much less.

(Notice that the position of the point corresponding to a bid of $4.50 has perhaps been moved by the possible adjustment made to the Bid Simulator’s numbers at that point.)

We can actually find the CPA of maximum profitability (that is, the target CPA) quite easily from what we already know.  CPA = Cost / Conversions, therefore:

It’s simple algebra to multiply the optimal bid, shown in an equation above, by ‘n’ and then add ‘g’.  Thus:

(Again: this equation is only true for the specific ‘Clicks vs bid’ and ‘CPC vs bid’ relationships used in Dr. Varian’s example.)  For the particular parameters that fit the sample data best, this optimal CPA is approximately $54.38.

It’s remarkable to see how easy in Dr. Varian’s example it appears to be to make a profit on AdWords. In his example, any bid in the range of $0.70 to $7.54 turns a profit.  A bid of about $4.07 yields the most, but any bid from about $3.32 to $4.82 gives an expected profit that’s within 95% of the maximum.  In other words, even though the purpose of Dr. Varian’s video was to demonstrate how to determine an optimal (or near-optimal) bid using Google’s Bid Simulator, one of the interesting lessons of the specific example he crafted is that (for this particular example only) you can bid anywhere in a $1.50-wide range surrounding the optimal bid and still basically be maximizing your profit.

Many account managers say that they would like to push down their CPAs as low as possible.  But another interesting lesson from this example is that in addition to a maximum profitable CPA ($100, where the advertiser makes no profit), there is also a minimum profitable CPA (in this case, about $8.75, corresponding to a bid of about $0.70, below which the advertiser also makes no profit).  So, account managers who are too successful in pushing down their CPAs might also be pushing down their profits, perhaps without even realizing it!

Unfortunately, determining the conversion rate, revenue per conversion, COGS per conversion, the relationships for ‘Clicks vs bid’ and ‘CPC vs bid’, and the CPA (or ROI) of maximum profitability in most real-world examples is not as simple as Dr. Varian’s example.  Therefore, The Search Agency (and the AdMax online marketing platform) are here to help you maximize the return on all of your online marketing efforts.  Please don’t hesitate to contact us if you need assistance with your online marketing efforts.

Thanks to Eric Sodomka of Brown University for examining the 2nd-order polynomial Click and CPC models.

In the film, Dr. Varian describes a 4-step process for finding an optimal bid:

1. determine your maximum profitable CPA,

2. determine your conversion rate,

3. calculate your value per click, and

4. adjust your bid so that the value per click equals the incremental cost per click.

The specific example he gives is of an online retailer of digital cameras where each camera sells for $300 and costs the retailer $200 (for a profit to the retailer of $100) and where each click has a 5% chance of converting into a sale. The basic question he addresses is: What is the retailer’s optimal bid (per click)?

Since the retailer can afford to spend up to $100 to acquire each new conversion and there’s a 5% chance any click will convert, he should be willing to spend up to $5.00 for each click, at the absolute most. At time 2:40 Dr. Varian says: “If you were to pay $5.00 for each click you would expect to just break-even on your marketing investment.” Obviously, then, having a cost-per-click (CPC) more than $5.00, which corresponds to a cost-per-acquisition (CPA) greater than $100, results in a loss to the retailer.

Since we now know that a $5.00 CPC results in no profit, Dr. Varian then begins to consider which bid does maximize profit. “While bidding at your value per click would generally lead to profitable results,” he says at time 3:20, “it may not produce the maximum possible profit for your marketing investment.”

He demonstrates this with a specific example, using sample values similar to the information that Google’s new Bid Simulator feature provides. For several reasonable bid values, we see the number of clicks expected per week, the total expected cost per week and the average CPC. Dr. Varian suggests calculating the expected revenue by taking the number of clicks and multiplying by the value per click and also calculating the expected profit by subtracting the ad cost from the revenue.

Figure 1. A screenshot from time 4:58 in 'Google AdWords Bidding Tutorial'

Doing this, we can see that a bid of $5.00 truly does not maximize the expected profit, since there is a higher expected profit if the bid is $4.00. At time 7:26, he says: “Actually, in this example, you probably want to bid a little bit more than $4.00 so you can get your ICC [incremental cost-per-click] as close as possible to the $5.00 value per click.”

The incremental cost-per-click from one bid to the next is simply the additional cost incurred divided by the additional number of clicks you receive. (Often, the ICC can be a large number, Dr. Varian explains, because raising a bid raises the CPC of all the clicks you’ll receive, not just the incremental ones.) To bid optimally, it is important to know the ICC, Dr. Varian says. “Whenever your value per click is less than the incremental cost per click, it will pay you to lower your bid in order to reduce your cost. Conversely, if your value per click is higher than your incremental cost per click, you should increase your bid.”

Even though Dr. Varian’s video is jam-packed with useful information about optimal bidding, it must be pointed out that he has also left out a great deal and made numerous simplifications and assumptions. Naturally, some simplifications are necessary for the purpose of maintaining comprehensibility (As the British writer Hector Hugh Munro once said, “A little inaccuracy sometimes saves tons of explanations”). However, some of the concepts that Dr. Varian skimmed over are so critical to determining optimal bids that they simply cannot be ignored when actually bidding on Google AdWords. I liken Dr. Varian’s remarkable new video to a power table saw – useful in skilled hands, but dangerous to unskilled ones. And it is only by knowing the limits of Dr. Varian’s description of optimal bidding, and how to make the most use of what he does describe, that search engine marketing managers can get the best performance out of their accounts at the lowest risk of causing themselves grievous financial harm.

The first limit is probably also the most obvious: to use Google’s Bid Simulator (GBS) for a given keyword requires it to be active for that keyword. Presently, the GBS only returns Click and Cost estimates for keywords that have gotten about 25 or more clicks in the past 7 days. For words that have gotten fewer clicks (which typically constitute the vast majority of the words in an account) you get an estimate only of the number of Impressions. And if a word recently was receiving Click and Cost estimates, but then fell below the minimum click traffic level, the Click and Cost estimates will be terminated, leaving you without a way to intelligently change the bid after that. In other words, Google’s Bid Simulator can be a helpful supplement to your primary means of calculating optimal bids, but you can’t rely on it to always return estimates when you need them.

In the video, Dr. Varian suggests performing tests of higher and lower bids on your own to see how Clicks and Cost change with the bid, but this too is problematic, since testing high bids is often expensive (with higher bids incurring greater total ad cost and lower bids forgoing potential conversions) and typically involves collecting data for more than 7 days (and those days shouldn’t be weekends or holidays or days when your tracking crashes etc., etc.)

The second point where careless bidders could harm themselves comes about by confusing your ‘maximum profitable CPA’ with your ‘target CPA’. Early in the video, when Dr. Varian introduces the example where the retailer sells a digital camera for $300 which has a wholesale cost of $200, he says, “Therefore a conversion for a user who buys a camera on your site generates $100 worth of revenue for you.” This $100 figure is labeled ‘max profitable CPA’. Just afterwards, at time 1:05, he says, “That $100 is your maximum profitable cost per acquisition, or ‘CPA’. You can pay up to $100 per conversion and still make a profit on the sale.”  Dr. Varian’s calculation is 100% correct – your maximum profitable CPA is your profit per conversion multiplied by your conversion rate (conversions per click).

However, please pause for a moment to ponder the difference in meaning between the phrases “maximum profitable CPA” and “CPA of maximum profitability”. The first refers to the CPA at which you just barely break even. To have even a slightly higher CPA means to lose money. The second phrase, ‘CPA of maximum profitability’ means the CPA at which you make the most profit per click, not just barely break even. This is your target CPA. Your target CPA (not your maximum profitable CPA) multiplied by the keyword’s conversion rate gives you your target cost-per-click (CPC), the amount you should attempt to pay per click.  Pay above this amount and you will spend more per click than you receive in additional profit. Pay less than this amount and you will refuse clicks whose value to you exceeds the amount you expect to receive from them.

A third area where viewers might encounter some confusion comes from the atypical way Dr. Varian describes the financial goals of the fictitious retailer in his example. Search engine marketers tend to classify their accounts as either ‘CPA-targeted’ or ‘ROI-targeted’ (return on investment) and CPA-targeted and ROI-targeted accounts, though generally very similar, are not identical. In Dr. Varian’s example, the retailer sells products online and apparently is able to track the number of items sold in each purchase, the price of the products, the cost of the goods sold (COGS) and, therefore, the profit per conversion, and so forth. Such accounts are typically classified as ‘ROI-targeted’, since their explicit objective is to maximize the total net margin (that is, profit after COGS, ad costs and all other costs have been accounted for) that they receive per day. In an abstract sense, the goal of all economic activity is to maximize the profit generated, but in cases where an account cannot track the number of items sold, value per item, etc. at a very granular level, the account manager is forced to resort to a proxy measure of profitability. With these accounts, called ‘CPA-targeted’ accounts, account managers often select a single type of event to call a conversion (as Dr. Varian lists, “the sale of a product, a new lead, a sign-up, or getting users to download or view some material on your site”) and then attempt to maximize the number of conversions received for each amount of spending.

It’s strange that Dr. Varian gives an example where the retailer has the ability to track conversions, revenue per conversion, COGS per conversion, profit and so forth, but then chooses to attempt to optimize to a target CPA, rather than to the target ROI that maximizes profit.

A fourth issue that Dr. Varian simply glosses over, most likely for the purpose of saving “tons of explanations”, is the fact that in his example he talks about a digital camera that brings in $100 in gross margin per sale, but that on Google AdWords, advertisers don’t bid on products, they bid on keywords. Dr. Varian admits this himself when he says, “In general, however, you don’t bid by CPA on Google; you bid by CPC, or ‘cost per click’.” So, the advertiser would not bid on each sale of that particular digital camera, but rather, each click on the ad associated with, say, the keyword “digital camera retailer” / exact match. A searcher might purchase the $300 camera after typing in that search query, giving the retailer a $100 gross margin (before ad cost). But the next person to search for that term might just purchase a $10 carrying case for a digital camera. Or, the searcher might be a professional photographer whose studio was recently robbed and spends $10,000 on a wide variety of equipment in one order.  Therefore, the advertiser has no way to know a priori when setting the bid whether the next click will bring in $10 in business or $10,000.  So, a profit-maximizing advertiser must estimate the expected Revenue per Conversion (RPC) and the expected Cost of Goods Sold per Conversion (COGSPC), two numbers that Dr. Varian discusses as if they are easy to calculate, but which in the real world are often known only to a certain level of accuracy.

A fifth factor with which the online advertiser must contend, but which Dr. Varian also assumes to be obvious, is the conversion rate for any given keyword is usually not known precisely.  At time 1:44, he says, “Your conversion rate is the number of conversions completed on your site, divided by the total number of ad clicks to your site.”   It is true that this quantity is your observed account-wide conversion rate.  However, advertisers bid at the keyword/matchtype level, not at the account level. Knowing the total number of conversions divided by the total number of clicks for your entire account is simply not good enough – you need this information for every individual keyword on which you intend to bid.  For very high traffic keywords (and only for very high traffic keywords) simply dividing the number of conversions by the number of clicks can give you a fairly decent estimate of the conversion rate.  But as any experienced online marketer can tell you, most keywords fall into ‘long tail’ traffic-levels, where there are simply not enough clicks and conversions per keyword to be able to pin a nice simple number like ‘5%’ to the conversion rate.

And, finally, a sixth issue which Dr. Varian glosses over is really an entire collection of related problems: keywords don’t exist in a vacuum.  He only talks about optimally picking one bid for one ad as if all the information necessary to set that bid is contained within the performance data of that keyword and that keyword only.  But optimally picking a set of bids for a set of ads requires considering account-level and campaign-level budget limits, cross-keyword attribution (that is, the fact that a non-converting click on an ad might prompt a later search on a different term which then does convert), conversion latency (which Dr. Varian addresses briefly at time 2:50 when he says that “you might want to bump this value [your assessed value per click] to reflect the fact the visitor might not convert on this particular visit but may return in the future to buy something”), cross-channel and cross-media influences, data errors, holidays, hour-of-day and day-of-week periodicity, seasonality and a gamut of other issues that would run the length of your arm.

The short point is, the topics Dr. Varian does not talk about in his new video are at least as important to optimal bidding as the ones he does, and if you take his advice at direct face value, you might get hurt.  Fortunately, The Search Agency (and AdMax, our online marketing platform) are here to maximize the return on all of your online marketing efforts, regardless of your level of expertise.

This post is continued in ‘Behind the Scenes of ‘Google AdWords Bidding Tutorial’, Part 2‘.

Clicking on the plus sign launches the VPB clip, expanding the video. If a user clicks on the ad after having already clicked on the VPB, the advertiser will not be charged for two clicks.  Advertisers pay based on a cost-per-click (CPC) bid.

VPB is simply an alternative to using Text ads for your Google AdWords search campaigns.  We have found that entertainment clients (with numerous video assets) make the best advertisers to use VPB.  At the moment, Google does not have the functionality to upload VPB clips in the AdWords UI or AdWords Editor, so advertisers have to submit their clips to Google.  The turnaround time is approximately one or two business days.  It can sometimes take longer.  The process of uploading VPB clips can be quite laborious on Google’s end.  As such, we only upload one VPB clip at a time in a handful of ad groups.  Submitting more than one VPB clip for a large number of ad groups might take more than a day or two for Google to complete.  Plan your campaigns accordingly.

If possible, use QuickTime clips for VPB.  They’re no larger than 200×150 and 10MB when zipped.  Anything larger will cause the email submission to be bounced back as undeliverable. The clip should be no longer than two minutes, but it’s recommended that the clips be much shorter.  Users tend to lose interest by the 30-second mark.

VPB has been an excellent fit for Travel Channel, enabling their users to immediately  watch previews of upcoming episodes, in addition to driving traffic to their web site.  Travel Channel is always open to testing new products, and VPB in particular has been a proven success for them.  CTR for Man v Food has gone from 2.18% (Jan 2009) to 6.34% (August 2009) since we implemented VPB.  That’s a 190% increase.

UPDATED:  Travel Channel VP, Digital Marketing Pete Dorogoff was quoted today in an AdAge article about media companies getting more into paid search.  “We buy search terms around the talent, food, locales and major tourist attractions,” Dorogoff said. “With search we’re casting a wide net and seeing what shakes out.”

But, on the AdWords Help page called ‘What is ‘Quality Score’ and how is it calculated?’, they give a whole laundry list of factors that they say help to determine QS, including: the historical clickthrough rate, your account’s history, the quality of the ad’s landing page, the relevance of the ad to the search query, and the catch-all category of “other relevance factors”.  Brad Geddes in one of his BGTheory posts says: “There are over 100 factors that can affect quality score.  However, not all will be triggered depending on the conditions involved.”  (The second sentence is a reference to the fact that there are actually multiple Quality Scores, not just one, and that they are determined in different ways and for different uses.  He provides a useful reference table.)

It’s no wonder that search marketers are often mystified to the point that some throw their hands in the air and resign themselves to thinking that Quality Score is just a fairly arbitrary measure of ‘how good Google thinks that your ad is’.  Google doesn’t go out of its way to help to clear the air much, either.  In ‘Introduction to the Google Ad Auction’ video on YouTube, Google’s chief economist Hal Varian spends the first few minutes explaining how their auction system works.  The concept is familiar to anyone who has been in the SEM industry long enough to know the difference between ‘CTR’ and ‘CPC’ – For each ad, Google multiplies that advertiser’s bid by the ad’s Quality Score to determine a value called the ‘AdRank’:

Each ad is then listed on the results page in order from highest AdRank to lowest. In this sense, AdRank is sort of a ‘weighted bid’, with the Quality Score acting to make dollars from some advertisers worth more to Google than other dollars. In a later part of the video, Dr. Varian shows a pie chart that breaks down the Quality Score into its constituent parts.  About 60% of the pie appears to be the ad’s CTR, about 30% its ‘relevance’, and the remaining 10% attributed to the ‘landing page quality’.  Any reasonable person watching the video should be impelled to blurt out: “Wait a minute, Hal!  What happened to those 97 other factors Brad Geddes mentioned?  You only show 3!”

Perhaps more important than what Dr. Varian does mention are the things he does not talk about, like the existence of multiple Quality Scores.  Discussing what constitutes a good-quality landing page, Hal mentions that the page should have “original content”, a “quick load time”, and be “easily navigable”.  A naive interpretation of this would be to think that Google loads your landing page, counts how many seconds it takes to receive the HTML file, and then tabulates this time as part of your ad’s Quality Score, like some sort of text-based beauty contest, with the participants scoring some points for poise, a few more for looks, a couple for talent and so forth.  In fact, most of those other 97 factors are likely what Craig Danuloff of ClickEquations calls ‘bozo filters,’ that is, pass/fail tests that determine whether or not your ad is allowed to participate in a given auction, but not used after that.  As long as the HTML arrives in less than some fairly generous cutoff value (say, 30 seconds), then your ad gets a passing grade on that factor.  Get passing grades on all those ~97 basic little tests and your ad is permitted to participate in the auction.  So, even though the CTR wedge looks like its about 60% of the Quality Score pie, its role within the ad auction is probably much greater than that.  (In fact, in his 2007 article “Position Auctions” analyzing the properties of in-auction metrics, Dr. Varian concerns himself with only one component of Quality Score: the clickthrough rate.  Suddenly, our list of ‘over 100′ factors is reduced to just one!)

Another thing that Dr. Varian doesn’t mention is how Google stores the Quality Score for a given ad.  On the whiteboard in his video, he shows the Quality Score as being numbers like ’1′, ’3′, and ’6′, but in fact, Quality Score is actually calculated and stored as a number between 0 and 1.  (This sounds minor, but is actually very important.)  Hal doesn’t mention that the value Google calculates for the Quality Score (the ‘actual Quality Score/s’) and the value that they report to you (the ‘reported Quality Score‘) are not the same thing.  In our experience, most reported values of the Quality Score in most accounts are 7, with a few ads having 8, 9 or 10 and a few being 4, 5 or 6.  This means that, if you want to maximize a given ad’s Quality Score, you will have a more-difficult time doing so by looking at the reported QS than just by looking at the ad’s CTR directly.  It takes an enormous change in CTR to see the reported QS go from 6 to 7, or from 7 to 8, but even a tiny increase in CTR will help improve the actual 0-to-1 number stored in Google’s system.  To maximize your Quality Score, then, just focus on maximizing your ad’s CTR.

He also doesn’t give us a list of things that (almost) certainly do not affect an ad’s Quality Score, like: your bid, your budget, and your keyword’s match types. (Quality Score is only calculated from exact-match queries for a given keyword. Thus, all the matchtypes for a given keyword have the same QS.)  Some people raise bids, especially on new terms, thinking that Google might reward them with a higher QS, but examining the equation for AdRank above shows why this is counterproductive to Google: for any value of QS that is based on the bid, there is some other value (not based on the bid) that yields the same AdRank.  Rather than include the bid in the calculation of the Quality Score, the search engine can just leave it to be a separate factor in the equation and get the same result.  (Another reason that some account managers raise bids on new terms is to help increase their CTR, thinking that the CTR’s from higher positions will lead to a higher Quality Score, but that brings up another point Dr. Varian was silent about.)

One last thing to mention that Dr. Varian neglects is the fact that observations of your CTR are actually adjusted to account for their position.  In other words, it’s not your absolute clickthrough rate that matters – it’s your CTR relative to other ads for that keyword.  Each time your ad is shown, Google records the position and whether or not the ad got a click.  This means that they not only know your ad’s observed CTR, but also the average CTR of all ads in all positions.  So they can estimate how an ad would have done if it had been shown in, say, position 1 even it has never (or not recently) been in that position.  Thus, every ad can be compared in a fair way to all of the other ads in the auction.

Google didn’t always take clickthrough rates into account when selling ad space.   In the excellent article ‘Secret of Googlenomics‘ in the May 2009 issue of Wired magazine, Steven Levy reminds us that, way back around the turn of the century, Google used to sell ads by the impression. “Advertisers were … billed by the number of user views, or impressions, regardless of whether anyone clicked on the ad,” he wrote.  The problem Google faced was: how to price those impressions.  Too low meant giving away revenue they could have gotten for themselves.  Too high meant lots of disappointed and angry advertisers.  And even the advertisers themselves didn’t know what the impressions were worth.  So, Google decided instead to sell their ad slots by a PPC auction system.  Perhaps the most-obvious way to do this is to just rank ads by their bid (that is, AdRank = bid) and leave the advertisers to jockey for position amongst themselves.  But the problem with this ranking method is that it encourages bad ads (that is, those with very low position-adjusted CTRs) to get bid into high position.  Since advertisers only pay for clicks, not impressions, those who have low CTRs can afford to drive legitimate advertisers out of the top positions.

For Google’s finance department, the only equation important to Google is:

So, there are only two ways for Google to make more money: Get more people to search more often, and make more money from each search.  Searchers were already taking care of the first part on their own.  All Google needed to worry about was maximizing revenue per impression by placing the ad with the highest ‘expected revenue per impression’ at the top and all others in order below it.  The bid is in ‘dollars per click’, so, if there is some factor that gets multiplied by the bid which has the dimensions of ‘clicks per impression’, then the AdRank has the dimensions of ‘dollars per impression’:

A more-general solution, rather than make the quality score be equal to the clickthrough rate, is to make the quality score be equal to CTR raised to some power (let’s call it ‘X’, though in academic literature it’s often called the ‘squashing parameter‘):

When X equals 0, then the AdRank equals the bid and the auction reverts back to the ‘rank by bid’ scheme. When X is 1, the auction becomes a ‘rank by revenue’ system and each ad gets sorted from highest expected revenue (to Google) per impression to lowest expected revenue per impression.

When X is very large (say, 3), then CTR becomes far more important than the bid and the system is then effectively ‘rank by relevance’ – Search engine users indicate which ads are the most relevant by clicking on them and those that get the most clicks then get the highest position. The bid in such a system plays no important role in ranking ads. (Since bids clearly do play an important role in PPC advertising, we can be certain that Google doesn’t use a value of X as high as this.)

So: Why is CTR the primary component of Quality Score? In short, because it has to be.  The fact remains that Google is in the business of selling impressions, not clicks, regardless of what rules and changes they make behind-the-scenes in order to do that as best as they can.  They can toss in some pass/fail filters to keep the Viagra-peddlers out of the top slots, and they can adjust the X value up or down (across-the-board, or, by keyword) to adjust the amount they listen to the bids relative to the amount they listen to the (searcher-generated) CTRs, but they can’t change the financial physics of selling ad space efficiently: Ads must be ranked by the bid times some quantity that depends largely on the CTR or else the auctioneer doesn’t maximize its own long-term revenue.  (Lahaie and Pennock, in their article “Revenue Analysis of a Family of Ranking Rules for Keyword Auctions”, found that even the adjustments to X can’t be made without consequence for Google: the optimal value at any time depends on the correlation between the advertisers’ value-per-click and their CTRs, not on any major factor that Google controls.)  Google has not chosen the formula for Quality Score arbitrarily; in fact, their hands are basically tied by the mathematics.  Why is CTR the primary component of Quality Score? Because some function of CTR is all that the QS needs to be.  There are three participants in the AdWords auction: the advertisers, the searchers, and the search engine.  Only a system that gives each a voice (the bid, the CTR, and the squashing parameter, respectively) will satisfy them all in the long run.

The lesson for the search marketer is simple: Regardless of what Google does to X (or any other factor you can’t see or change anyhow), you only need to (1) make certain you pass the bozo filters and, when you do, (2) optimize your ads’ clickthrough rates relative to your competitors.  No other question about Quality Score seems to matter very much in comparison.

There are few restrictions placed on the form of the f-function, but in general, it should decline as the position drops.  (Average position can be a perverse metric, so for this post I have only chosen examples where this effect is minimal.)

Both in SEM industry practice and in academic investigations, two general theories have been proposed for the form of the f-function: the ‘separability’ approach and the ‘cascade’ approach.

The hypothesis behind the separability model is that there are two or more independent factors that govern the probability that an individual web surfer will click on a given ad.  To get clicked, the ad must first be noticed, and eye-tracking studies such as those by Enquiro have shown that web surfers tend to look more at the top and left-side of a webpage.  Therefore it is reasonable to assume that ads in those positions are more likely to be noticed than ads in lower positions.  Additionally, under the separability hypothesis, there exists another position-dependent probabilistic factor: that once an ad has already been noticed, the chance it will be clicked depends on its position in the ad listings.  The rationale is that users might place greater trust in an ad that has a higher position over an essentially identical one in a lower position.  So, the combination of being placed higher not only makes an ad more likely to get noticed, but, when it is noticed, it is also given a greater level of trust than ads in lower positions.

One way of expressing the separability relationship is in a form like:

where pos_max represents the position below which each of these components is essentially zero and is approximately equal to the number of ads that appear on a given SERP (i.e., about 6-10).  The brackets indicate selecting the minimum of the ratio contain in them and the number 1.  The value of F is selectable and depends on the number of independent factors and their linearity.  (In practice, the value of F is often seen to be about 3.)

The other predominant hypothesis concerning the dependence of CTR on position is the cascade model.  This relationship assumes that users sequentially examine ads from the highest position to the lowest, choosing at each step whether to click on the ad or not before proceeding to the next ad.  (Some versions of this model also assume that the user has a chance at each ad of dropping out of the process entirely.)

Mathematically, this hypothesis can be expressed in several forms, one of which is:

where Q is some number greater than 1.  When Q is 2, this equation is identical to Zipf’s Law, though in continuous form.  Examination of real-world click data indicates that Q varies by keyword, but is typically in the range of 1.2-1.9, with values around 1.4 being seen commonly. (In ‘A Formal Analysis of Search Auctions Including Predictions on Click Fraud and Bidding Tactics’, Kitts et al. propose an exponential form of this relationship. This is effectively a cascade model, not a third type of hypothesis for the dependence of CTR on average position.)

Though the two models appear radically different, they are actually very similar in their general behavior.  Under both hypotheses, CTR declines monotonically with position (that is, it never experiences an increase as position drops).  Neither accounts for any discontinuity that might be present in the CTR when going from an ad in the top promoted positions to the right-hand rail.  Also, both models have adjustable parameters that give enormous flexibility in ability to match performance data, but at the expense of having to perform a fitting process.

Where these two models differ most, for the range of realistic parameter values, is in their estimation of the clickthrough rate at positions lower than about position 4.  Unlike the separability model, which assumes that the CTR reaches zero at pos_max, in the cascade model the CTR never reaches zero.

Comparisons to experimental data might help determine which of these categories of models is more realistic.  For organic listings (not paid search ads), Craswell et al. (in An Experimental Comparison of Click Position-Bias Models) found the cascade model to be more appropriate.  For paid search ads, the Bid Simulator, a new feature that Google has developed for AdWords, is proving to be enormously useful.

For those who are unfamiliar with the Google Bid Simulator (GBS), for each word that gets sufficient traffic, the GBS provides estimates, for 4-7 various possible bids, of the cost, number of impressions and number of clicks each word could have gotten based on the actual data from the past 7 days.  In the previous AdWords interface, the GBS also provided estimates of the average position at various bid levels, so, by dividing the estimated clicks and estimated impressions column and comparing to estimated average position, we can get the GBS’s estimate of the clickthrough rate (CTR) vs. the average position.

One thing that’s important to realize about how the GBS works is that the estimates it provides are ‘model free’.  That is, the numbers aren’t really Google’s estimates, because Google engineers haven’t told the Bid Simulator their beliefs about the relationship between say, position and impressions or position and clicks.  Instead, the GBS just gathers data from auctions that have already occurred and reports a condensed version of that data to you.  So, when we study the estimates from the Bid Simulator, we are actually studying the behavior of the participants in the auction (the advertisers and the search engine users), not Google’s guesses (for better or worse) about those behaviors.

In Figure 2, I’ve shown the Bid Simulator’s estimates of CTR vs. average position for two keywords in the same account from early August 2009, the actual performance for the 7-day period which the simulator examined (plus a couple of days before that period and a couple of days after), and the best-fit separability and cascade model (the equations for which were provided above).

The top figure shows a broad-match keyword related to finding a company that provides Internet service.  We can see that both the separability and cascade models give very similar estimates for the CTR down to an average position of about 4.  The separability model more-closely approximates both the actual performance for this word and also the Google Bid Simulator’s estimates of the CTR at lower average positions.  The cascade model can be adjusted to better fit the actual data and GBS estimates at low positions, but only by making the closeness of the fit to the GBS estimates at positions 1-5 much worse.

The lower graph is for an exact-match keyword which is also related providing Internet service, from the same account (but different campaigns, adgroups and matchtypes as the broad-match term just described).  In this case, we can see that the best-fit cascade model seems to better describe the CTR.  (The separability model can be made to fit somewhat better at low position, but as with the cascade model previously, doing so make the fit at high position much worse.)

Over the past weeks I’ve looked at simulations for hundreds of high-traffic words in accounts from a wide variety of industries and can find that in some cases the separability model works better, in some the cascade model seems to work better and in some neither seem to work well.  Of course, when the average positions are all above about 4.0, both approaches seem to be reasonable, provided that their parameters are chosen well.  Below position 4.0, which model works better seems to depend on the keyword, but Google’s Bid Simulator is proving to extremely useful in helping to differentiate between them.

So there’s definitely a curiosity factor with Bing.  And my SEO colleagues have already given their review of the new interface and how the organic search results compare to Google.  But what will Microsoft’s huge investment in Bing mean for advertisers?

Virtually all of The Search Agency‘s SEM clients already include Microsoft in their paid search portfolio.  So we conducted an analysis of a cross-section of accounts, comparing their performance for the last three weeks of Live Search to their first three weeks with Bing.

The initial results have certainly been positive both for both Microsoft and our advertisers:

Click Through Rate (CTR) up 15%

Conversions up 6%

Conversion rate up 18%

Cost per Acquisition (CPA) down 3%

Although Bing’s search volume has increased on the heels of their aggressive advertising campaign, Microsoft has been more selective on which ads they serve on each search results page, often times electing not to serve any ads at all.  As a result, we saw a 22% drop in total impressions.  But Bing has significantly increased the relevancy of those impressions, yielding double digit growth in CTR and conversion rate.

We will be monitoring this data over the coming weeks to see if Bing can maintain this growth in search volume and conversion rates.  It’s entirely possible that this improved performance could represent the “curiosity factor” — explorers clicking on all sorts of listings to see how the new engine performs.  Either way, these early results do suggest some great opportunities for advertisers to obtain higher-converting search traffic with Bing.

What type of results are you seeing from your accounts?  How have your account management strategies changed since the launch of Bing?