Cost per Acquisition (CPA) is a Funny Beast

Posted on Monday, July 6th, 2009 by Print This Post Print This Post

Categories - SEM

If you manage bids on search engine ad auctions, you probably spend a lot of time puzzling over spreadsheets looking for opportunities to improve your account's Cost per Acquisition (CPA).  But CPA is a funny beast and if you don't know how funny it can act (even in the best of times), you might be wasting some of your effort.

When managing PPC search auction bids on a weekly basis, it's fairly common to look at each keyword's performance data for a recent time period (a week, or longer) and compare it to the performance the previous time period.  The idea is to make a fair comparison of the keyword's recent performance to the prior performance so that you can make good bid management decisions.

If you see that a keyword's CPA is well above its target, you might have the tendency to reduce the bid.  Too low, you might increase it.  Near the target, you might leave that bid alone.  I call this 'Goldilocks Bid Management' and while there are many variations of that folk tale, remember that in the version by Roald Dahl (author of Charlie and the Chocolate Factory), Goldilocks gets eaten in the end.  (Even in the versions suitable for younger children, Goldilocks still winds up running away screaming, which is probably what you've been inclined to do after looking at endless spreadsheets for too long.)

The difficulty in getting optimal performance with this style of bid management is not with you, dear reader, but rather with the nature of that beast called CPA.  Perhaps I can prove it to you by example.  Rather than bore you with actual performance data, I can just as easily bore you with totally made-up performance data.  So, let's consider the simplest possible case, a fictitious keyword that acts like a metronome.  Every day, our keyword gets 10 clicks for a $1 Cost per Click (CPC).  Exactly every 50 clicks (that is, every 5 days) like clockwork it gets 1 conversion.  $50 in spend every 5 days.  10 clicks per day.  $1 per click.  1 conversion for every $50.  Here's a very simple question for you:  What is this keyword's CPA?

Every bid manager knows that's there's no such thing as keyword that behaves this nicely.  Most are much nastier creatures.  But even so, I want to know this keyword's CPA, so, like any good analyst, I'll make a spreadsheet.  Let's say that I do my bid management on Mondays.  This word got a conversion the previous Monday and 1 conversion every 5 days thereafter.  (So therefore, it also got a conversion 2 days ago, on Saturday, and will get one 5 days after that - that is, this coming Thursday.)  Here's a sample of my data:

Metronome Spreadsheet

Today is Monday the 1st.  Last week (Monday to Sunday), this word spent $70 and got 2 conversions (one last Monday and one Saturday).  So, it has a 7-day CPA of $35.  On Tuesday the 2nd, the word spent $70 the previous 7 days (as it does every single 7-day period), but got only 1 conversion in the past 7 days (Saturday's conversion).  So, the 7-day CPA jumps to $70.  It stays there until Friday, when there are again 2 conversions in the past 7 days and it drops back down to $35.

To answer my very simple question, "What is this keyword's CPA?", we all know is that it's actually $50, but the CPA that you see depends on when you measure it, and for which duration.

CPA sawtooth

The actual CPA is $50.  It must be, because there's $50 in spend for every 1 conversion.  But look at the graph - our average CPA is never $50.  Some days we see $70 and others we see $35.  (For this particular example, a repeating pattern of three days of $70, followed by two of $35.)  I call this behavior 'The CPA Sawtooth', and those people who don't know about the tendency of CPA data to act like this can get hurt.

Unfortunately, if you try to avoid this problem by looking only at daily data, you won't find refuge.  On the days in which a conversion occurs, the CPA those days is $10.  But on days when there are no conversions, your spreadsheet won't even display a number for the CPA that day.  (This isn't a bug in Excel, it's a bug in mathematics.  If I spent $10 today, but go no conversions, then my CPA wasn't $10, it's actually not a mathematically defined number.)  If you filter out all that days where conversions are zero from your spreadsheet and only look at days where a conversion occurred (to try to get a feel for that keyword's CPA on any given day, it will look like the CPA is only $10 (even though it's actually $50!).

If I'm the type of manager who compares keyword-level performance data week over week, then on the 1st I'll see a 7-day CPA of $35.  A week later, on the 8th, it will be $70.  On the 15th, back down to $35.  On the 22nd, back up to $70.  A simple alternating pattern, right?  On the 29th, will it go back down to $35?  No!  CPA is more feral than that.  On Monday the 29th, the 7-day CPA stays at $70.  It only goes back down to $35 the following week.

But, keep in mind, the underlying CPA stayed constant at $50 the entire time.  By adjusting your bids based on the Goldilocks approach, you might wind up increasing your bids some weeks and decreasing them other weeks for absolutely no underlying change in performance.

Of course, no real keyword is this well-behaved.  If instead of having 1 conversion happen every 5 days we instead say that on any given day, there is a 20% chance a conversion will happen, then on average there will still be 1 conversion every 5 days (for an average CPA of $50).  But this time, a graph of the 7-day CPA won't be so predictable.  (This isn't the method I personally would use to generate realistic data, but it's good enough for the purposes of this demonstration.)

Obviously, the results depend on the specific sequence of converting days and non-converting days.  If a conversion happens on day 1, then there still a 20% chance that one will happen on day 2, and a 20% chance after that that a conversion will happen on day 3, guaranteeing at least 3 conversions this week.  So, we could go several days in a row with conversions, rather than 1 conversion every 5 days.  (Conversely, since there's an 80% chance of not converting on any given day, we could go many days without a conversion.  If we go at least 7 days without one, then the 7-day CPA that week will actually be an undefined number.)

But, in general, the results will look as crazy as the ones shown in the graph below.  Some days, the 7-day CPA could be well below the $50 average.  Some days, it might spike above.  In some periods, it will change quickly.  In others, it will stay steady for days at a time.  In other times, it might not be possible to calculate the CPA at all!

Random sawtooth

And this is for a word that still, on average, generates 1 conversion every 5 days.  Throw in some variation in click traffic, changes in the CPC and maybe a data error or two, and the CPA will be even more variable.

The important lesson here is that the CPA you see on your screen depends in part on how you choose the timeframe and when you do you your analysis.  If you take the numbers at face value, you're liable to get bitten.

Tags | , , ,

Leave a Reply

Follow Us on Twitter

Authors