The Search Agency  has historically employed a number of testing strategies to identify the best-performing marketing messages for our clients. However, as anybody familiar with SEM can attest, the number of variables involved in creative testing can make the process extremely challenging. Each group of ads is shown in rotation over varying sets of keywords that constantly change in bid and position, making it difficult for account managers to implement the sort of rigorous, controlled tests that produce real, actionable results for their clients.
For these reasons, Analysts and Editors in the SEM business unit have spent much of the last year drawing on their collective experience and expertise to design a new, proprietary Creative Testing Tool.
When designing the tool, TSA made the following assumptions:
- A text ad that performs well over one set of terms may not show the same performance over a second set of terms. Therefore, creative testing results should not be generalized (i.e., adding a good performing ad from one ad group to the whole account). Instead, every single ad group within an account that is rotating more than one ad should be considered a test, and discreet results should be acted upon for each individual ad group.
- CTR is not the best measure of an advertisement’s performance, even though search engine ‘Auto-Optimization’ settings will begin to show the ad with the best CTR to users more often. TSA believes that while CTR should be weighed, it is conversion rate that best speaks to an ad’s success. For example, an ad using the text “Free Beer” may generate a very good CTR, but won’t necessarily motivate users to buy something once they’re on the landing page. Therefore, TSA began preparing for use of the new tool by disabling Auto-Optimization settings on their campaigns, allowing ads to be rotated evenly.
TSA began testing the new Creative Testing Tool with clients at the beginning of the year with extremely positive results. Version 1.0 performs the following functions:
- Reviews data on each active ad group in the account, looking at all user behavior from the last time a creative change was made in the ad group to the present.
- Based on this data, the tool compares the data on the best performing ad in the group vs. the other ads in the group, using common statistical methods to identify significant differences in conversion rate, CTR and CPA.
- Based on this comparative analysis, the tool makes recommendations on which ads to pause, which to leave active, and where new creatives should be added.
- If all of the recommendations are implemented, the tool predicts the effect on the account (“client can expect x additional conversions a day based on this action”).
Results of an 82-day test for one TSA client showed that weekly use of the Creative Testing Tool resulted in a 30% boost in conversion rate for the ad groups in which pausing decisions were made. Version 2.0 of the Creative Testing Tool is currently in development, and will feature more sophisticated analytics as well as a simplified user interface.