Big news in the world of search engine optimization this week, as Yahoo! announced their plans to shut down their Paid Inclusion program at the end of the year. The removal comes has part of Yahoo’s new partnership with Bing, which will switch Yahoo’s search platform to Bing’s algorithm. Yahoo’s Paid Inclusion program has been a controversial topic in the search industry since its advent in 2001. To read more about the history of Yahoo’s paid inclusion, check out Matt Kain’s ‘Eulogy for Yahoo’s SSP’, and for a discussion about how this change will effect SEO optimization, check out Otto’s blog post ‘ The Death Knell for Yahoo Paid Inclusion.’

Levinson Leaves Google Board

Google announced early this week that Arthur Levinson has left his position on the company’s board of directors, effective immediately. Levinson leaves his post of five years in the middle of a federal antitrust investigation into Google and Apple’s boards. The investigation has been ongoing since May 2009. Since then Google CEO Eric Schmidt has resigned his seat on the board of directors of Apple. Although no official explanation was given, some speculate that his resignation reflects growing tension between Apple and Google over competition in the mobile market. To read more about the topic, check out Cecilia Kang’s article in The Washington Post.

Shazam Attracts Kleiner Perkins

Shazam announced this week the popular mobile music discovery engine had reached the 50 million user milestone. This news was coupled with an announcement that Shazam will receive funding from Kleiner Perkins iFund, the same venture capital firm that backed Google and Amazon.

Digg’s Ad Commenting to Increase Revenue

Digg executives tripled their revenue expectations for the year based on the growing success of their ad commenting platform. The advertising platform was first introduced back in June 2009. It enables users to comment on advertisements the same way they interact with posted articles. To read more about why this platform has proven to be so popular and profitable, check out  Andrew LaVallee’s article in the Wall Street Journal.

Google’s Profits in Third Quarter Point to Growth in Search Ads

As economists all over the world argue about the future of America’s economy, Google surprised everyone this quarter with a 27% jump in profit. It is still unclear whether this improvement reflects overall economic progress or a rebound for the online advertising industry. To read more about how Google’s announcement reflects the economy at large, check out David Sarno’s article and to find out more about how other search engines are fairing, check out Miguel Helft’s article in the NYTimes.

Bits and Bolts

• Google Books saga got more interesting this week, when Barnes & Noble announced their plans to team up with Google Android to develop an eReader.
• Google removed PageRank data from its webmaster tools this week, making it a very dramatic week for SEO.
• AT&T and Google continue to squabble over Google Voice.
• In an interview with Steve Lohr, Internet Godfather Tim Berners-Lee confesses his one Internet regret.
• Urbanspoon introduced a new point-and-shoot restaurant finder iPhone application.
• Taiwan’s Acer became the first non-American computer company to rank as the #2 seller of personal computers. 
• Spigit is raising capital to begin offeringworkspace social network services to the small and medium businesses.

- Yahoo! SSP (with apologies to The Animals)

Although not publicly confirmed by Yahoo! at the time of writing, it seems that the Search Submit Pro program will be terminated at the end of this year. Often referred to as Paid Inclusion, this powerful, effective and highly profitable program has been much maligned historically by people who did not really understand it, or who were not able to achieve reseller status to benefit from it.

I have had a long personal history with the program, originally with Decide Interactive, one of the first resellers of the program and the originator of Yahoo!’s less-well-known Trusted Feed Generator (TFG) program. Through Decide Interactive’s acquisition by 24/7 Real Media and subsequently by WPP, I have seen and been actively involved in the growth and evolution of SSP for 8 years now across hundreds of advertisers and hundreds of millions in spend (I would estimate that my past companies have managed about 25% of Yahoo’s SSP revenues, give or take, depending on the year). So yes I am biased through commerce, but I also know the product very well. If SSP must RIP, then please indulge me in this eulogy.

HISTORY OF SSP

Back in 2001, portals and indexes were separate. Inktomi, Fast and Altavista provided the indexes for Yahoo, MSN etc. on limited term licenses. Google was rolling its own, but didn’t dominate like it does now. It was a much more even 3-horse race back then. The value was seen in owning the users, not in the results, and the users were less sophisticated. Far fewer people knew how to customize their start page or search engine, toolbars were far less common, and companies like LookSmart were still creating manual directories. Yahoo! had a good grip on the internet user, and would license the best solution on the market each time it evaluated vendors, so all the innovation was with the indexes.

Crawl frequency was monthly at best, yet database driven content (classifieds, jobs, retail inventory etc) changed more frequently. If an engine indexed dynamic content, it ran the risk that that content would be removed before the index was refreshed, leaving a 404 error for users. So, as a consequence the indexes didn’t crawl pages with a “?” in a URL, meaning that all of the most timely, and often most relevant content was absent from the indexes. We complain about the slow advent of true realtime search – imagine a world where you couldn’t get any news articles, product level pages, or other dynamic content in the search results at all! It was not that long ago. The engine wars consisted of “Pages Indexed” battles, publicized and promoted as a barometer for comprehensiveness.

So obviously this provided an opportunity for ingenuity – retailers, publishers and classified sites wanted their content indexed, the index owners wanted this content to inflate page count and provide differentiation, and there was a business model waiting to be invented. We at Decide Interactive wrote a white paper about the value of accepting feeds from trusted providers in a proscribed format, and the resultant incremental revenue opportunity for charging on a per URL or CPC basis. Directories were already charging for URL inclusion so this wasn’t a stretch. I wouldn’t be surprised if other companies proposed similar solutions at the same time – it was an obvious case of demand looking for supply.

Inktomi was the first to adopt a “Trusted Feed” or “Paid Inclusion” program, followed quickly by Altavista and Fast. AskJeeves even had one for a while. Then the consolidation began…

In October 2002, Yahoo! bought Inktomi (interestingly it was still licensing Sponsored Links from Google at this time!) In 2003 Overture bought Fast and Altavista, and then Yahoo! bought Overture. Each of these indexes had different formats, so different XML schemas were required, presenting an opportunity for feed technologies to build 1 in many out content transformers. Yahoo! had 3 formats and distracted staff from 5 rapidly merged companies, so the reseller and outsourced market filled the gap. Decide Interactive, Position Technologies, Rawhide, Newgate, and others built tools to manage the manipulation and control of client data, and “Paid” to have it “Included” in the index.

CONTROVERSY

Depending on which side of the fence you sat on, all of this was either marvelously effective, or a travesty against the purity of SEO. Not coincidentally, that fence tended to separate the resellers who could participate and benefit from the program, and the SEO’s who could not. The haves loved the control, the trackability, the ability to test different descriptions, the reporting data, the fact that it had better ROI than any other channel. The have-nots complained that the results were now full of ads. The smartest of the lot bridged both worlds and looked at SSP not simply as a PPC distribution option, but potentially as an SEO tool.

Contrary to popular misunderstanding, you never actually paid for rank with Paid Inclusion – you literally were paying for Inclusion, on a category rate-card basis, and for the opportunity to control the content which was included. Of course this meant that you could programmatically ensure that your included content was extremely well optimized, so it tended to rank well. In a way it was like a controlled and sanctioned cloaking program – the resellers were trusted to place highly optimized versions of pages into the index. A fluctuating but small (as low as 5 and as high as 30) community of resellers were responsible for controlling this portion of Yahoo’s index. Quality went up and down, often inverse to the reseller count, and Yahoo! developed very active programs for assessment and certification of reseller technologies, processes, and staff training.

EVOLUTION OF FEATURES

The program’s features evolved over the years as well – initially a program for “Product Level Pages” (retail SKUs, travel origin/destination pairs etc), Yahoo! added feed formats for “Category Level Pages” and “Top Level Pages”, allowing control over the presentation of important traffic driving pages on your site. To add value to paying for clicks on your brand terms (which many argued you would get anyway through the most basic of SEO), Yahoo! introduced Quick Links allowing testing and tracking of different traffic paths directly to deep content. This feature was unique for a while, but is not a common search experience through Google’s Site Links. Value was also added by the development of Yahoo’s “Silk” reports, providing audience geographic and demographic data, as well as impression data, for traffic through SSP.

BINGHOO AND THE END OF SSP

When the Binghoo partnership was announced this year, there was a lot of speculation that this might be the end of SSP, and that turned out to be right. I was kind of hoping that Ms. Bartz had her eye on a return to the glory days of the portal – that Yahoo! owned the audience so it didn’t need to own the content. Instead it could license the index and use tools like Search Monkey to layer in customization, personalization and differentiation. This would be what made a user choose Yahoo! instead of MSN even if both were powered by Bing – blended indexes, insertion of proprietary content, query refinement etc. Perhaps some of that is still in the plan, but it looks like we’re all going to need another way to get in front of those queries – two engine SEO is back on the table.

For me and others who have been involved in this little sub-community for years, this certainly is the end of an era. Paul McCarney, Keith Lambert, Paul Ford, Gour Lentell, David Turner, Jim Staub, Jared Luskin, Dan Boberg, Chris Bolte, John Galatea, Jason Lehmbeck, John Zimmerman, Nick Sheth, Chad Hinds, Kristen Contieri and many others – it was fun while it lasted!

It looks like Microsoft won the battle with Yahoo! over the Paid Inclusion of results.  Yahoo is currently letting partners know that their Paid Inclusion program (aka Search Submit Pro or SSP) is coming to an end.  Danny Sullivan has received confirmation from a Yahoo! representative that the program will cease operation at the end of Q4.

As discussed in my article from July, BingHoo is Coming – Is the Bell Tolling for SSP?, the inclusion of paid results in the Bing results wouldn’t have been a good move.  Yahoo results have lost the trust of users and their market share went along with it.  Microsoft has been working extremely hard to match and try to exceed the quality of Google results.  The last thing that Microsoft wants is for the quality and credibility of Bing results to be damaged or diminished due to the inclusion of Paid results.

The remaining question was basically a fiscal one.  Projected revenue from market share stabilization (and hopefully increase) versus revenue generated by the SSP program balanced against the technology and development cost to try to blend SSP listings with the Bing results.  It looks like the decision has been made.

Paid inclusion had become more of a crutch for Yahoo’s ailing algorithm.   Here is an example of how extreme it got.  At one point, 9 out of the Top 10 listings for ‘refinance’ were Paid Inclusion results.  The only reason why 10 out of the Top 10 weren’t Paid Inclusion was because finance.yahoo.com was number one.  So, that meant that it was impossible to rank Top 10 organically in Yahoo for ‘refinance’.  Out of the Top 100 for ‘refinance’, 85% of the Top 100 were SSP results.  For the term ‘home mortgage’, it was not as bad for the Top 10.  Eight of the Top 10 were Paid Inclusion.  However, those two organic results were the only 2 in the Top 100.  98% of the Top 100 results for ‘home mortgage’ were SSP listings.  Obviously the SSP penetration varies by keyword, time of day, date proximity to the end of a fiscal reporting quarter, etc…  These are just some extreme examples.

With the Paid Inclusion SSP program discontinued, dual engine search engine optimization will have a renewed focus.  Optimizing for Bing means optimizing for Yahoo as well.  For the companies that have been relying on Search Submit Pro for their traffic listings in Yahoo, they better get their SEO efforts in high gear.  If they are not ranking in Bing, they won’t be ranking in Yahoo and their traffic will dry up at the end of Q4.

SSP – RIP