- The Search Agents - http://www.thesearchagents.com -

This is Not the Year of Big Data

big data 2 [1]

At the start of every year there are inevitable announcements that this is the year of X, Y, or Z. For PPC, it’s been the year of mobile for several years. This year a number of online marketers proclaimed this year as “The Year of Big Data”.

They’re wrong and here’s why:

What is Big Data?

As much as people try to sell it, ‘Big Data’ isn’t actually a thing. There are as many definitions of ‘Big Data’ as there are companies dealing with it. It really just means lots of data. In fact, it’s so much data that most people aren’t able to process it using standard tools.

Today’s Big Data is tomorrow’s data; the definition is constantly evolving. 100 years ago gathering and analysing 1 million rows of data would have been difficult, especially if you were trying to gather timely insight. 100 years later 1 million rows has moved from ‘Big Data’ to just ‘data’.

The other definition of ‘Big Data’ refers to the technologies that it’s built on, including Hadoop and other Map-Reduce systems. The problem here is we’re not describing the data but the tools used to manage it. The tools are constantly evolving too; Facebook’s new Unicorn database potentially makes Hadoop look small in the way Hadoop makes MySQL, an older database system, seem tiny.

What’s wrong with ‘Big Data’?

At the moment information warehouses are selling ‘Big Data’ as the solution to every problem. There are definitely cool uses of big data, like using sentiment analysis [2] on twitter data to predict the stock market. Google, using data gathered from search queries, is able to accurately predict flu epidemics [3] and feed that information back to world governments.

The issue is that Big Data is nothing without analysis. The UK’s largest Big Data supplier started the year off with an analysis of the most searched for terms over Christmas. Their conclusions [4]: onesies were the most searched for item and ‘the demand for onesies seems to be coming more from adult males rather than females.’ The data they provide seems to support this analysis with the top searches in order being ‘onesie’, ‘onesies’, ‘mens onesie’, ‘onesies for men’, ‘onesies for women’. This appears to tell a great story until some context is out around that data.

Searches for ‘onesie’ and ‘onsies’ produce search results that are largely catered towards women. Looking at the context that the data sits in changes the picture from one in which demand for onesies by men outstrips demand from women, to one where men don’t feel they are being adequately targeted and have to repeat the search with a qualification. Another view could be that men inherently consider the onesie to be a feminine product, so instinctively qualify their searches.

The other problem with Big Data is that it only tells you what has happened. It’s possible to make predictions based off the data, but without a full understanding of the context it’s not possible to know how likely those predictions are. The knowledge that onesies was the most searched term last Christmas is useless. Next year it’ll be a completely different picture. What does probably exist in the data, if you look back over several years, are clues as to what causes Christmas products to peak, from celebrity associations to expensive marketing campaigns.  You just might be able to spot another product with the same trend this year and bet on that. So, did people see the trend for onesies coming? Well, yes. But did Big Data help exactly? No.

 What is this year if not “The Year of Big Data”?

On the one hand, I do have sympathy with the January soothsayers. Big Data will soon become more affordable and more accessible to a lot of people. There also appears to be a huge marketing push by Big Data sellers. However, I just don’t believe that for 90% of businesses there is an advantage to be gained dealing with the data. There are much easier wins still to be had.

Like I said before, Big Data is just data and it doesn’t solve any problems on its own. It’s as easily misinterpreted as small data but with the added danger of having a lot of people believing the conclusions with more conviction.

What this year is going to be about is using regular data to close the loop on marketing attribution. The question asked by businesses and brands should be: how did those 100 people that walked into my store find me and how can I tie that back into my marketing activities? This isn’t about Big Data, it’s about sampling the population that really matters to you. This is also the data your competitors won’t have and simply can’t get at.

Either way, whether dealing with Big Data or just data, the truth lies in the analysis, not the gathering. An analyst looking into the data, no matter what the size, is the difference between a guess and a likely outcome when making conclusions.

So this year, and every year henceforth, should be “The Year of the Analyst”.

About Daniel Bowtell

Prior to joining The Search Agency as an Account Executive Daniel could be found touring the world as a professional musician. He is now firmly ensconced in the world of SEM and is dedicated to delivering the best value to clients, big or small. When he's not doing that Daniel is working his way round every trendy restaurant in London.