Three reasons why Big Data is awesome

In my last post, Three reasons why Big Data is a big load of baloney, I threw a stone down the streets of London and New York and wound up hitting a few Big Data advocates. 

And it is abundantly clear that Big Data arouses passion in people (or at least, as much passion as one can humanly feel for data.)

The thing about Big Data is that it can be interpreted in many ways. In this post, I take a step back and look at how Big Data is affecting digital marketing as a whole, and how maybe, just maybe, it's not a cliché but a fundamental shift in how we do business. 

As promised, here's part two in my series. Is Big Data all it's cracked up to be? Read on to find out more. 

1. Find competitive advantages

Big Data is defined by having Variety, Volume and Velocity of data  However, processing this was always hard. Now that there's powerful software to do just that, you can highlight asymmetries in your marketing mix and determine lucrative competitive advantages.

Let me illustrate with an example from the sport of Gods ('ice hockey' as they call it in Britain, or 'hockey' for everyone else.)

Back in 2009, the prevailing strategy was to send out your best players to play against the opposition's best players.

So, when your opponent sent his stars onto the ice, you'd match with your stars. This was commonplace and had been so since Wayne Gretzky was but a twinkle in his daddy's eye.

However, with the advent of big data, one smart analyst noticed an inefficiency in the marketplace, so to speak. 

When your top goal scorers are on the ice, you want them to start in a position that increases their likelihood of scoring. However, if they're starting a shift in the defensive zone against the opposition's best players, the chance of them scoring is extremely low.

The Vancouver Canucks noticed this, and decided to try something different.

They noticed that when their stars started a shift in the offensive zone, their scoring rate went up. When their defensive specialists started shifts in the defensive zone, they stopped more goals. 

And importantly, they learned that this was a pervasive phenomenon league-wide. 

Not rocket science, right?  But this is the thing. Without access to this large amount of information, and without the ability to process and analyse it in near real-time, there was no statistical foundation for their hunch.

So, check out the graph below (h/t Some Kind of Ninja.) It outlines the 'game state' (that is, in which zone, be it defensive or offensive, a shift was started) per player. 

In the 2009/20110 season, the Canucks' stars, two prolific, Swedish identical twins called Henrik & Daniel Sedin, were leading the league in offensive production.

  

The thing to look at here is the X-axis, which is the offensive zone starting rate.  Daniel and Henrik, while both starting more in the offensive zone, still spent nearly half their time in the defensive zone. 

This was effectively time spent not shooting the puck, which is counter-intuitive when their job is to score.

In 2010/2011, things changed.  They decided to send out their players based upon game state, not based upon opponent matching.  Here is the deployment chart from that season:

Notice how the Sedin twins overwhelmingly started in the offensive zone. More than 70%, along with their finger-biting Quebecois linemate Alex Burrows. 

In fact, those three started more in the offensive zone than anyone else in the league that year and their defensive counterpart, a good Canadian kid called Manny Malhotra, started in the defensive zone more than anyone else. 

For those of you who are interested, here's the league wide chart. 

The results? The Sedins were the top offensive pairing that year, and the Canucks finished the season as the best team in the league. They made it all the way to the seventh game of the Stanley Cup Finals. 

Sadly, they lost that seventh game to the Boston Bruins, and the streets of Vancouver burned. I blame this guy:

But, the point remains: if they hadn't had access to large amounts of real-time data, and didn't use it to determine and exploit market inefficiencies, they'd have never been so successful.

There's a litany of examples in the digital marketing sphere. And these examples wouldn't have existed were it not for marketers interrogating their data to find interesting relationships. 

These relationships then turn into competitive advantage, and force your competitors to play defence.

What competitive advantages have you found? Stick them in the comments below. 

So take a look at your data, and how you are acting in your marketplace. What are your assumptions?  What do you believe to be true?  By questioning these preconceived notions and using Big Data to interrogate whether or not they hold true, you can be the Canucks. 

Only leave the rioting to Vancouverites, it's their specialty. 

And therefore, Big Data is awesome. 

2.  Get data on the board's agenda

I've known a lot of data people over my career. In marketing departments, they're unavoidable (and are usually pretty nice people.) However, often they're (perhaps unfairly) perceived as the bane of marketers' existence. 

They're the people who set rules up around data usage. They're the people who build business cases to invest in complicated analytical tools. They're the people who talk about segmentation strategies, clusters, quartiles, and statistical significance, to many a blank stare.

And historically data people have been ignored by C-level executives. That is, until now.

I guess we can thank (blame?) this guy for being the first person ever to use the #bigdata hashtag:

Before long, the term was picked up by digital thought leaders, such as this guy:

And when Web 2.0 as a buzzword was waning, well, they were already talking about Big Data: 

This is not to say that Big Data was always used for good:

Anyways, CEOs aren't always on Twitter. But the mainstream intelligentsia picked up Big Data not long after. For example, it was first mentioned in The Economist on 25th February 2010.   

And of course, the surefire signal of something making it BIG in the business world is the quick proliferation of conferences, trade shows and summits. Yep, this has happened, there's loads.

Perhaps unsurprisingly, on Monster.com (US) there are more than 1000 jobs listed for Big Data. People are not only investing in technology, but also in people.

What this means is that boards of directors have taken notice of the fundamental importance of data to their business. While not using the phrase "Big Data", Econsultancy's own Marketing Manifesto lists data as one of the cornerstones of business – and there didn't seem to be a lot of disagreement on that point.

Maybe Big Data isn't the right term, and maybe it won't last. But what will last is the effect it's had on business culture. Decision makers now realise that the power held within their data is, in a word, awesome.

And therefore, Big Data is awesome.

3. Drive innovative products & startups

Having access to all this data is great, but one of the challenges is using it to make money. While point one above looks at situation-specific use cases, one of the real gifts Big Data has given the digital sphere is the genesis of innovative products and startups.

Take, for example, real-time bidding (RTB.)

A few years ago, RTB was sort of a little bit possible-ish if you did it in house and had a marketer/octopus hybrid, but it was hard and didn't really work. However, there was clearly lots of data out there.  The question was how to use it.  Enter RTB.

The whole theory behind RTB is that it takes in massive amounts of data, runs it through computational algorithms, and then spits out (and ultimately co-ordinates) the highest value media buys. RTB exchanges look at all kinds of data, both structured and otherwise, and use it to provide their customers with new, profitable advertising channels. 

So the innovators in this space have seen this opportunity and are actively monetising it. RTB compiles Big Data and uses it to reduce market asymmetries. It gives media buyers greater reach for lower cost, and lets them control spend and ROI transparently.

Here's what Dave Hendricks, COO @ LiveIntent has to say about Big Data and how it has allowed LiveIntent to grow 4485% over the last 3 years:

Big data for email is starting small with 'decision' time:  Mobile first?  Or responsive design?  This is 'Big Data' sticking it's foot in email's door.  Because, unlike the past where email marketers had the luxury of ignoring real-time signal, the growing trend towards mobile email consumption drives a very important 'decision' that is out of the control of the marketer:  do I display a mobile friendly template or the default desktop?  This is not something you segment for, it's something that is out of your control, just like where the opener is and when someone opens the message you sent.  This is only the start of the impact of real-time on the email ecosystem.

See? That's a great example of innovation that is a direct result of Big Data. 

And they're not alone.  Laurent Gibb, VP Media & Publishers at myThings, which won top prize for 'Best Use of Data' at the Performance Marketing Awards 2012, explains how the company has profited from Big Data :

Big Data, although quite a meaningless term in itself, translates for myThings into the ability to enrich the 1st party information we already use when processing over 3bn real time bid requests per day. Be it CRM, Publisher, third party, or any other source of data, Big Data enables us to provide a custom solution that fits perfectly around the advertiser's business objectives, using advanced custom segments and buying strategies that are completely algorithmic and programmatic.

Another fantastic example of an RTB technology that simply couldn't exist without Big Data and all that comes with it. What cool products have you found that monetise big data processing? Feel free to chuck some links in the comments.

There's loads more outside of the RTB sphere of course.  The list of startups that have developed cutting-edge products based upon Big Data is endless. 

Will they all still be around in 5 years? Maybe not. But the fact is that access to data is fundamentally a good thing for the digital marketing economy.  It levels playing fields while simultaneously stretching them. 

And therefore, Big Data is awesome.

Disagree?

Well, join the club. I've looked at it from both sides and have come to the following conclusion:

Big Data is awesome baloney.

If you use your data wisely, it'll work great for you  If you use it stupidly, well, like they say, stupid is as stupid does. The phrase Big Data is going to go the way of the Dodo in a year or two. But what won't change is its lasting effect on business culture. 

And as always, there's a debate to be had. Feel free to stick your point of view in the comments below.

All that I know is that these days you cannot run a marketing programme without looking at data. And if you are ignoring your data, I sure hope you're competing against us.