Big Data may be tough on our technology stacks, but the real challenges lie elsewhere.
Promises. Promises. Big Data sure makes a lot of them.
Increase the effectiveness of your sales (or political) campaigns by using behavioural data to divide customers into micro-segments.
Improve brand perception by monitoring the complex web of conversations across Twitter, Facebook and other channels and then engaging carefully with key influencers.
Analyse internal processes to find opportunities to reduce costs and increase responsiveness.
Sounds great, but is it real? Are people actually doing such stuff, or is it all vendor hype?
I'm in two minds. I've seen signs that the promises are meaningful. Real people can analyse real data and get useful insights from it, driving real business value.
For example, I've seen a university find ways to identify students who were at risk of failing. They could then provide the added support needed to help them pass. (Important if you care about people, but also from a business perspective: they only get paid for the ones who pass).
I've seen a truck manufacturer improve operations across its dealer network using the insight gained from vehicle breakdown records. I've seen a telco learn a lot about the type of products its customers want.
But these were only glimpses. People had started to explore their data and were getting the odd, random benefit. They were still falling well short of the grand promises. They ran into brick walls at least as often as they made headway. And I've seen other organisations give up almost as soon as they began.
What's going on?
Part of the problem is hype. The tech vendors are using Big Data as a hook to sell stuff. That's the way the tech industry works. Create a lot of buzz around a vague concept (read Wikipedia's definition of Big Data: almost anything could fit this definition) and hope enough people will latch on to it to drive Big Sales.
That fuzz doesn't help the rest of us. Worse, it pushes us to focus on technology. But the real challenges of Big Data lie elsewhere.
In retrospect, I've been dealing with Big Data for a long time, for more than three decades. I started as a geophysicist, processing oil and mineral exploration data.
We were certainly doing Big Data then: we needed dedicated supercomputers to handle it all. From there I moved into satellite image processing, and more special purpose hardware. Next I was building customer data warehouses, using yet another generation of specialist boxes.
Now it's the web and social media. But the hardware has changed – now we're using a cloud of commodity kit.
Somewhere along this journey, we switched emphasis. Until about a decade ago, we worked as generalist teams on specialist hardware. (I say "generalist" because everyone had to do a bit of everything – if you wanted to analyse data, you had to know enough about hardware and software to make it do what you wanted).
After that, we formed ever more specialist teams – the web team, the customer insight team, the IT team, and so on – and started to use general-purpose hardware.
I'm not sure that was a useful switch.
Yes, Big Data has always challenged our hardware and software stacks. (That's the definition of Big Data.) The complexities of modern cloud architectures and software stacks such as Hadoop certainly need a lot of skills to pull everything together. That drives a degree of specialisation within the team.
But the real challenge of Big Data has always been in integrating multiple perspectives. You need to generate meaningful business hypotheses about what might be going on. You need to map those hypotheses onto the underlying algorithms and statistical concepts in a way that makes sense. (It's easy to create bizarre conclusions if you don't respect the limitations of the data and the algorithms).
You need to meld together data from a host of different sources. And yes, you need to understand the technology well enough to make it all happen.
Integrating such a diverse range of skills is tough. Pushing everyone into silos just makes this integration a lot harder. Yet that's just how most companies have chosen to organise themselves.
Before these companies gain the full benefits of Big Data, they're going to have to think about how they get people to work together. Many organisations talk about cross-functional teams, but they rarely support them well.
They expect people to report into functional silos. When they do create "teams", they refuse to co-locate them. Their facilities police pounce on people if they shift the furniture around. They align rewards to individual rather than team objectives. And so on.
Maybe Big Data, with its promise of extremely attractive benefits, will provide the impetus we need to start thinking more about how we create cross-functional teams?
In the meantime, most of the talk – all those tools and technology – is just pulling our attention in the wrong direction.
(Image credit: Wikipedia)
Graham Oakes helps people untangle complex technology, processes, relationships and governance. He is a guest blogger on Econsultancy.
No hay comentarios:
Publicar un comentario