So you’re trying to figure out whether your company should invest in big data analytics. You may be a CIO, you may be in marketing; doesn’t matter. You’re looking at what other people are doing to see whether this technology is going the way of smartphones or Microsoft Clippy.
You have my sympathy. There are those of us who find big data inherently fascinating and ultimately fruitful, as long as you have the data sources and you know the analysis that will help you the most. But as indicated by my Dickensian title, reading about big data may lead someone to think that they’re thumbing through a tale of two completely different technologies.
On the “best of times” side, it’s frequently easy to find stories of enterprises using big data to great advantage. Bob Violino wrote in Information Management last week about United Healthcare deriving benefits across multiple departments, including “financial analysis, fraud and waste monitoring, cost management, pharmacy benefit management, clinical improvements and more.” He quoted Ravi Shanbhag, the firm’s director of data science, solutions, and strategy, as saying, ““The data is so interrelated across business groups, I would be hard-pressed to find an area that is not directly or indirectly touched by these initiatives.”
Though the article was short on quantification, it’s clear that United Healthcare has made a big committed and is on the road to getting a big data bang for the buck.
Ditto for game manufacturer King Digital Entertainment, best known for the big entry in what a colleague of mine used to call “fritterware”: Candy Crush. It uses Hadoop and other tools together, under the auspices of more than 60 data scientists. They analyze “large swaths of player data,” including time spent playing, transactions made, and devices used. In turn, they’ve made design tweaks to increase player retention, according to Computing article from last week.
On the “worst of times” side, it’s equally easy to find pundits and surveyors who give the impression that big data is more hype than hope.
Even frequent big data supporter Mary Shacklett wrote last month on Tech Republic about why big data analytics strikes out sometimes, including as she notes, failing to predict flu outbreaks, oil slumps, and even baseball success. She notes accurately, “Big data analytics is still in an early learning period when it comes to predicting human behavior. Modern humans have been on the planet for 200,000 years, and we still haven't figured out why we act the way we do! Even with associational ‘thinking’ and processing, machines are limited in predicting human behavioral outcomes.” Would that kind of outlook convince your C-suite to invest millions of dollars, though?
eWeek reported last week on a study commissioned by Snowflake Computing, a developer of data warehouse solutions for structured and semi-structured data, reported that big data is “hyped, but not deployed.” Its survey of more than 300 technology and analytics professionals showed that just 16 percent have invested in big data, and “[o]f the companies that have invested, 5 percent have fully deployed big data strategies and 11 percent are in a pilot.” According to the results, 41 percent said they are “intrigued” but have yet to investigate big data further.
To invoke another long-dead English writer (this time Alexander Pope), that falls under the category of “damning with faint praise.”
For every survey, though, there is usually an equal and opposite survey. This one falls into a third category, best labelled as “cautious optimism.” Computerworld reported a couple of weeks ago that according to a survey of more than 1,100 IT executives, “interest in big data continues its steady rise,” with more than half currently implementing or planning to implement data-driven projects in the next year. As author Johanna Ambrosio notes, however, there are still budgetary limitations, security issues, and lingering issues with product quality. “[S]ome 40% judged existing products and services as excellent or good, but 44% called them adequate or poor.”
Well, nobody’s perfect.
Let me close with a reference to a story that looks like it’s going one way and is really going another, Bernard Marr’s piece on Business Insider a couple of weeks ago on why big data is just a fad. I’m with him on this one because I feel the same way about cloud computing. Just as someday we’ll stop talking about cloud and just refer to infrastructure, we’ll also someday stop talking about big data and refer to it for what it really and truly is: just data.
