Jun 23, 2015
One of the fascinating changes I have seen during my career is how data analysis has changed from something carried out almost unnoticed in some back room by a couple of statisticians into a strategic imperative for every company. Our modern business systems generate massive amounts of data but it is not easy to winnow that information into insights that can be used to make business decisions.
Part of the challenge is that data has become more diverse and complex than ever. We still have transactional data, but for many companies the most interesting insights come from incorporating machine-generated data. Data from business applications, web servers, devices, and sensors can be used to attain business insight that is priceless. But success with this “Big Data” is often elusive. While Big Data receives a lot of hype, the reality is that few companies have cracked the secrets contained in their Big Data. In fact, Gartner has determined that while many companies are making investments in Big data, only a small percentage have actually deployed their solution.
What’s the problem? Organizations are trying to achieve data-driven insight the wrong way – by making investments in IT megaprojects. They pour time, effort, and dollars into hardware and software infrastructure as well as people to service that infrastructure. All of this effort is focused on just the first step – putting the data infrastructure in place. The payoff, if any, comes much later when people can use the data infrastructure to get insights that enable data-driven decisions. Given all of this complexity, it is not at all surprising that so few customers are achieving the results they desire from their data investments. These huge projects add so much cost and complexity that it can take years to see results – many don’t even get that far.
What if there was a different way, a way to avoid the data megaproject? Suppose you could replace it with a data service that immediately solved the problems that the megaproject was meant to address?
How would your business change if starting tomorrow morning, your analysts could come into work and find the data infrastructure already in place? What if that infrastructure already had high availability, security, and data protection as built-in features? What if you could begin working with that infrastructure immediately at only a fraction of the cost of what you’re currently using?
Now for extra credit, what if that data service made all of the constraints you’ve experienced with other data systems go away? What if you could store all of your business data and query it at any time using the tools and skills you and your team already have? Suppose you could work with both transactional and machine-generated data the same way and the data service automatically optimized storage and querying for both? What if queries ran so fast you almost can’t believe it is real? And that data service could grow and shrink as needed to support one user, ten users, one hundred users, or even thousands of users. The data service always has your up-to-the-minute business data; nightly data loads are a thing of the past.
If you could do all of this, how would it change your business?
- You could perform experiments you never thought possible. These could be done in an ad-hoc way without exhaustive advance planning.
- You could iterate and learn without constraints. The service is always available, always ready to answer that immediate business question.
- The limits that previously constrained your people go away. Your business team can quickly answer the questions they need using all of the data, all of the time.
- You could dramatically improve business results while still controlling costs.
So where can you find this data service? The cloud provides part of the answer, but by itself it is insufficient. Many vendors are delivering data infrastructure in the cloud, and yes, the cloud removes the need to build out hardware infrastructure. But these products are not the limitless data service described above. They use traditional data technology that has been hoisted into the cloud with all of the complexity and constraints that come with that. These cloud products still require an IT megaproject – they are complex, they need same management expertise as an on premise solution, and they still have all the same limitations.
To get that limitless data service, you need to rethink the problem. You need to start from the beginning and build an all-new service that is focused on solving today’s data challenges. In fact, it needs to be designed and built from the cloud up. Created to take complete advantage of the new technology that a modern cloud delivers to redefine the data solution. Naturally, this software is delivered as a turnkey SaaS service so IT is not burdened with the complexity of making it work. Like the cloud itself, the service is limitless and fully elastic. It supports 1000s of simultaneous uses. It works with all business data, and it delivers the best possible performance at the lowest price. It is blazingly fast. It is secure. There is no megaproject. Just answers that enable data-driven decisions.
In fact, the data megaproject is a relic of the past.
Starting today, the service that delivers data-driven decisions in the cloud era is available to everyone. Welcome to Snowflake.