Rethink What You Know About Creating a Data Lake for JSON

Over the last 10 years, the notion has been that to quickly and cost-effectively gain insights from a variety of data sources (e.g. weblogs, clickstreams, events, IoT and other machine-born JSON or semi-structured data), you needed a Hadoop platform. The proposition with Hadoop-based data processing is having the flexibility, capacity and performance to store and analyze an array of data types, in a data lake, within a single repository.

In reality, analyzing data with a Hadoop-based platform is not simple. Hadoop platforms start you with an HDFS file system, or equivalent. You then need to piece together about a half-dozen software packages (minimum) just to provide basic enterprise-level functionality, such as provisioning, security, system management, data protection, database management and the necessary interface to explore and query data.

Despite the efforts of open-source communities to provide tools to improve the capabilities to operate at the highest enterprise-class level, there is the constant need for highly skilled resources–those who can continually support Hadoop to keep it up and running while enabling users to do more than just explore data.

Snowflake, which is built for the cloud and delivered as a service, provides you with a different option for handling JSON and semi-structured data. Just point your data pipelines to Snowflake, land the data in our elastic storage repository and you have instant access to a bottomless data lake and a full-fledged data warehouse. With Snowflake, you can easily load JSON and query the data with relational, robust SQL. You can mix JSON with traditional structured data and data from other sources, all from within the same database. You can also support endless concurrent analytic workloads and work groups against the JSON data in Snowflake. This is all possible without any impact to performance, concurrency or data consistency-virtually any at scale of concurrency, one or 1,000.

As a combined data lake and data warehouse platform, there is much more you can do with Snowflake. Read more about it with our new eBook, Beyond Hadoop: Modern Cloud Data Warehousing.

You can try out Snowflake for free. Sign up today and you’ll receive $400 dollars of free usage credits, plus the capability to create a sandbox or launch a production implementation.

Rethink what you’ve been told

Hadoop is not a prerequisite in order to gain insights from JSON or other machine data.

When you need to store, warehouse and analyze JSON and other machine data to develop insights or to uncover relationships that can drive business decisions, rethink what you’ve been told. You can support all of your structured and semi-structured data warehousing and analytic workload business needs with a single tool–a tool that is built for the cloud and is an ACID-compliant, fully relational SQL environment that millions of SQL users and programmers already know and are accustomed to. And, Hadoop is not required.

Be sure to keep an eye on this blog or follow us on Twitter (@snowflakedb and @miclnixon1) for all the news and happenings here at Snowflake.

 

 

 

 

3 Ways a Data Sharehouse™ Can Catapult Your Data-Driven Initiatives

How would your organization benefit from effortlessly sharing limitless amounts of data with business partners and commercializing that data to share with other organizations?

With most data sharing methods today, the best you can do is imagine these benefits because it’s cumbersome, time consuming and costly to share even small slices of data. If you share data using an FTP approach, you will spend time deconstructing, scripting, securing and governing the data, and your data consumers will spend time reconstructing, scripting and rebuilding the data. Sharing data via email can be very slow and insecure. Email is not even a practical option for large data sizes and is exponentially more difficult if sharing a large database.

Sharing data using a basic cloud storage service is equally inefficient, including the lack of ability for you or for your data consumers to query the data directly. And if you want to enable “direct” data queries, without loading, you will likely evaluate a traditional cloud data warehouse or Hadoop platform that is “integrated” with a data lake storage platform. Do you think this will be simple? Not so fast! You’ll have to factor in the need to manage a separate data catalogue and external tables to share data, or you’ll have to contend with data inconsistency and performance issues.

These limitations all add up to mean that traditional options to share data are more difficult to take data sharing to the next level of capabilities, which goes beyond finding insights to commercializing your data. If any of this feels familiar, consider these three reasons why Snowflake Data Sharing – the technology making it possible to create a Data Sharehouse™ – is so compelling.

1 – Drive simplicity: Allocate more of your time and resources to strategic data sharing projects

Traditional options to share data create unnecessary data processing and platform complexity. Complexity adds burdens and requires a lot of extra time and resources, including infrastructure costs, for you and for your data consumers. 

Snowflake Data Sharing, on the other hand, is an all-in-one, data warehouse-as-service solution that makes sharing data simple and easy. The vision for the Snowflake architecture, from day one, featured separation of compute, storage, and services to allow unlimited scaling, concurrency and data sharing.  

 

 

For example, with Snowflake, it is not necessary to manage an external data catalogue and not required to have a separate effort to build data security. It’s all built-in. In addition, Snowflake metadata management makes it possible to keep tabs and track all data warehouse activities. Thus, to share data, follow these steps:

  • Point your pipeline to land your data into Snowflake and set up your warehouse.
  • To share any portions of your data warehouse, use Snowflake’s included SQL client to CREATE a new database shell that could contain the entire database or any portions of it. CREATE a reference list for the objects you wish to share.
  • Issue GRANT statements that will enable access to the shared-database and any objects referenced within “the share”.

These are all the steps you will need to quickly and easily share data. All of the related metadata activity, including cataloguing, happens automatically within the Snowflake data warehouse service. No additional infrastructure or separate cataloguing effort required.

2 – Enable more valuable insights: Your data consumers will always operate with live and current data

Most options to share data require you to unload the data from your data warehouse in order to transmit or email to your data consumers. Or if using a cloud data warehouse platform to avoid this, it relies on a physically separate storage pool in order to scale and share data. The downside with either option is that the data set is read-only and disconnected from any updates that occur within the data warehouse. If you modify the data set, or if the data set is regularly updated, you must reload (ETL) it into the data warehouse, operate on it, and then unload it again to retransmit or place it in the shared storage pool.

For data consumers that received or were connected to the old data set, this means there is a period of time during which they’ll be exposed to stale and inconsistent data. Fresh data won’t be available until a new transmission is made or until a new connection is made to the fresh data. No one wants to run analytics on stale data.

Snowflake Data sharing delivers a better choice. Because data sets shared with data consumers within the Snowflake environment are live, in real time, and from the data warehouse, data consumers will immediately see fresh data as soon as an update or transaction is committed and successfully executed. Data consistency is maintained, without any extra effort from you. Your data consumers will not have struggle with decisions to run analytics now or wait for a new update.

3 – Support any scale, with ease: Seamlessly and cost-effectively share data with any number of data consumers

At the end of the day, it’s all about growing and expanding your business and services while providing excellent experiences for the customers and consumers of your data. If you anticipate sharing data with tens, hundreds, or thousands of data consumers, each of which with unique data sharing requirements, how can you easily support this? And support the growth without manually building more clusters, managing external tables or metadata stores, suffering through performance penalties or creating data inconsistencies? It would be very difficult or impossible with other architecture approaches to accomplish these objectives.

Snowflake data sharing allows you to scale and easily add data consumers and specify granular secure views, all at the highest performance profile and with data consistency. On the other end, with Snowflake, your data consumers can immediately use and query the shared data, also at the highest performance profile.

Summary: Share and imagine more

These are just a few compelling examples of how data Snowflake Data Sharing, and the Data Sharehouse approach, can transform the creation of high-value business assets from data. It’s a new, exciting, powerful and easy-to-use feature of all Snowflake data warehouse deployments.

Also check out data sharing blogs from Snowflake customers Snagajob and Playfab. To learn more, you can easily signup for Snowflake here. Or jump to our webpage to review our data sharing ebook, white paper, demo, and other resources. In future posts, we’ll cover more details about the technical capabilities and business-value advantages of Snowflake Data Sharing – built for the cloud. Stay tuned!

Snowflake establishes UK foothold with Thibaut Ceyrolle as VP of Sales

Snowflake’s journey to streamline access to data-driven insight in the cloud age continues to capture the imagination. From the beginning, we knew our built-for-the-cloud data warehouse would solve a widespread industry problem. But competing in the data warehouse industry means we’ve adopted a degree of modesty in the face of gigantic opposition. In light of our growing customer base, and support from innumerable sectors, we can start to be a little less bashful.

Earlier this month we secured another $100 million in funding. This will help expand our current operations and establish new footholds globally. We’re pleased to announce we’ve begun trading in the UK with super-smart new offices in Paddington, London as we increase our global reach.

To meet the demands that come with our ambitions, we have appointed Thibaut Ceyrolle as our Vice President of Sales for EMEA.

Thibaut joins us following a career of nearly two decades at the forefront of disruptive technology. Following a start at Hewlett-Packard in ‘98, he has been an instrumental player in driving the cloud revolution from the outset. He quickly ascended through the ranks of digital transformation proponents DevoTeam and BMC Software, before serving as VP of EMEA Sales at Bazaarvoice. An expert in bringing complex technologies to the wider market, Thibaut brings the drive, experience and international outlook needed to conduct strategic promotion of our service. Thibaut is also experienced in launching new offices and expanding operations to new regions and building a strong corporate culture similar to Snowflake’s.

In early 2015, we offered the first commercial version of Snowflake. The one and only data warehouse built for the cloud. We didn’t set out to improve a flawed legacy architecture, we set out to create something new: a fresh start in and industry filled with legacy products. And we’ve been noticed.

In the beginning, it was mostly early cloud adopters that saw the potential of our technology. Since then, we’ve seen horizontal expansion as every industry has come to see the benefits. Having recently signed our 500th customer, it seems we’re starting to snowball (sorry!).

Snowflake is a service we believe will continue to replace legacy on-premises and cloud systems quickly, quietly and, with Thibaut at the UK helm, ubiquitously. Many would think us overly ambitious given the monolithic competition, but the data warehousing industry has let inefficiencies fester. Snowflake’s a new breed entirely and companies want what we’re offering.

The tech world is fed up with wrestling for access to data and Snowflake is just too good a product to pass up. The challenges associated with handling big data is a limitation of legacy technology, not a fact of life. The Cloud has been around for quite some time, but surprisingly, Snowflake is the only cloud data warehouse solution built from the ground up for the cloud. We’re honoured to be the ones bringing relief to customers moving to and accelerating their business in the cloud.

We’re overjoyed with our recent round of funding, but it is based in hard evidence. The incomparable flexibility and speeds we have shown Snowflake to be capable of: up to 200 times faster for a tenth of the price.

No-one has done what we’re doing for the industry. We expect Thibaut to find a hugely rewarding experience here.

Snowflake Cloud Analytics City Tour

Join Snowflake at The Cloud Analytics City Tour in June to hear from leading cloud analytics and data practitioners. The international tour kicks off in London on June 1 and will visit eight US cities and will center on the theme “Your Data Struggle Ends Now”. The one-day events will bring together the brightest minds in data and analytics to discuss the latest trends, best practices and lessons learned in data warehousing, big data and analytics.

Register here for the London Cloud Analytics City Tour stop before 5 May, 2017 to receive the Early Bird discount. To receive an additional 15% off the Early Bird price, use the registration code: SPECIALBLOG.

Register here for the US Cloud Analytics City Tour stops. For the latest updates on events, speakers, and registration, visit the Cloud Analytics City Tour website.

Snowflake Vision Emerges as Industry Benchmark

Technology research and analysis firm Gigaom has ranked Snowflake as the #1 cloud data warehouse in a recent study. We surpassed enterprise data warehouse products including, Google BigQuery, Teradata, IBM dashDB, HPE Vertica, Microsoft Azure SQL, SAP HANA and Oracle Exadata. Snowflake emerged with a top score of 4.85 out of a possible 5.0. The competition averaged a score of 3.5. The six “disruption vectors” Gigaom used as its key scoring criteria are congruent with what we wanted to achieve back in the summer of 2012, when we started Snowflake.

But long before we wrote the first line of Snowflake code, we asked one another: “What should a data warehouse deliver that no other product has before? How can we enable organizations to make the best, data-driven decisions? And how will the world’s most powerful data warehouse help organizations achieve their existing goals and help reveal their future goals?” We then set out to answer those questions.

We wanted to enable organizations to easily and affordably store all of their data in one location, and make that data accessible to all concurrent users without degrading performance. We also wanted Snowflake to scale infinitely, with ease, and cost effectively so organizations would only pay for the compute and storage they used. And the product had to work with the tools that users already knew and loved. Finally, we wanted a data warehouse that required zero management by our customers – nothing to tweak, no tuning required. These defining qualities aligned with the new world of cloud services, and they are what formed the foundation of Snowflake.

What’s happened since the early days of Snowflake? We got to work, and we stuck to hiring the best engineers the world has to offer. We built Snowflake from the ground up, for the cloud, and incorporated all of these elements as the core of the product. In early 2015, we offered the first commercial version of Snowflake – the one and only data warehouse built for the cloud. Since then, our engineering team has added more and more industry-leading capabilities to Snowflake, leapfrogging the traditional data warehouse vendors.

Along the way, we’ve hired high-calibre teams to execute the sales, marketing and finance functions of the company so our customers and partners get the highest value from working with Snowflake. We also built a great customer support organization, providing the level of service our users love. In more recent times, we’ve expanded operations outside of North America to Europe, with Asia-Pacific and other regions coming online soon. We’ve also added Snowflake On Demand™ – the easiest way to get started with Snowflake by simply signing up on our website with just a credit card. All of these efforts over the past four years have led to Snowflake’s most recent inflection point – being chosen as the number one cloud data warehouse.

What does all this mean? Snowflake’s current and future customers have every opportunity to explore all of their data in ways they never thought possible. They can gain the insight, solve the problems and create the opportunities they simply couldn’t with their previous data platforms. We committed to building the world’s best data warehouse – the only data warehouse built for the cloud. Our customers, our partners and now the industry have indicated we’ve likely achieved what we set out to do back in the summer of 2012. Going forward, we’ll continue to serve our customers and partners with the best technology, the best solutions and the best services available.

Read the full report >

Migrating to the Cloud? Why you should start with your EDW

Many organizations we engage with are seriously considering transforming their business and moving some (or all) of their IT operations into the cloud. A lot of executives I have encountered are struggling with the same question: “How do I get started?” There is a strong case to be made that starting with your Enterprise Data Warehouse (EDW), or at least a data mart, is the fastest, and most risk-free path, with added upside potential to increase revenue and set you up for future growth. As operational data volumes continue to grow at exponential rates, it’s not a matter of if you go to the cloud to manage your enterprise data, but when.

Before going too far on your cloud journey, I would recommend an exercise in segmenting your business from an IT perspective in a very simple way. To get you started, let me suggest five possible categories, along with some risks to consider for each:

  • Customer-facing Applications – This is the heart and soul of your business. If something goes wrong, you lose business and revenue, and people potentially get fired. Risk: HIGH
  • Internal Applications – Mail, Payroll, General Ledger, AP, AR, things like that. Every person inside the organization relies on at least one of these services, and a lot of analysis needs to take place to figure out all the integration points to ensure nothing gets missed during a migration to the cloud. Risk: HIGH
  • Desktop/Laptop OS and Applications – There are whole books and schools of thought about how to migrate these, which means it’s a big decision and a big deal. Impacting everyone in the company on your first cloud initiative? Risk: HIGH
  • Operations Monitoring and Alerting – Got a Network Operation Center (NOC)? These guys are integrated with every system that is important, so moving them to the cloud could be a large undertaking. Risk: HIGH
  • Reporting and Analytics – Hmmm….if my constituents don’t get their weekly or monthly reports on time, is that a disaster? Can they get by with a small outage during the migration? Risk: LOW

Starting with the Data

Let’s take a closer look at why starting your cloud journey with your EDW could be a viable option, and even have some benefits that could help sell the idea (of the cloud) internally. In no particular order, I would highlight these points:

  • Doesn’t disrupt the business – Many EDW implementations are not mission critical today (as compared to enterprise applications). As more data becomes available through social media or Internet of Things (IOT) applications, businesses need access to much larger volumes of data and they will want access to it earlier in the data pipeline. Traditional DWs contain aggregations and are used for doing trend analysis, analyzing data over a period of time to make strategic, rather than tactical decisions. They are not architected to handle this new influx of raw data in a cost-effective manner. By starting your cloud journey with the EDW, you reduce risk (by going to a more flexible architecture) while getting your team early exposure to working with cloud services.
  • Doesn’t disrupt internal users – When moving to the cloud, you want to show incremental success and don’t want to add a lot of unnecessary risk. It’s simple to keep running your existing EDW in parallel with your new cloud DW, giving you a built-in fall-back plan for the early stages. Or you may decide to start with a small data mart as a pilot project.
  • Start-up costs are a fraction of on-premises, appliance solutions – Some of our customers invested as much as $10 million (or more) years ago on a data warehouse appliance that is now outdated technologically. And the renewal costs to keep that tech going are coming due. If they re-invest another huge sum of money, this will delay them getting to the cloud by another 4-5 years, putting them behind their competition. Rather than outlaying a large capital expenditure to extend the life of the older technology, it may make better sense to move to the cloud. The cloud offers a utility-based model, allowing you to pay for what you use and when you use it, as opposed to what you think you are going to need 2-3 years in the future. As a result, not only is the cost of entry lower, but you are not risking a huge sum of money to make the move.
  • Data is growing at an exponential rate – Will you ever have less data to worry about in your business? If you plan on being successful, I don’t think so. Many organizations are looking at new and different ways to manage and analyze ever-increasing volumes of data coming in various formats from multiple sources (such as semi-structured web logs). Your current on-premises EDW was not designed for this kind of workload or data.  If you are considering changing infrastructure platforms to accommodate it, why not select tools that were built for today’s modern data challenges instead of legacy-based architectures? Moving to the cloud also gives you the opportunity to consolidate operations and streamline business processes.
  • Enable new capability – There are some new analytic paradigms happening in the cloud (such as machine learning). Cloud-based platforms allow you to work with both detailed and aggregated data at scales never imaged (see the case study about DoubleDown as an example). Need to run a complex analytic job with a 256-node Massively Parallel Processing (MPP) cluster for an hour, and then shut it down? No problem. Can your platform support a thousand users without concurrency issues?  How would that change your business if it could dynamically adjust to handle those new demands?

As with any infrastructure move, the benefits have to be clear enough that the status quo mentality can be overcome and analysis paralysis doesn’t push out your journey to the cloud for months or even years. The beauty of the cloud model is that it is easy to start small and scale without risking a huge investment up front. Every business needs some proof before committing time and resources to move anything to the cloud and your EDW is a perfect candidate. Snowflake is the first and only EDW built for the cloud to be truly elastic for all of your analytic and big data needs.

Please feel free to reach out to us at info@snowflake.net. We would love to help you on your journey to the cloud. And keep an eye on this blog or follow us on Twitter (@snowflakedb) to keep up with all the news and happenings here at Snowflake Computing.

Looking Back at 2016 Predictions

Last December, I made some predictions for 2016. As we approach the end of the year, I thought it only fair to look back and compare what I predicted to what has happened.

Do or die for big old tech

This was an easy one to get right. Big old enterprise tech companies are hunkering down and watching the world pass them by. HP and Dell are vying to be the king of legacy. There is money in this but who really wants to wear that crown?

IBM is trying to move on with Watson but can Ginni Rometty really pivot that aircraft carrier? And can Watson provide Jeopardy-winning answers for a variety of industries without an army of IBM consultants to spoon feed it? Only time will tell but there is reason to be skeptical.

At Oracle, Larry seems to have discovered the cloud (and will probably soon claim that he invented it). But he remains confused about what a cloud really is. When Oracle talks about Exadata Cloud Service, legacy hardware in a managed services datacenter, they demonstrate they’re still lost in the fog.

Overall, 2016 was not a good year for big old enterprise tech.

Public cloud wins, but who loses?

My prediction on the progress of private clouds was almost an understatement. This year, the move towards private clouds has been slower than molasses on a cold winter day. VMware continues to miss the mark, failing to deliver a cost-effective private cloud solution. And Openstack is a confusing grab bag that requires a huge SI investment, which is beyond the reach of almost all customers.

Meanwhile, almost every company, including most financial services, is now committed to adopting the public cloud. Amazon of course is the big winner but Microsoft has shown once again they will persevere and succeed. Last year, I picked Google as the wildcard. Diane Greene appears to have brought focus to Google and they clearly gained ground in 2016. Google possess the technical capability but they still need to get a lot more serious on the sales side as they have no enterprise experience. A recent query on LinkedIn shows 465 sales openings for Microsoft, 604 sales positions for Amazon, and only 85 open sales roles for Google cloud.  Google can’t compete against Amazon and Microsoft with just 85 more sales people.

The other major public cloud player that has emerged strong in 2016 is Alibaba. China cloud is set to explode in 2017. While it will be tough for Alibaba to gain traction in the US, in China it will almost certainly be the winning player.

All of the other public cloud wannabe’s are in a world of hurt. It looks like we’ll have four public clouds – Amazon, Microsoft, Google and Alibaba.

Spark divorces Hadoop

As I predicted last year, 2016 was not a good year for Hadoop and specifically for Hadoop distribution vendors. Hortonworks is trading at one-third its IPO price and the open source projects are wandering off. IaaS cloud vendors are offering their own implementations of the open source compute engines – Hive, Presto, Impala and Spark. HDFS is legacy in the cloud and is rapidly being replaced by blob storage such as S3. Hadoop demonstrates the perils of being an open source vendor in a cloud-centric world. IaaS vendors incorporate the open source technology and leave the open source service vendor high and dry.

Open source data analysis remains a complicated and confusing world. Wouldn’t it be nice if there were one database that could do it all? Wait, there is one, it’s called Snowflake.

What do Donald Trump and EU bureaucrats have in common?

Looking back at 2016, I guess not much. 2016 is a year that EU bureaucrats would rather forget and The Donald will remember forever.

On the privacy side, we saw some encouraging news with the creation of Privacy Shield. That said,  Privacy Shield is already being challenged and this space remains uncertain. On a purely positive note, Microsoft won the case in Ireland that prevents the US government from grabbing data stored in other countries. The ruling was critical for any U.S. cloud company that has a global footprint.

Perhaps the most encouraging thing from 2016 is that Europe has a full plate given the challenges of Brexit, a Donald Trump-led America, ongoing immigration issues and upcoming elections with strong populist candidates. Given these problems, concerns about privacy are likely to take a back seat so the bureaucrats may be content to stand behind Privacy Shield.

About that wall, Donald hasn’t said too much lately but I think we will see something go up on the border. He loves construction.

The True Value of Cloud Data Storage Continues to Emerge

We’re in interesting times. Like most significant trends, the data-driven economy revealed a powerful approach that was unique but always in plain sight. We listened and watched closely as experts across industries and different roles promulgated the benefits of capturing, storing and using data from every corner of cyberspace. And not far behind came a related and more interesting topic of connecting the offline world to capture previously unimagined amounts of data, ranging from kitchen appliances to jet engines. This we now know to be the Internet of Things (IoT).

We all acknowledged this data shift would change how companies do business and how we live our lives. As with all significant themes, comes additional thought on the ‘how’. Once we capture all of this data, how will we manage it? How will we effectively store and access petabytes of data, and more, so we can put that data to work?

These aren’t questions just for governments of the largest countries or for global enterprises. All organizations, from the garage start-up to mid-size companies are keen to harness the insight derived from more and more data. As wonderful as this seems, it all comes down to technology and cost. The cost of storing that data, and the technology to easily derive insight from data. But how does an organization accomplish this within their financial limits?

Our founders placed this at the heart of Snowflake. Before they typed the first line of code that ultimately brought the Snowflake cloud data warehouse to life, they wanted to enable data without limits. Snowflake’s built-for-the-cloud architecture truly separates compute from storage, allowing customers to easily scale either resource up and down. This also means Snowflake customers can focus their efforts on the highest value of data warehousing – compute. This is just one of many strategic advances, along with our unmatched technology, that makes Snowflake the most powerful and affordable data warehouse for all of an organization’s data warehousing and analytics.

With that said, Snowflake lowered its storage pricing in October to match Amazon’s S3 storage price. Today, Snowflake again lowered its price to match Amazon’s latest S3 price reduction. This strategy is a crucial component to truly realizing a data-driven world for all – data without limits. The amount of data the world creates continues to increase at an exponential rate. And to harness the insight from that data, organizations need the best technology at the best price. Snowflake has always been there and always will be.

To read more about our latest pricing announcement, click here.

Challenges and New Opportunities in Data Analytics

Fall is conference season in the industry, and this fall there has been no shortage of discussions and insights about data analytics at events both big and small. The Cloud Analytics City Tour has been a highlight here at Snowflake, but we’ve also seen the analytics conversation front and center at big conferences like Dreamforce.

The Challenges of Data Analytics

Our Cloud Analytics City Tour, now entering its home stretch, has brought together a diverse set of attendees, with small entrepreneurs sharing the room with people from some of the most established companies around. That diverse audience and the thought leaders who participated as speakers have provided some great discussion and insights.

For one, it’s clear that data analytics in the cloud has quickly become a topic of mainstream interest to organizations of all stripes and sizes. In fact, the conversation has moved on from “should we consider data analytics in the cloud at all” to “how do we figure out what to do in the cloud and how”?

That shift was reflected in some of the key themes and insights we’ve been hearing on the City Tour. Among those themes and insights:

  • The challenges are more than just technology. We heard repeatedly that one of the biggest challenges in cloud analytics is getting organizational buy-in. Even though acceptance of cloud has grown, getting people to do things differently still takes a lot of work.
  • Data integration and analytics now need to be a continuous process. The batch, scheduled approach to making updated data and analytics available no longer meets the needs people have today. Continuous data integration is becoming vital as organizations look to drive agile, data-driven decision-making throughout their organizations.
  • Finding great analytics people remains hard. The “people issue” – finding the right talent for analyzing data, is now even more urgent. However, it’s still hard to solve even as a greater number of people become data savvy.
  • Data quality still matters. While the technology to manage large and disparate sets of data is far more accessible in part because of the cloud, the quality of the data is still a challenge – how do you verify and normalize the data as quickly as your system can deliver and parse it?

Bringing Data Analytics to All

The importance of data analytics was also front and center at other conferences. At Dreamforce, the former Salesforce CRM conference that has now evolved into a much broader event encompassing wide-ranging business and technical topics, data-driven decision making for competitive advantage was a key theme. However, the conversation at Dreamforce has evolved from last year’s spotlight on the importance of using “big data” to a focus this year on how the nature of this data is changing, and on how to practically use more of the new types of data in everyday decision-making without being overwhelmed by its complexity.

What was most interesting about this discussion was that there were clearly two camps: increasingly sophisticated organizations with access to the skills and resources to be able to apply the latest data analytics approaches, and organizations that do not have in place or within reach the skills and resources to enable data-driven decision-making for greater insight. Those deep-pocketed enterprises who are rebuilding their entire infrastructures with the help of consultants like Accenture are leap-frogging into new productive use cases and revolutionary advances in deep learning.

The result is that well-funded start-ups who can attract highly skilled resources (and who can start from scratch) and those deep-pocketed enterprises who are rebuilding their entire infrastructures with the help of consultants like Accenture threaten to leapfrog the millions of organizations stuck in the middle who may know what they want to do with data and analytics, but don’t know how to get there. To add to the complexity, not only the technical infrastructure but the mindset within the organization and across departments needs to change.

For organizations across that spectrum, new solutions have emerged. Salesforce’s announcement of Einstein, a data analysis solution for data in Salesforce systems, is one example. But even more importantly, cloud analytics and systems designed to support it are making analytics accessible to more than just the well-resourced 1% of organizations.

As we have learned from the nimble companies that have gone from startup to billion-dollar unicorn in the last five years, thinking and operating in the cloud is the ultimate enabler. For more established companies hindered by legacy systems, changing the technology is now the easy part with solutions such as Snowflake available. But the rewards in overcoming these cultural and process barriers are invaluable to any organization that doesn’t want to be left behind in this next wave data revolution.

To connect with like-minded revolutionaries and learn more about how to move your organization’s data sophistication to the next level, join us at one of our next Data Analytics forums, including this week’s event in San Francisco as well as upcoming events in Chicago and Los Angeles. The best learning happens in person, and we hope you have or will take advantage of our Cloud Analytics City Tour as a great forum for intelligent discussions and meaningful insight.

The most powerful and easy-to-use data warehouse is now the most affordable

When I joined Snowflake nearly three years ago, I knew the seasoned engineers who envisioned and developed the product were on to something big. I’ve spent nearly my entire career in the database business but hadn’t seen anything comparable to Snowflake – a built-for-the-cloud data warehouse that’s powerful and easy to use for analyzing all of your data.

Snowflake has now taken another leap forward for our customers. We’ve launched three key initiatives to enable organizations to easily and affordably store and analyze all of their data in one location: Snowflake. No more data silos, data lakes or duplicate systems to manage the volume, variety, velocity and cost of today’s data.

Our first initiative upends the traditional cost structure that has made the data warehouse a precious, limited resource in the past. We’re making Snowflake the most affordable data warehouse available by removing the cost of data storage as a barrier to bringing together all data in one place. Taking advantage of our unique architecture, we’ve lowered our storage price to as low as $30/TB/month, which represents a 75 percent storage cost savings to our customers. The most powerful and easy-to-use data warehouse is now the most affordable. Thanks to our built-for-the-cloud architecture, specifically our separation of compute and storage, we’re able to price storage at the same price as Amazon S3 – Snowflake’s cloud storage provider.

Why have we done this? Our customers continue to tell us how much Snowflake has changed the way they use data. But they’re understandably concerned about the cost to store much larger volumes of data in Snowflake. Therefore, we’ve changed our storage pricing to help customers focus on the much more important process: analyzing data for insights.

Secondly, we’ve added a quick and simple way to get up and running with Snowflake. Snowflake On Demand allows customers to start using Snowflake with just a credit card and a simple sign-up process via our website. Data users of all types can now experience the many benefits of Snowflake without friction or delay.

And thirdly, Snowflake has launched is newest deployment in the Frankfurt (EU) region. Organizations with EU headquarters, and multinational organizations with EU operations can now keep their data in the EU. This is huge for any organization keen to keep their EU data close to home, and any for organization that wants to advance their global data initiatives from region to region.

All of these offerings will be available in November.

At Snowflake, we love to hear what our customers have to say. Their input, and our hunger to deliver and evolve the best data warehouse available, drive everyone at Snowflake to serve our customers the best way possible. Snowflake’s technology, solutions and customer-centric strategy will never cease to evolve. We look forward to continuing our mission of helping organizations advance their operations, serve their customers and lead their industries with the insight derived from data without limits.

For more details on all of Snowflake’s newest initiatives, view Snowflake’s main announcement and our individual announcements on our price reduction, Snowflake On Demand and Snowflake’s deployment in Frankfurt.

As always, keep an eye on this blog site and our Snowflake-related Twitter feeds (@SnowflakeDB) for more interesting things about Snowflake, and for updates on all the latest action and activities here at Snowflake Computing.

Are all databases in the cloud, built for the cloud?

“Crap on-premises, move it to the cloud; it is still crap, just in the cloud”

David Linthicum, speaking in NYC at a recent CloudAnalyticsCityTour.com

Traveling around the country for the CloudAnalyticsCityTour.com, we see a fundamental shift in the world of analytics and the impact cloud has had on analytics. As more data sources come online (IoT, Mobile, Applications), the cloud economies of scale allow companies to bring in larger data sets and analyze them effectively. Increasingly, all major legacy data warehouse and big data platforms are moving to the cloud, to make sure they can accommodate customer workloads. However, listening to data management and analytics users in NYC and Boston, there is a genuine sense of frustration that existing platforms, whether on-premises or moved to the cloud, are not serving their purposes. Even in the cloud, these legacy platforms still require the customer to either sign up, typically in advance, for excess capacity to cater for peak performance or concurrency requirements, or they must risk sacrificing performance and business value to offset costs. Plus, none of the platforms offer the simplicity of a true, modern, built for the cloud SaaS-based solution. So you actually end up needing the same expensive resources that you did on-premises and just connecting to them in the cloud. Hence the quote: “Crap on-premises, move it to the cloud; it is still crap, just in the cloud.”

Additionally, the picture has only gotten more crowded, and confusing, with the introduction of open source platforms. As David Menninger (@dmenninger) puts it: “Hadoop is like a free puppy”. Whether in the cloud or on-premises, these open source “free puppies” need all sorts of care and feeding. They end up using even more resources than more traditional alternatives. According to a recent study by Ventana Research (link, registration required), only 1 in 6 organizations have Hadoop skills while 6 out of 10 have SQL skills. As Michal Klos, Engineering Director, Data Platform, Localytics hinted in a recent talk at the Boston edition of the CloudAnalyticsCityTour.com: they developed a data warehouse on their own in Presto for 1 year and eventually gave away that puppy because of all the additional effort it would take to get any real business value.

Then what is the alternative?

The true alternative has to be a modern cloud data warehouse that is simple to manage, offers performance and is able to meet your concurrency needs, on demand. It has to be built for the cloud, to make sure you can bring in all your diverse data and effectively scale when you need to scale, rather than requiring you to overbook capacity upfront. Around the country, companies are begging for a solution that helps them better analyze data and speed their data pipeline, rather than being stuck managing the entire environment in yet another data center. Plus, it has to fit inside the ecosystem of modern ETL/ELT and BI partners that help with moving the data from source to analysts.

Want to hear how your peers are benefitting from the shift to cloud analytics? See how data driven companies, like Snag-a-job, are able to analyze 1.5 million impressions a day and, at the same time, 18 months of historical records to understand where their customers can improve their product. Come join us for the CloudAnalyticsCityTour.com when it visits DC (October 4th) and Philly (October 6th) to hear thought leaders from AWS, Looker, Informatica, Tableau, Snowflake, and the likes of David Menninger, about how legacy platforms and free puppies keep giving us more headaches. Or join us at the TDWI San Diego conference where IAC Publishing Labs (Ask.com, About.com Investopedia.com) will talk about how they won the TDWI Best Practice award for Emerging Technologies and Methods by using Snowflake, the data warehouse built for the cloud, to turn the BI group from a cost center to a valued part of the organization. You can also visit our booth at the TDWI exhibitors hall

As always, keep an eye on this blog site and our Snowflake-related Twitter feeds (@SnowflakeDB, @drkoalz, and @kentgraziano) for more “Cool Things About Snowflake”, and for updates on all the latest action and activities here at Snowflake Computing.