The Data Deluge | SUSE Communities

The Data Deluge


We keep hearing everywhere about the growth that the storage industry is seeing.  But where is this growth coming from?  Are the drivers of this growth still what they were a few years ago?  And what’s how’s the industry responding to that?  Over the course of next few weeks, I will examine the data growth and the corresponding storage landscape in a series of a blogs.  In today’s blog I am primarily going to focus on the source of the data and the value that brings to those who are able to harness its power.

In a recent forecast by IDC, by 2025 the total amount of data generated worldwide will exceed 163 ZB (163 followed by 21 zeros!).  That’s a 10x growth from the 2016 number of 16.1 ZB from last year.  Just as a comparison, that number is over 21,000 higher than the total grains of sand in the world.   And although only a part of it will be retained, most of it will be consumed and decisions will be made upon it in real time.

A lot of the data that is being created is for end user consumption.  Think of all the videos that are created and consumed on media and social media websites.  Because of changing consumer preferences, many of these videos are high resolution videos.  According to a study by Cisco[1] by 2021, 82% of all consumer internet traffic will be videos.  For businesses, video is one of the best advertisement vehicles with the highest end user engagement.  Video display advertisements generate the highest advertisement revenue for the websites that display them.

Among enterprises, video surveillance data is emerging as a fast growing category of data.  Video surveillance solutions are getting deployed among a variety of industries, including banking, retail, government and transportation.  Features such as facial recognition, video tracking, automotive number plate tracking etc. are the largest growth drivers.  The business value of video surveillance stems not just from safety and security but also loss and vandalism prevention and protection from lawsuits.

One of the largest non-video categories of data is the data from Internet of Things (IoT) devices.  IoT devices are connected devices that individually generate a small amount of data, but the aggregate amount of data for these devices can be huge.  These devices can range from the sensors on an airplane to the utility smart meters.   A single airplane like the Airbus A350 can have close to 6,000 sensors and generate 2.5TBs of data per day.  This data is used to monitor the aircraft’s health and reliability as well as for performing predictive maintenance.  Smart meters allow for data collection every second (or even multiple times per second).  This not only reduces utilities’ costs but also allows customers to reduce their own cost by providing them detailed usage information.  In fact smart meters are the first step in creating smart grids – where the grid will determine and balance the energy flow and increase its reliability.

These are just a few examples of where the data growth is coming from.  Others include mobile data, social data, geo-tracking data –  many which didn’t even exist a few years ago.  Various industries are being revolutionized by the flood of data that’s coming and the businesses that will harness the power of data will emerge as the winners in this economy.  In future blogs I will examine some of the frameworks that are used to capture and analyze the data; as well as the data lakes and storage retention solutions that are helping power this new data driven economy.


(Visited 1 times, 1 visits today)

Leave a Reply

Your email address will not be published.

No comments yet