“Where’s the Big Data?”

Chris Lemay from Ventyx, an ABB company, provides some additional input to the comments I posted in August regarding “Big Data.” He touches on the three “Vs” of Big Data: velocity, volume, and variety from an electric utility viewpoint. Some of the industry experts extend the discussion to four “Vs” or even five “Vs” by including the Variability of the data which is the inherent fuzziness of the data in terms of context and meaning. The fifth “V” is Value which is quite important since Big Data becomes an academic exercise if no value is created.

----------------------

In his August post, Gary pointed to the growing trend of utilities investing in Big Data. It’s probably healthy, however, to have a dose of skepticism around all the hype. After all, even 10 million of today’s smart-meters will take a decade to generate over a petabyte of data. Looked at objectively, the sheer volume of data generated by the smart grid is dwarfed by what financial and retail market players experience. That’s where the other aspects of “Big Data” come in to play: velocity and variety.

If you’re familiar with utility control room operations, you already know about data velocity. The electric grid is real-time; supply and demand need to be kept in balance at all times. Traditionally, we’ve managed with a limited amount of SCADA and a healthy contingency margin on supply. However, the intermittency of renewables and moves to shift peak consumption are driving a need for smarter management of the end-to-end grid. Better control systems are needed to manage a greater variety of supply sources, including distributed generation. In order to make more optimal use of the available capital resources, we also need more accurate and more granular predictions of demand, so that supply and demand can be managed together. Although the volume of data exchanged between the various devices on a modern grid may be modest by “Big Data” standards, the requirements for speed and accuracy of analysis are very demanding. 

Utilities are also very familiar with data variety. This is especially true if you wander out of the control room and into the field. The data utilities have about their assets is so varied and scattered that gathering it all together for a complete picture of the health of each asset is a daunting task. The first problem is that most utilities have many silos of information. One example is that information collected by operations about assets isn’t usually available in the maintenance department and vice-versa. Through consolidation, many US utilities also have geographical or organizational silos of information that make it difficult to get a consistent view of asset performance in different parts of the enterprise. Another source of data variety is a by-product of the fact that most grid assets have a long lifetime relative to the IT assets collecting and storing the data today; it is likely that there is much less data available on assets commissioned 30 years ago than those installed in the last decade. Furthermore, as sensors on assets and in the grid are added or upgraded, they produce a richer variety of information about these long-lived assets. Utilities need IT systems that are flexible enough to handle these changing sources of data, and are also extensible so that they can also handle less structured data such as observations recorded by technicians in the field, and even images taken of assets over their lifetime.

Writing in the October 2012 issue of the Harvard Business Review, Andrew McAfee and Erik Brynjolfsson state that they are “convinced that almost no sphere of business activity will remain untouched by this movement.” Although the “Big Data” needs of utilities are somewhat different than those of other industries, I believe it would be naïve to think that the increases in data volumes, velocity and variety will not transform their business practices in a significant way. Putting in place the new technology and adopting the new processes to take advantage of this revolution in data acquisition and processing is just one more component of the smarter grid.
---------------