Not so Big data

For mortals like me, “Big data” is bigger than I am used to, which also seems to be the accepted definition of this loosely defined term almost everybody seems to use, with diverse definition which always is the case with buzz words. For most of us “big” is GB and above, at least when handling ordinary business problems. What does then these new technologies mean to us mortals?
As an engineer I have always been fascinated of breaking barriers, which is what happens with the new technologies around “Big data”. What does they mean to me in my job with my customers? Should or must I adapt? “Big data” would never have occurred if not the fantastic progress in “areal density” had occurred during the last decades. So for me, data is not so much bigger, but calls for a big change.

Big changes are always pushed on us, if we like them or not, this goes for bulbs or source separation of garbage. Most of us are lazy, change means action. We like to stroll around the same way we’ve always done, it is much more comfortable.
In the BI-area where I earn my living, the new technology means more or less a revolution, even if most people involved still not seams to have realized that. In most common sizes of DW the need for summarizing data to produce acceptable response times have vanished. This means that central technology in this area like cubes and OLAP no more is needed, and the need of consultants keen on MDX will disappear in the next five years to come. Traditional actors like Cognos(IBM) and  Business Objects(SAP) have to react to rookies like QlickView, Tableau and PowerPivot which to my understanding do well without using cubes and MDX. They simply store most data in RAM and calculates what is asked for on demand. This is in my mind making full use of new technology, based on the big changes in prices of storage and multiprocessor capacity the last decade . This shift also makes developing new applications in the BI-area so much cheaper. The new actors all focus on visualisation and self-service in their marketing, which is made possible with technological leadership.

I guess you also drive Audi Quattro like me: “Vorsprung durch Technik”….