The melting cube

The cube has never been a stable configuration in nature, looking at ice cubes or Sodium Chloride crystals. Sooner or later they melt down to nothing. If you intermix them they will even melt faster..

image image

Is the same thing happening with the OLAP-cube in BI?

The OLAP cube was an invention of the seventies, but not really used until mid-nineties. The demand from management of summary information with short response time made really at that time no other solution possible. You had to foresee  the needs and have most sums calculated in the night shift or on weekends, when the load on the server was moderate. With it came a new programming language – MDX, a rather tough piece to grasp. Still understood by few humans compared to SQL. Do we need MDX any more? Most problem for BI could be solved by both.

With the dramatic price reduction of memory, the increasing speed of hard disks not to mention SSD and the price fall of computing power (though thru parallelism today) has in the last years made a dramatic change. We now talk about “Big Data” and the possibilities to scan thru massive data and calculate what in most BI-applications is needed on demand with sub second response times.

imageSo is there a need to calculate in advance? In many common scenarios today where the cube is the standard, it is no longer needed. This is also the way the newer BI-tools work, QlikView, PowerPivot and ordinary Pivot-tables in Excel 2007/2010.

As Donald Farmer stated on a seminar a month ago, when the remaining 72% want their data for decision support, they will not pay the price of the expensive old-fashioned cube based solutions. They will buy a multicore server with some SSD:s and some dozens of GB of storage (around $10k). They load some millions of raw data into the data warehouse and build a hand full of dimension tables and they are done with the the implementation. While the user is clicking around in the tool, weather it is QlikView or Excel, the calculations are done on demand in sub seconds.

In some posts to follow I will try to give more insight in what I call “Not so BIG Data”.

Inspiration: Areal density, SSD for DW, Memory 1957-2012, OLAP Codd, Fast Track DW.