Summary
While most application frameworks are in some ways suited to certain types of application domains, such as Web applications or Web services, almost all such frameworks still access the same type of relational databases. In a recent blog post, Mike Stonebraker argues that one-size-fits all databases are an idea whose time has passed.
Advertisement
Although relational database technology is nearly thirty years old, no other data management concept could supplant its popularity up to the present. Indeed, all the major enterprise RDMBs products in use today are based on what is essentially the same fundamental technology.
In his inaugural post One Size Fits All - A Concept Whose Time Has Come and Gone on The Database Column, a new group blog of some of the database community's most prolific luminaries, Mike Stonebraker, one of the inventors and first implementors of the relational database, notes that:
In short, the world of 2007 is radically different from the world of the late 1970s. However, none of the major vendors have performed a complete redesign to deal with this changed landscape. As such they should be considered legacy technology, more than a quarter of century in age and "long in the tooth".
Stonebraker especially takes issue with the one-size-fits-all approach to databases: While enterprise developers might choose an application framework suited to the specific problem domain they wish to solve, they almost always default to using one of the major relational database products. Just as application frameworks or developer tools specialize in certain types of problem domains, databases should also be considered on the bases on their suitability for the kind of task at hand, argues Stonebraker:
In every major application area I can think of, it is possible to build a SQL DBMS engine with vertical market-specific internals that outperforms the "one size fits all" engines by a factor of 50 or so. Vertica is an example of this claim in the data warehouse market. It achieves blindingly fast warehouse performance by:
Column orientation - i.e. rotate your thinking 90 degrees
Aggressive data compression
Query executor runs against compressed data
My prediction is that column stores will take over the warehouse market over time, completely displacing row stores. Since many warehouse users are in considerable pain (can't load in the available load window, can't support ad-hoc queries, can't get better performance without a "fork-lift" upgrade), I expect this transition to column stores will occur fairly quickly, as customers search for better ways to improve performance.
In the longer term, I expect a transition of the same sort to occur in other markets where there is great user pain and the possibility of radical performance improvement from a specialized software architecture.
Stonebraker argues that such domain-specific databases could take better advantage of new hardware, and possibly even software, frameworks than their general-purpose brethren.
Do you agree with Stonebraker that some enterprise problems can be better served with specialized database products? Do you agree that the time of the one-size-fits-all database has passed?
Mike Stonebraker has certainly thrown out interesting food for thought. As with any prediction, only time will tell. However, if his numbers are accurate with respect to performance improvements I would certainly like more detail on specialized databases. Some points come to mind.
1. When he says "Column orientation - i.e. rotate your thinking 90 degrees", is he talking about query developers having to rotate their thinking or developers of DBMS's? In other words, is the column orientation not going to affect the expression of queries as we write them now?
2. If query expression changes, then it seems like necessarily the industry will develop more specialities in database programming. Currently, writing analytic queries is about the only specialty I can think of and that's really a matter of learning extensions to the basic SQL language.
3. OS's and languages have already seen the coming of specialization. It would not surprise me if database software likewise sees it.
4. Newer and better technology sometimes doesn't get adopted due to an "ecosystem" around existing technologies. Variables such as finding trained individuals, having existing applications written on top of, language and tool support, etc. make predicting adoption difficult.
With regards the user needs to adjust their thinking or not, it is my understanding that the user doesn't need to adjust. The data storage layers will store your data in a way that is "column-oriented" (and more importantly compressed) and the query execution layer will scan through the columns and construct result sets out of those columns. Hope that helps.
> If you are truly interested in the topic he has a paper > located here: > http://web.mit.edu/dna/www/vldb.pdf > > With regards the user needs to adjust their thinking or > not, it is my understanding that the user doesn't need to > adjust. The data storage layers will store your data in a > way that is "column-oriented" (and more importantly > compressed) and the query execution layer will scan > through the columns and construct result sets out of those > columns. Hope that helps.
Excellent. Thank you. I think this topic was an article in MIT's Tech Review a few months ago.