The Fall and Rise of Embedded Database Technologies

October 1, 2007

The Fall and Rise of Embedded Database Technologies

Some recent work I have been doing has rekindled an old love affair I had with embedded databases.  Ok, I admit “love affair” to be a bit over the top but there is something intrinsically satisfying with a database that just sits there and does stuff, with no messing about and no complicated database administration jobs.


I guess the most interesting aspects of embedded databases are the engineering brilliance that enables a whole load of database smarts to be crammed into a virtual black box. I had the same level of respect when Microsoft managed to squeeze SQL Server onto mobile devices a while back – how on earth did they do it? 


I guess the answer is by stripping out a whole heap of functionality that is unnecessary on a mobile device and then cramming the code base into as small an executable as possible. One could suggest that Microsoft had lost the art of writing small, effective executables a long time ago but that would be mean.


Whilst the Microsoft approach of downsizing SQL Server is laudable, it is hard to believe that such a product could be as good as a database designed from the ground up for the purpose. There have to be some design compromises and unavoidable penalties from straight-forward downsizing.


In most cases embedded products appear to deliver just what developers want from an application database – basically a product that just sits there processing and managing the data and making no fuss. An old colleague of mine, who didn’t share my passion for the subject, accused all databases of being giant bit buckets. Well, I guess in the case of an embedded product that is about right if you are talking extremes – but bit buckets are very useful.


The performance of embedded databases can often outshine that of relational databases as they can be tuned to fit the needs of the solution and circumvent the requirement for optimization. You lose the elegance of a fully normalized relational database, but who cares if I have repeating datasets when I can process a gazillion records in 2 seconds?


Many moons ago I was involved with the FoxPro PC database product. At the time it was seen as a competitor to Dbase IV (remember Dbase?) and it was snapped up by Microsoft for the sake of direct competition. But Microsoft never really “got” FoxPro, eventually preferring the more [superficially] elegant and relational-esque  Access product. An aspect of the FoxPro database that was truly amazing was the Rushmore optimized query processing engine. This used a bit mapped index technology and was able to process this gazillion record set in less than 2 seconds. Properly relational? Heck no, but it sure did perform. I have not worked with the product for many years, but I hope that it retains its speed and performance.


Does this mean that relational databases are passé and we should all be moving to embedded databases? Of course not, that would be ridiculous. What I am suggesting is that there maybe quite a few line-of-business applications out there that would be quite happy processing data in an embedded database without the rigmarole of a fully fledged relational database back-end. If you put enough thought into the design, you can build fast and robust systems with this approach. Next time you are looking at delivering another business solution give it some thought.

Newsletters 2007
About admin

Leave a Reply

Your email address will not be published.