Do we really need “noSQL”? Yes of course. Creation of noSQL is the only logical step in development of applications and in evolution of databases. And in the future they will be used very wildly and will change dramatically the way we usually think about data. A traditional relational data models will be used only for some specific problems.

Of course vendors talk about this topic like noSQL approach already won. No exactly truth but this day is coming… Why? Let me tell you more about history. But please take following lines only as my humble opinion. This how I experienced “history of the databases”. Real “nerds” could have very different memories…

First databases were born “ages ago” out of necessity to store data as efficiently as possible. This is basically the main idea of “data normalization” – to eliminate duplicities in data. This way classical databases saved disk space (which was the big issue in those days) and simplified statistic computation. Because predefined list of values made easier to categorize data. I am telling “classical” because these first old databases were not “relational”. This feature has come a bit later…

I started to develop database applications with first already relational databases. It was on old 8bit and 16bit PCs (do you remember PC XT, PC AT? ?) using dBASE and ForPro databases. In those days computers used disks with capacity like 85MB (this is not a mistake – really Megabytes) and diskettes have max 1,44 MB. There was even limit 640 KB for computer memory in MS-DOS – do you remember?

As I said these old databases were already relational but did not use transactions – they did not need them. Those databases were intended only for one computer and one user. Because only a very few users knew in those days what does it mean when you say “network”…

All rows in tables had fixed length and update meant that database engine wrote data always into same places. In those “ancient times” the speed of response was not exactly the highest goal. And also “historical data” were usually not stored on the same machine also because of not enough disk space. In old database systems application usually produced some summarizing reports and historical data have been “archived”  and deleted from the database. “Archiving” was usually just another word for throwing old data into trash.

But this all changed drastically with first networks. (Do you remember Novell-network or first Windows network in “legendary” Windows 3.11 ?)

Companies suddenly wanted to use their existing databases globally in the whole network. This has brought new challenge – how to solve conflicts on the same data. Well – first approach was very simple – exclusive locks. Record edited by some user was locked and no other user could ever read it. Of course this solution did not last for ever and developers had to use new databases with transactions. And transactions allowed to read changed records with different isolation levels…

Also during following decades databases had to face many new challenges and the biggest one of them was still higher and higher demand on speed. I have seen this development working with Oracle – from version 7 up to version 11. And also with “ancient” MySQL 4 and with first versions of SQL-Server… At 1990s run times in dozens of minutes where no problem. In year 2012 the same operation was required to be completed in only a few seconds or miliseconds.

Of course internet has brought even new level of problems. In the times of small local networks only a few users could work simultaneously on the same data. But internet helped to interconnect huge amount of these small networks. So in very big companies suddenly dozens even hundreds of users could use the same database. It caused very high traffic and also very big amount of queries and data changes during busy periods of the day…

I think you know it for yourself – there are always some limits. And the biggest problem are usually limits in investments. So you have always some application which has to run on older version of database on older hardware or with not enough attention from administrators. Simply because there are not enough money to upgrade to the newest version, newest hardware or to hire more people…

This is the main reason why database and application developers have to be creative to answer these challenges. Very soon they realized that highly normalized data model is not exactly the best solution for complicated analytical queries. And therefore clever people created the idea of data warehouses which use denormalized data model. And this was – at least from my perspective – the first really important step towards today’s noSQL databases.