(c) Copyright 2000 by Fabian Pascal

All rights reserved

WHAT'S WRONG WITH THIS (DATABASE) PICTURE?

(OR WHATEVER IS LEFT OF IT)

by Fabian Pascal

"Well, it's really a judgment call and I think a lot of experience comes into it. It's a little bit like building a shack. Say you want to build a skyscraper, and you started out building a shack and you just keep trying to add onto it. After a while you have this severe structural problem ... So there is a fallacy to the build-upon-a-simple structure approach. Sometimes you get up to three stories and you have to do some major structural changes, and I just accept that."

Wayne Ratliffe, developer of dBase.

"Client Servers were a tremendous mistake. And we are sorry that we sold it to you. Instead of applications running on the desktop and data sitting on the server, everything will be Internet based. The only things running on the desktop will be a browser and a word processor. What people want is simple, inexpensive hardware that functions as a window on to the Net. The PC was ludicrously complex with stacks of manuals, helplines and IT support needed to make it function. Client server was supposed to alleviate this problem, but it was a step in the wrong direction. We are paying through the nose to be ignorant".

Larry Ellison, CEO, Oracle Corp.

 

On Fashion and Cookbooks

The computer industry-- its database sector in particular -- is much like the fashion industry: it is driven by fads. And more often than not it profits from the accelerated obsolescence on which fads are predicated. DBMS vendors (Oracle's CEO in particular) were "wrong" more than once before. But it's the users, not the vendors, who paid through the nose, because the industry, with help from the trade media, hype and obscure the lack of soundness and ensuing serious deficiencies of "new" fads, which frequently are nothing but old concepts we already discarded with different labels.. The Internet and the browser are just the latest fad in a long series and are as much a panacea for information management as the PC, SQL, client/server, object orientation, "universal" and multidimensional DBMSs, data warehouses mining were before them, which were not preached with equal fervor.

The fact is that sound database technology and practices are prerequisites for effective and efficient information management, whether Internet-based or not. Sadly, however, the database field is in disarray, with Internet practices being not lesser, but actually worse offenders than its predecessor panaceas. While this is, to a degree, true of computing in general, in the database field the problems are so acute that, claims to the contrary notwithstanding knowledge and, therefore, technology are actually regressing!

Even a cursory inspection of problems encountered in real world database practice reveals that most are due to the persistent failure by both DBMS vendors and users, including DBAs, application developers, and managers, to educate themselves and rely on a sound foundation in their practices. Indeed, it is lack of proper education that makes fads and accelerating obsolescence possible in the first place! As Chris Date explains in the Foreword to my book:

"SQL [DBMS] deficiencies are, it seems to me, directly due to the widespread lack of understanding (not least on the part of vendors), of fundamental database principles. Certainly it is undeniable that they flout those principles in numerous way. And the practical consequences are all too obvious: First, users must understand where the deficiencies lie; second, they have to understand just why they are deficiencies; third, they have to understand how to work around them; and fourth, they have to devote time and effort in persuading the vendors to remedy them. The trouble is, of course, users too tend to be unaware of those same fundamental principles and, hence, find themselves unable to carry out their side of the "contract" (a "contract" that should not have been allowed, or agreed to in the first place, of course). It's a vicious cycle. What is more, this sad state of affairs is not likely to change, given the apparent lack of interest on the part of the trade press--itself ignorant of those same principles--in trying to improve matters."

Consider, for example, the following two cases, one raised by a novice

The other is raised by a consultant assessing a database constructed, supposedly, by experienced professionals.

They are not only indicative of how database work is approached these days and with what results, but quite representative. What should be obvious is that

Yet it is practically impossible to get the attention of database practitioners for anything other than product-specific recipes, essentially a cookbook approach. Examples:

Education vs. Training

The fad-driven, tool-focused, cookbook approach to data management is due in large part to the business culture in general, and the way in which information technology professionals are inducted into the field in particular. A vast majority are self-taught and start their database involvement via work with some specific DBMS software (e.g. Oracle, Access, SQL Server) and tools (frequently imposed on them by their employer. Having not been exposed to general database concepts, principles and methods, practitioners are either unaware of the field's fundamentals, assume that they are acquired implicitly by learning or working with the software or, most commonly, deem them "theory" and, therefore, without a practical value. These fallacious perceptions are exacerbated by a growing generation of Internet practitioners who know little beyond HTML, Java and XML (not even DBMS or tool software), and who, therefore, think that's all there is to know.

But we should not expect anything different. The sole technical qualification for practically all database positions is experience with some DBMS software and development tools (mainly programming) on specific platforms (hardware and operating systems). Nothing else. Examples:

Not only isn't foundational knowledge -- as distinct from sheer experience with tools -- a job requirement, but more often than not it is actually a liability. Functions such as requirements analysis and database logical design are bundled together with database administration, application development and physical implementation and assigned to the (mythical) position of "programmer/analyst", without realizing that they require fundamentally different skill and knowledge sets which are rarely found in one person, because they inherently interfere with one another (particularly with currently flawed DBMS products). If you wanted to build a house, would you hire a building contractor to design it?

In fact, under industry pressure there is little database education to be had. Product-specific training reigns supreme and even academic computer science programs are becoming increasingly vocational in character.

Examples:

An analogy can serve to drive the perils of this state of affairs home. Suppose you must select a personal physician and have two candidates: one educated in, among other things, some anatomy, biology and chemistry, and one trained in a "cookbook" approach: identifying symptoms from a list and matching treatments from another. Chances are you will opt with the majority for the former rather than the latter, and for a very good reason: in the absence of knowledge and understanding of some health fundamentals, serious problems can be expected. This is generally clear in all applied fields with a scientific foundation except, it seems, database management.

Is there any wonder that practitioners, seasoned ones included, can't offer a useful definition of a database? That neither DBMS designers, nor technically proficient users have heard of crucial concepts such as data independence? That many believe that not only should duplicates not be prohibited, but that they are actually essential?

The consequences are visible all over the business -- and Internet -- world and are horrendous. A vast majority of databases and practically all DBMS products and are riddled with flaws and unnecessary complications. Examples:

"You might ask what is wrong? Well, it is a client/server application, using a Sybase database (SQL anywhere). The database server has a single login user DBA - using the default password. Every application user connects to the database via this login level, and security is handled by the front end

- despite the fact that any semi-aware user could use MS Access to destroy any data.

There are also about 300 tables in the database, with no indexes! Agreed there are primary key indexes created automatically by the database, but still... The front end is Visual Basic, which for me is OK, but there are at least three different data access methodologies, from ODBC-API to the latest ADO.

But what is killing me, is that I seem to have been hired as a "bug-fixer", to me different than an engineer. They are in a position where release schedules are forcing a continual maintenance mode, rather than an admittedly necessary rebuild of some components."

"In the short term you have two options a) disable referential integrity checking and make the change (not recommended unless you're willing to assume total responsibility for the data consistency checking yourself; and you have to ensure you have exclusive access to the database when you're doing this) b) use our [DBMS's] triggers and stored procedures to implement the referential integrity procedurally."

A Vicious Cycle

Correcting this sad state of affairs is a nontrivial proposition, because it is a deep seated, vicious cycle that is cultural and systemic and, therefore, extremely difficult to break. It is much easier (and profitable) to go with the flow, rather than uphill against it, so the vast majority of trade magazines, books, web sites, conferences, and education programs ignore fundamentals, rely exclusively on vendor sources and focus completely on product-specific "recipes", reinforcing, rather than combating the cookbook approach.

DBMS and tool vendors, database professionals and users, desire accurate answers from databases. Yet the vast majority are unaware that, as Hugh Darwen states

The proof, of course, can only be as sound and consistent as the rules are. That makes the DBMS a deductive logic system: it derives new facts (query results) from a set of user asserted facts (the database). The derived facts are true (query results are correct) if and only if

Neither are most practitioners explicitly aware that the truth of the initial assertions (the correctness of the database) must be ensured by the DBMS's integrity function, and the correctness of the derivations (query executions) by its manipulation function, only if the design of both databases and DBMSs adheres to the sound and fundamental principles of logic.

Because they have been socialized into, and rewarded for discounting fundamental principles as "theory" without practical value, practitioners are largely unaware that the tools they employ and the practices they induce fail at these functions and, therefore, cannot and do not guarantee the accuracy and efficiency they desire. The result, as I have amply demonstrated and documented in my writings, is that a lot of what is being said, written, and especially done in the database management field -- or whatever is left of it -- is increasingly confused, irrelevant, misleading, or outright wrong.

Fabian Pascal has a national and international reputation as an independent database industry analyst and critic, consultant, author and lecturer. For more than 13 years he held various analytical, management and analytical positions in the private and public sectors, was affiliated with Codd & Date, has taught and lectured at the business and academic levels, and advised vendor and user organizations on database technology and implementation. Clients include IBM, Census Bureau, CIA, Apple, Borland, Cognos, UCSF, IRS.

He is co-founder and editor of DATABASE DEBUNKING (www.firstsql.com/dbdebunk), a web site dedicated to dispelling prevailing fallacies and misconceptions in the database industry, to be launched in May, with C.J. Date a senior contributor. He has contributed extensively to most trade publications, including Database Programming and Design, DBMS, DataBased Advisor, Byte, Infoworld and Computerworld.

His third book, PRACTICAL ISSUES IN DATABASE MANAGEMENT - A GUIDE FOR THE THINKING PRACTITIONER (http://cseng.aw.com/book/0,3828,0201485559,00.html) is forthcoming in June from Addison Wesley, and serves as text for a seminar bearing the same name.

He can be reached at

 

Questions about the NAIUA? E-mail .
Questions and Suggestions about these web pages? E-mail .

Last updated: January 10, 2000