Systems Engineering and RDBMS

Archive for January 29th, 2007

Name Value Pair Design (EAV)

Posted by decipherinfosys on January 29, 2007

This is one of the design anomalies in data modeling.  Infact, this even has a name for it: EAV (Entity Attribute Value).  Database Guru Joe Celko has written a lot about it in newsgroups as well as in his presentations.  The reason why I am blogging about this today is because I have seen this approach time and again and it is either done by novice DBAs/database developers or by application programmers who try to use their programming context in the relational world.  Here are the disadvantages of this approach :

1)  It is next to impossible to enforce business rule constraints like uniqueness, enforcing check constraints per attribute, adding defaults per attribute, enforcing PK/FK relationships based on attributes etc.  Since the attributes are being modeled as data elements, it is very error prone since we have now opened up our attribute design to the users.  All data integrity is destroyed (violation of 1NF itself).  Any typo becomes a new attribute.

2)  Different data-type attributes will be modeled using the string data-type for the value column and this will result into performance issues while doing comparisons since the index sorts will be an issue and not only that, an effective index design startegy cannot be formulated for this.  This also increases the index size un-necessarily.  In addition, it will require un-necessary cast operations like for the date/timestamp data-types, the numeric data-types with precision and a non-zero scale etc.

3)  It would lead to the usage of cursor based (and dynamic SQL) approach for reads since the number of attributes is virtually un-known.  In the case of complex reports or other read operations, this will be a nightmare…un-necessary CASE statements, un-necessary sub-queries with self references and un-necessary pivoting/un-pivoting of data would ensure that the reports are very slow…as your data grows, performance will get exponentially worse.

If you do have a situation where you are dealing with an entity that has thousands of attributes (I have seen requirements like these) and all of them are static i.e. once they are done and have data populated in them, they never change.  In those scenarios, in order to not exceed the rowlength (like SQL 2000 has a limitation on the rowsize being 8060), one can either chose to create a hard 1:1 relationship tables or chose this design to accomodate for such requirements.  Since the list of attributes is static and the data in them is static as well, a simple pivot view can be used to flatten it out.  But that is the only scenario where this approach can be used/considered.

If one really needs to model a system where the attributes can be generated by the end user (which itself is questionable but can happen in certain verticles – especially those where the end user needs to use the software for planning and forecasting), then a better approach will be to use a static list of attributes (for the different data-types) and have another table for their name look up so that storage is done in a relational manner but for the display piece, one can use the static list and can call an attribute whatever they want.  This is the most reasonable compromise for data integrity, performance and sufficing the need of the application.  We will be writing up a “How-To” article or a whitepaper on this taking an example from the real world and demostrating how you can solve this issue in a reasonable fashion using the above approach rather than using an EAV design.

Posted in Data Model | 2 Comments »

Could not obtain exclusive lock on database ‘Model’

Posted by decipherinfosys on January 29, 2007

How many times have you faced this error during the installation and wondered what the heck is SQL Server complaining about.  All that I want is to create my database and move on.  The error that I am talking about is shown below:

model.JPG

In SQL Server,  whenever a new user database is created, it takes it’s properties from the “model” database (this system database acts as a template for new user databases) and if there are over-rides to some of the parameters for the new user database, those are overridden based on what is specified in the create database script.  SQL Server attempts to obtain an EXCLUSIVE lock on the model database during this process.  This is to prevent other processes from updating the model database’s schema whilst the new database is being created, so that the new database’s schema is in a known, consistent state.

There are four reasons why you can get this error:

1)  It can happen if someone is executing something against the MODEL database via Enterprise Manager/Query Analyzer/Management Studio (SQL 2005) or some other tool i.e. if there is an active session working against the “MODEL” database when the create database command is being executed, you will get this error (this is very simple to prove also..in QA, open up a window and select model database from the drop down :

USE MODEL

GO

And from a second session, try running “CREATE DATABASE TEST”…this will fail to obtain exclusive access to the “model” database and will give the 1807 error.

In order to circumvent the problem from happening via your install scripts, you need to check and make sure of that no sessions are open for the model database…if there are, get their spid and kill ‘em before running the “CREATE DATABASE command”.  Also, if 2 “create database” commands are running in parallel and one takes a long time to finish, the other one would time out with this error.

If it is really because someone else is accessing the model database at this stage, you can re-start the service and re-try since that will release the connection to the model database (or check the existing connections using sp_who2 and kill the one that connects to the model database).  If it is an issue with the installation code, then that would need to be fixed.

2)  It can also occur if the “AUTOCLOSE” property is set to ON for the “model” database as a result of which when a new user database is being created, if the model database is closed at that time (when this property is on, the database is shutdown cleanly when it is not in use), you can get this error because the new database takes it’s default properties from the model database and it is close at that time as a result of a log-off from some other session.

NOTE:  AutoClose should always be OFF for all databases.  Setting this to ON is strictly against the recommendations.

If it is occuring because of this reason, correct the setting and re-start the service.

3)  It can also happen if the machine name has been changed (you need to follow proper steps to re-register the new name) but if this is the cause, then the error will happen all the time till that issue is corrected.  select @@servername will tell you the name of the local machine for point #3.

4)  Another reason is that if you have previously opened the model database in Enterprise Manager/Management Studio (SQL 2005), then closed it, the connection to the database remains open, which means that the Create Database command cannot acquire the exclusive access lock that it requires to execute successfully.  For the AutoClose:

SET NOCOUNT ON
go
SELECT DATABASEPROPERTYEX( ‘model’ , ‘IsAutoClose’ ) AS [AutoClose]
go
ALTER DATABASE [model] SET AUTO_CLOSE OFF
go
For the connections:

select spid, hostname, program_name
from master..sysprocesses
where db_name(dbid) = ‘model’
go

Posted in SQL Server | Leave a Comment »

Auditing capabilities in Oracle

Posted by decipherinfosys on January 29, 2007

There are many different ways to audit the data, the users, the DML/DDL, the “select” statements within Oracle.  Here are the options that you have at your disposal (this does not talk about the third party tools that are out there but only the OEM functionality that is available):

  •  You can write auditing triggers for the objects in the schema.  This is done for the DML statements.  This is one of the most common auditing techniques that are used and can help in building up an audit trail as well as help in troubleshooting specific application issues like the quantity for a particular product going negative in the schema because of an application bug.
  • You can use LogMiner utility which helps in the auditing of both the DML statements as well as the DDL statements.
  • You can write up system level triggers – this can be used for auditing both the DML as well as the DDL statements and is a very powerful auditing technique for auditing changes to your system.
  • You can also chose to use the flashback feature of Oracle to go back in time and do DML statements auditing.
  • You can use FGA (Fine Grained Auditing).  Using this feature, you can audit the DML statements as well as plain simple read (select) statements in Oracle 10g.  In 9i, you can audit only the select statements.

The object level triggers, LogMiner and flashback are the most commonly used techniques.  LogMiner approach ensures that all changes are recorded and kept for auditing purposes.   FGA approach has the additional benefit of recording the select statement that was issued by the application.   It does have it’s advantages over the trigger approach if you have a highly transactional system.  Since the trigger is per row, if the DML is modifying say 1 million records, it will create a lot of overhead while auditing.  Using a statement level trigger will not help in this case since you cannot capture the before and after values for the individual records using that type of trigger.  In the FGA approach, only one record gets created in the audit log since it executes once per statement, not once per row and thus the effect on performance is negligible.  For extracting the before and after values, flashback queries can be used (size your UNDO properly for it).

Obviously, one single solution does not fit everyone’s needs which explains why Oracle has provided so many different options.  You can pick whichever suits your need depending upon the type of application that you have and your specific needs.

Posted in Oracle | 1 Comment »

Installing FTP on Red Hat Linux AS 3.0

Posted by decipherinfosys on January 29, 2007

The RPM files referenced in the document can be found in the following location on the installation media: the sub-directories containing the RPMS have the following Format: “CDn/RedHatRPMS”

To set up an FTP server, make sure the vsftpd package is installed by issuing the following command as root:
rpm -q vsftpd

You should see:
vsftpd-1.2.0-4

If not installed, then use the following directions.

1)    The vsftp package can be obtained on your distribution media, or through your system appropriate channel of the Red Hat Network

vsftpd-1.2.0-4.i386.rpm  (CD1/RedHat/RPMS)

2)    If the package is not installed, insert the installation CD-ROM containing the appropriate package, change to the /mnt/cdrom/RedHat/RPMS/ directory, and, as root, type the following command:

rpm -i vsftpd-1.2.0-4.i386.rpm

3)    Check for installation:

rpm -qa | grep vsftp               (alternatively rpm -i vsftp)

You should see:
vsftpd-1.2.0-4

4)    Once installed, start the vsftpd service by issuing the following command:

service vsftpd start

5)    To make the service available for the next boot sequence chkconfig the service on by issuing the following command:

chkconfig vsftpd on

Additional information can be found in the vsftpd man page:
man vsftpd

Posted in Linux | Leave a Comment »

Installing telnet on Red-Hat Linux AS 3.0

Posted by decipherinfosys on January 29, 2007

The RPM files referenced in this blog post can be found in the following location on the installation media.: the sub-directories containing the RPMS have the following Format: “CDn/RedHatRPMS”

1) Both telnet-server-0.17-26 and telnet-0.17-26 need to be installed

Check this by executing: rpm -qa | grep telnet

2) If missing, then you will need to get the required packages from the installation media, or a compatible source, and install them.
telnet-0.17-26.i386.rpm (CD2/RedHat/RPMS)
telnet-server-0.17-26.i386.rpm (CD3/RedHat/RPMS)

3) Once the installation packages are on the machine (*.rpm files), you can install them in the following manner. (It is common to
place these rpm files in the /tmp or /temp directory).

rpm -i <rpm file>

4) Execute the following command to ensure the packages are installed:

rpm -qa | grep telnet

You should see the following: telnet-server-0.17-26
telnet-0.17-26

5) Make sure the installation was done properly:

chkconfig –list telnet

You should see:
telnet off

6) You now will need to change the telnet configuration to on

chkconfig telnet on

7) Check the configuration:

chkconfig –list telnet

You should see:
telnet on

8 To start the service, you will need to restart xinetd:

service xinetd restart

You should see:

Stopping xinetd: [ OK ]
Starting xinetd:  [ OK ]

Posted in Linux | Leave a Comment »