How to Optimize Indexes in SQL Server

SQL Server automatically maintains indexes whenever insert, update, or delete operations are made to the underlying data. These modifications cause the information within the index to become scattered in the database. Fragmentation exists when indexes within the index to become scattered in the database.

Fragmentation exists when indexes have pages where the logical ordering does not match the physical ordering within the data file. Heavily fragmented indexes can degrade the query performance and cause your application to respond slowly.

Fragmentation normally occurs when a large number of insert and update operations are performed on the table. Fragmented data can cause the SQL Server to perform unnecessary data reads. This affects the performance of the query. Therefore, index defragmentation is necessary to speed up the query performance.

The first step in deciding which defragmentation method to use is to determine the degree of index fragmentation. You can detect index fragmentation by using the sys.dm_db_index_physical_stats system function.

The following statement displays a list of all the indexes on the HumanResources.Employee table with their fragmentation level:

SELCET a.index_id AS IndexID, name AS IndexName,
Avg_fragmentation_in_percent AS Fragmentation
FROM sys.dm_db_index_physical_stats (DB_ID (N’AdventureWorks’),
OBJECT_ID (‘HumanREsources.Employee’), NULL, NULL, NULL ) AS a
JOIN sys.indexes AS b ON  a.object_id = b.object_id AND a.index_id = b.index_id ORDER BY Fragmentation desc

In the preceding output given by this query, you can notice that the AK_Employee_LoginID index shows a high level of fragmentation.

After the degree of fragmentation is known, the fragmentation needs to be corrected. The following table lists the actions to be taken at different fragmentation levels to defragment the index.

Fragmentation Level                         Action to be Taken
>5% and <=30%  ALTER INDEX REORGANIZE
>30%  ALTER INDEX REBUILD WITH (ONLINE – ON)

Method of Degragmentation

You can execute the following command to defragment the AK_Employee_LoginID index:

ALTER INDEX AK_Employee_LoginID ON HumanResources.Employee REBUILD
After executing the preceding command, the degree of fragmentation is reduced.

Managing indexes with pre-defined Query: SQL Server

In addition to creating indexes in sql server, database developer also need to maintain them to ensure their continued optimal performance. The common index maintenance tasks include disabling, enabling, renaming, and dropping an index. As a database developer, you need to regularly monitor the performance of the index and optimize it.

Disabling Indexes

When an index is disabled, the user is not able to access the index. If a clustered index is disabled then the table data is not accessible to the user. However, the data still remains in the table, but is unavailable for Data Modification Language (DML) operations until the index is dropped or rebuilt.

To rebuild and enable a disabled index, use the ALTER INDEX REBUILD statement or the CREATE INDEX WITH DROP_EXISTING statement.

The following query disables a non-clustered index, IX_EmployeeID, on the Employee table.

ALTER INDEX IX_EmployeeID
ON Employee DISABLE

Enabling Indexes

After an index is disabled, it remains in the disabled state until it is rebuilt or dropped. You can enable a disabled index by rebuilding it through one of the following methods:

• Using the ALTER INDEX statement with the REBUILD clause
• Using the CREATE INDEX statement with the DROP_EXISTING clause
• Using the DBCC DBREINDEX

By using one of the preceding statements, the index is rebuilt and the index status is set to enable. You can rebuild a disabled clustered index, when the ONLINE option is set to ON.

Renaming Indexes

You can rename the current index with the help of the sp_rename system stored procedure.
The following statement renames the IX_JobCandidate_EmployeeID index on the JobCandidate table to IX_EmployeeID.

EXEC sp_rename
‘HumanResources.JobCandidate.IX_JobCandidate_EmployeeID’,
‘IX_EmployeeID’, ‘index’

Dropping Indexes

When you no longer need an index, you can remove it from a database. You cannot drop an index used by either a PRIMARY KEY or UNIQUE constraint, except by dropping the constraint.

The following statement drops the IDX_Employee_ManagerID index on the Employee table:

DROP INDEX IDX_Employee_ManagerID
ON Employee

How to Create Partition Scheme and Clustered Index: SQL Server

After creating the partition function, programmer needs to create a partition scheme to associate it with the partition function. Based on the boundary values defined in the partition function, there will be five partitions. The data of each partition is stored in a filegroup. You should have the same number of filegroups as partitions. If there are five partitions, you need to create five filegroups: fg1, fg2, fg3, fg4, fg5.

The following statements create the PSOrderDate partition scheme, associating it with the PFOrderDate partition function:

CREATE PARTITION SCHEME PSOrderDate
AS PARTITION PFOrderDate
TO (fg1, fg2, fg3, fg4, fg5)

The partition scheme, PSOrderDate, created in the preceding statement directs each partition to a separate filegroup.

Creating a Clustered Index

After creating the partition scheme, you need to associate it with a clustered index. As the clustered index is created on the attribute having unique and non-null values, you can create the index on the SalesOrderID column of the SalesOrderHeader table. To create the partitioned index, you need to associate the clustered index with the partition scheme as follows:

CREATE CLUSTERED INDEX ix_SalesOrderID
ON Sales.MySalesOrderHeader (SalesOrderID)
ON PSOrderDate (OrderDate)

The preceding statement will distribute the table data into five filegroups based on the yearly data of orders stored in the OrderDate column.

How to Create Partitioned Indexes in SQL Server

In SQL Server, indexes can also be partitioned based on the value ranges. Similar to the partitioned tables, the partitioned indexes also improve query performance. Partitioning enables you to manage and access subsets of data quickly and efficiently. When indexes become very large, you can partition the data into smaller, more manageable sections as the data is spread across file groups in a database.

Consider an example. In the Adventure Works database, the SalesOrderHeader table contains the details about the order received by Adventure Works, Inc. As the data in this table is large, the query takes a long time to execute. To solve this problem, you can create a partitioned index on the table. Partitioning an index will distribute the data in the table into multiple filegroups, thereby partitioning the table. This will enable the database engine to read or write data quickly. This also helps in maintaining the data efficiently.

Partitioning is allowed only in the Enterprise Edition of SQL Server. To create a partitioned index, you need to perform the following tasks:

• Create a partition function
• Create a partition scheme.
• Create a clustered index.

Creating a Partition Function

Similar to creating a partitioned table, you need to create a partition function to create a partitioned index. The partition function will determine the boundary values for creating partitions.

Consider an example. The queries on the SalesOrderHeader table are mostly base on the OrderData column. The sales manager of Adventure Works requires the details of the orders received, on a yearly basis. The table contains the details of orders for the last five years beginning from 2001. Based on this information, you can create a partition function as follows:

CREATE PARTITION FUNCTION PFOrderDate (date time)
AS RANGE RIGHT FOR VALUES (‘2002-01-01’, ‘2003-01-01’, ‘2004-01-01’, ‘2005-01-01’)

The preceding statement creates a partition function, PFOrderDate, by using the date time data type. It specifies four boundary values. Therefore, there will be five partitions. As range right is specified for partitioning, the first partition will contain data less than the first boundary value, 2002-01-01. The second partition will contain data greater than or equal to 2002-01-01 and less than or equal to 2003-01-01. Similarly, other partitions will store data.

Types of Secondary XML Indexes in SQL Server

These are non-clustered index of the primary XML index. There must be a primary xml index before each secondary xml index. Following are some types of secondary xml indexes:

Path Indexes

The path index is built on the path value columns of the primary XML indexes. This index improves the performance of queries that use paths and values to select data.

For example, if you execute a query that checks for the existence of a product model ID using an XQuery expression as /PD:ProductDescription/@ProductModelID[.=”19”], you can create a path secondary index on the CatalogDescription column of the ProductModel table. In this path index, you can use the primary index created previously.

The following statement creates a Path index on the CatalogDescription column:

CREATE XML INDEX PIdx_ProductModel_CatalogDescription_PATH ON Production.ProductModel (CatalogDesctiption)USING XML INDEX PXML_ProductModel_CatalogDescription FOR PATH

The preceding code create a path index, Pldx_ProductModel_CatalogDesctiption_PATH

Value Indexes

The value indexes contain the same items as path indexes but in the reverse order. It contains the value of the column first and then the path id. This index improves the performance of queries that use paths to select data.

For example, if you execute a query that checks the existence of a node in an XQuery expression such as//Item@ProductID=”1”], you can create a value secondary index by using the primary index created previously.

The following statement creates a value index on the CatalogDesctiption column:

CREATE XML INDEX PIdx_ProductModel_CatalogDesctiption_VALUE ON Production.ProductModel (CatalogDesctiption)
USING XML INDEX PXML_ProductModel_CatalogDescription
FOR VALUE

The preceding code creates a value index,
PIdx_ProductModel_CatalogDescription_VALUE on the CatalogDescription column of the table.

Property Indexes

The property index contains the primary key of the base table, path id, and the clause columns of primary XML indexes. This index improves the performance of queries that use paths to select data.

For example, if you execute a query that returns a value of the node in an XQuery expression, such as /ItemList/Item/@ProductID)[1], you can create a property secondary index on the CatalogDescription column of the ProductModel table by using the following statement:

CREATE XML INDEX PIdx_ProductModel_CatalogDescription_PROPERTY ON Production.ProductModel (CatalogDescription)
USING XML INDEX PXML_ProductModel_CatalogDescription FOR PROPERTY

The preceding code creates a property index, PIdx_ProductModel_CatalogDescription_PROPERTY, on the CatalogDescription column of the table.

You need to consider the following guidelines while creating an XML index:

• XML indexes can be created only on XML columns.
• XML indexes only support indexing a single XML column.
• XML indexes can only be added to tables, views, table-valued variables with XML column or XML variables.
• XML indexes created on a table do not allow you to modify the primary key. To do so, you first need to drop all the XML indexes on the table.

Types of XML Indexes used in SQL Server: Speed up Execution

When a query is based on an XML column, the query processor needs to parse the XML data each time the query is executed. In SQL Server, and XML data value can be of a maximum of two gigabytes (GB).

Therefore, the XML values can be very large and the server might take time to generate the result set. To speed up the execution of the query based on the XML data type, SQL Server allows you to create an index that is based on columns storing XML data values. Such indexes are called XML indexes.

Primary XML Index

This is a clustered B-Tree representation of the nodes in the XML data. When an index is created on a column with the XML data type, an entry will be created for all the nodes in the XML data. Therefore, the index creates several rows of data for each XML value in the column.

You can create XML indexes on XML columns by using the CREATE PRIMARY XML INDEX and CREATE XML INDEX T-SQL commands. For example, the ProductModel table contains the CatalogDescription column that stores XML values. You can create a primary XML index on this column by using the following statement:

CREATE PRIMARY XML INDEX PXML_ProductModel_CatalogDesctiption ON Production.ProductModel (CatalogDescription)

The preceding statement will create an index for all the nodes in the XML data stored in the CatalogDescription column.

Secondary XML Index

This is a non-clustered index of the primary XML index. A primary XML index must exist before any secondary index can be created. After you have created the primary XML index, an additional three kinds of secondary XML indexes can be defined on the table. The secondary XML indexes assist in the XQuery processing.

The three types of secondary XML indexes are:

• Path indexes
• Value indexes
• Property indexes