Practical Business Intelligence with SQL Server 2005: Data Mining

As you have seen, Analysis Services enables you to build powerful Business Intelligence (BI) solutions that enable users to really understand the business. However, many business problems rely on the ability to spot patterns and trends across data sets that are far too large or complex for human analysts. Data mining can be used to explore your data and find these patterns, allowing you to begin to ask why things happen and to predict what will happen in the future.


In this chapter, we look at how to use some of the data mining features in Analysis Services 2005 to perform tasks such as customer segmentation and market basket analysis. The data mining results are presented in the form of new dimensions in cubes and are used in Web applications.
Business Problem

Our customer for this chapter is a large music retailer with stores across the country, and which also has an e-commerce site where customers can buy CDs. The retailer has also moved into the broader entertainment market and added product lines such as videos, computer games, and, more recently, DVDs. This latest product line has just been added to the Web site so that customers can buy DVDs online.
Problem Statement

The retailer faces strong competition in the online DVD market and is struggling to achieve profitability and gain market share. Its e-commerce system has built-in capabilities for conducting marketing campaigns and performing analysis; however, this is restricted to information learned from customers' online behavior and does not tie back into the retailer's extensive data warehouse, which is populated mostly with information from their stores.

This has led to the following challenges:

* There is currently no way to segment customers by combining the extensive customer profile information with the Internet-usage metrics. This segmentation is needed so that they can target direct mail and other marketing to segments that will potentially use the Internet channel.
* The profit margin on DVD sales is low because of extensive competition. The retailer needs to find ways to increase the value of items sold in a transaction, such as by promoting and cross-selling additional products at the time of the purchase.

Solution Overview

We will build an extraction, transformation, and loading (ETL) process to add the Web site's visit-tracking data to the corporate data warehouse. We will use the data mining features of Analysis Services to help discover patterns in this data and provide the information back to the business.
Business Requirements

The high-level requirements to support the business objectives are as follows:

* Customer segmentation. The data warehouse already has excellent profiling information on customers that is obtained through a popular store loyalty card program. This information includes demographic profiles and detailed purchasing histories, because the customer's unique card number can be used to identify store transactions. However, the business also needs a profile of customers' online activities.

The main areas of interest are frequency, or how often the customer uses the Web site, and recency, or how much time has elapsed since they visited the site. There is already information in the data warehouse on the third area of interest, which is intensity, or how much money the customer is spending through the Internet channel.

When these Internet profiling attributes are available, customers can be segmented into groups with relatively similar behavior. Analysts can use the information for marketing purposes, such as producing lists of customers for direct mail campaigns, as well as performing further analysis using the attributes and groups that we identified.
* Online recommendations. They would like to add an online recommendations feature to the new DVD area of the Web site to drive additional profit per online transaction. When a customer adds a DVD to her shopping basket, she must be prompted with a short list of other titles that she may be interested in.The performance of this recommendation needs to be good because any delay in the responsiveness of the Web site has been shown to lead to more abandoned transactions. Also, the recommendation must include items sold through the physical stores as well as the Web site, because the stores currently make up the bulk of the sales.

High-Level Architecture

We will add the Internet visit information to the existing data warehouse and Analysis Services cubes. Because the e-commerce application already extracts data from the Web logs and inserts it into a relational database, we will use this as the source for the ETL process. The data in this source database already has discrete user sessions identified.

Many e-commerce applications (including those based on the Microsoft Commerce Server platform) provide this kind of extraction and log processing functionality, but for custom Web sites, the only available tracking information may be the raw Internet Information Server (IIS) logs. A full treatment of the steps to extract this kind of information from Web log files is beyond the scope of this chapter; see the sidebar "Extracting Information from IIS Logs" for a high-level explanation.

After this information is in the data warehouse, we will use the data mining features of Analysis Services to achieve the business goals for segmentation and recommendations, as shown in Figure 10-1. For each area, we will create a data mining structure that describes the underlying business problem and then run the appropriate data mining algorithm against the data to build a mathematical model. This model can then be used both for predictions such as recommending a list of products or for grouping information in cubes together in new ways to enable more complex analyses.

read more this news at : http://www.informit.com/articles