Architecting A Data Warehouse - Designmind

ARCHITECTING A DATA WAREHOUSE: A CASE STUDY Project: zBis Carl Zeiss Vision North America

SPEAKER: ELIZABETH DIAMOND SAN FRANCISCO SQL SERVER USER GROUP SEPTEMBER 9, 2009 MARK GINNEBAUGH, USER GROUP LEADER

www.bayareasql.org

The Journey
Determined Need for Enterprise Data Warehouse
Worked with Business Users to Understand Business

Requirements
Determined Software Requirements
MS SQL Server 2005 & 2008
MS SSIS (ETL Tool)
MS SSAS (Analytic Cube Tool)
MS SSRS & Excel (Reporting Tools)
SharePoint for Deploying Reports over Company Intranet
Designed and Developed zBis Data Warehouse

Z BIS = What We Will Deliver

This project team will deliver the following:
Consolidated reporting for Carl Zeiss Vision X

North America

X
Reporting that is consistent and from one data warehouse

X
Reporting that is easy to use and easy to access

X
Toolset will be flexible and able to grow and change with our business

X Phase I rock solid download from ERP/Manf
– Providing ability to review lab information as a lab network – not individual silos – with accurate reporting across all products and services We will deliver the best product possible based on the information we can place in our data warehouse!

• Reporting from cubes – off source systems only – No data warehouse • Disparate data systems with different results from each • Most systems not balanced to GL • Reporting for each business unit only • No reporting across all business units

Transactional Cube of Approach Sales Queries

Other Reports

Sales Reports

Corporate

Download Data Mart Finance

Data Mart Inventory

Data Mart Sales & Marketing

ETL Loads ETL Load

ODS/Staging Operational Data Store ETL Load

ERP

Manufacturing

Other

BI Tools/Analytics

Active Reports

Excel

Static Reports

PerformancePoint Server SharePoint SQL Reporting Server

SQL Analytics Server (SSAS)

Aggregated Data Mart TBD

Finance Data Mart

Inventory Data Mart

ETL Load (SSIS) Data Warehouse

ETL Load (SSIS) ODS/Staging Operational Data Store

ETL Load (SSIS) ERP

Manufacturing SW

Other Data Sources

Sales Data Mart

Introduction to Data Warehousing
What is a Data Warehouse System
Why a Data Warehouse Vs. Cubes on Source Systems
Star Schema Vs. Transactional Data Warehouses
Star Schemas ease of system integrating
Star Schemas provide substantial performance gains
Star Schemas hierarchy capabilities or Drill Down

Capabilities
Ralph Kimball Developed Current Industry Standards for Star Schema – Dimensions and Facts

Data Warehouse Project Lifecycle Technical Architecture Design Project Planning

Business Requirement Definition

Dimensional Modeling

Product Selection & Installation Physical Design

Report Specifications

Data Staging Design & Development Report Development

Testing ETL & DW/DM Report Testing

Project Management

Deployment

Maintenance

4 + 1 – Steps Dimensional Design Process Ralph Kimball’s Process for Developing Star Schemas 1. Determine Business Process




2.

Identify the Grain of the Fact • •

3. 4.

Model business Processes Each Process will determine 1 or more Facts Design DW by Business Process Not Business Unit What does 1 row in Fact table represent Transactional or Summary

Design the DW Dimensions Design the DW Facts

+1 Determine Hierarchies

Business Driven vs. Data Driven
Design DW/BI System via Business Process
Develop DW/BI System via Data from Source Systems
Profile Data as early as possible
Understand data and design DW using existing data


Design & Develop using both Business Process and available

Data if possible

Understanding Your Business
Identify key business sponsors for DW project
Use Corporate Org Chart


Setup initial interviews with key sponsors
Develop Business Process diagrams
Develop high level Use Case Diagrams
Determine Business Hierarchies

The Business Executive Interview • What are the objectives of your organization? • What Business goals do you want to accomplish with the

development of zBis data warehouse System? • How do you measure success? How do you know you are doing

well? How often do you measure your corporate performance? • What are your key business issues that you are trying to solve

from the zBis system? If these issues are not justified what is the impact to your department and organization?

The Business Executive Interview • How do you identify problems or know when you might be

headed for trouble? • How do you spot exceptions in your business? What

opportunities exist to dramatically impact your business based on improved access to information? What is the financial impact • If you could….., What would it mean to your business? • What is your vision to better leverage information within your

organization? • How do you anticipate that your staff will interact directly with

this information?

The Business Manager Interview • What are the objectives of your department? • What are you trying to accomplish? How would do you go

about achieving your objectives? • What are your success metrics? • How do you know you are doing well? • How often do you measure your department/team? • How do you anticipate that your staff will interact directly with

this information?

Business Process Diagrams
Understand Business Requirements for building DW/BI system.
Defines the Measures and Dimensions for data warehouse

Determine Hierarchies
Customer Hierarchies – Sales Channels
Distribution Channels
Business Channels
Customer Channels
Product Divisions
Sales Organizations
Sales Office
Buy Groups/Directly Purchase

Determine Hierarchies
Product Hierarchy
Manufacturer
Brand
Product Type – Each product type had own Hierarchy

Lens
Service
Equipment
etc…
Design
Make/Model


Determine Hierarchies
Geo Hierarchy
Sales Division
Sales Region
Sales Territory

Conformed Dimensions
Standardized dimensions across data warehouse
Dimensions are associated with multiple business

processes
Determine by using Bus Matrix & enforced in ETL
Conformed Dimensions are shared and consistent across fact tables

Use Data Warehouse BUS Matrix
Use Data Warehouse BUS Matrix for
Understanding & mapping of Business Processes and

Dimensions
Ongoing DW/BI planning efforts
Team & Management Communications
Understand Business Process unions across the enterprise

Data Warehouse BUS Matrix Date

Company

Customer

Product

Geo

Dist Ctr

Promo

Company Sales

X

X

X

X

X

X

Customer Discounts

X

X

X

X

X

X

Product Cost

X

X

X

X

X

Company Inventory

X

X

X

Dist Ctr Inventory

X

X

X

X

X

Develop Dimensional Schema

Slow Changing Dimensions
Type 1 – Overwrite existing Dimension Row
Use when don’t need to keep history data row
Can be used to correct bad data


Type 2 – Create a new Dimension Row
Use date and/or active non-active fields to identify current

and inactive data rows
Type 3 – Keep old and add new attributes in Dimension Row
Allow Alternate realities to exist simultaneously in one Dimension Row
Slow Changing Dimensions are handled in the ETL

Type of Dimensions
Mini-Dimension
Junk Dimensions
Outrigger Dimensions
Small Static Dimensions
Lookup tables

Type of Facts
Transaction Fact Tables
Snapshot Fact Tables
Accumulating Snapshot Fact Tables
Consolidated or Aggregated Fact Tables

Bridge Tables

Bridge Tables

Recommended Reading list
The Data Warehouse Toolkit: The Complete Guide to Dimensional

Modeling (Second Edition) by Ralph Kimball and Margy Ross
The MicrosoftData Warehouse Toolkit: With SQL Server2005 and the

MicrosoftBusiness Intelligence Toolset by Joy Mundy, Warren Thornthwaite, and Ralph Kimball
Building a Data Warehouse: With Examples in SQL Server (Expert's Voice)

by Vincent Rainardi
The Data Warehouse Lifecycle Toolkit by Ralph Kimball, Margy Ross,

Warren Thornthwaite, and Joy Mundy
The Data Warehouse ETL Toolkit: Practical Techniques for Extracting,

Cleanin by Ralph Kimball and Joe Caserta

Elizabeth Diamond LSI Data Management / Partner with DesignMind Sr. Data Warehouse Architect [email protected] Phone: 415-990-8204

www.bayareasql.org To attend our meetings or inquire about speaking opportunities, please contact: Mark Ginnebaugh, User Group Leader [email protected]