Informatica PowerMart / PowerCenter 6 Basics Hands-on Lab Guide PC6B-20030512
Informatica Corporation, 2003.
All rights reserved.
Duplication prohibited without written consent from Informatica Corporation
Table of Contents Document Conventions........................................................ ..............5 Lab Guidelines.................................................................. .................5 The Design Process .......................................................................... .7 Lab 1 – Create a Folder........................................................... ...........7 Start Repository Manager..........................................................................7 Connect to the Repository.........................................................................8 Create a Folder........................................................................................8 Close the Repository Manager..................................................................10
Lab 2 – Analyze Source Data........................................................... .12 Start the Designer..................................................................................12 Open Folder...........................................................................................12 Open the Source Analyzer........................................................................14 Analyze Source Definitions.......................................................................16 Edit the Source Definition........................................................................18 Create Metadata Extensions.....................................................................20 Clear the Workspace...............................................................................22 Save Work.............................................................................................23
Lab 3 – Import Target Schema........................................................25 Open the Warehouse Designer.................................................................25 Import Target Table Definitions.................................................................26
Lab 4 – Create a Mapping................................................................29 Open Mapping Designer...........................................................................29 Create Mapping ....................................................................................30 Add Source and Target Definitions............................................................31 Source Qualifier Transformation................................................................34 Create an Expression Transformation .......................................................36 Modify / Add Ports..................................................................................39 Create Expression Formula......................................................................40 Validate the Mapping ..............................................................................45
Lab 5 – Create a Workflow......................................................... ......50 Start Workflow Manager..........................................................................50 Create Workflow.....................................................................................51 Edit the Workflow...................................................................................54 Create Metadata Extensions.....................................................................54 Working with Tasks.................................................................................55 Create Session Task................................................................................56 Edit Session Task....................................................................................57 Link Workflow Tasks................................................................................60 Validate the Workflow .............................................................................62
Lab 6 – Start and Monitor the Workflow..........................................65 Open Workflow.......................................................................................65 Close the Workflow Monitor .....................................................................67 Start Workflow.......................................................................................68 The Workflow Monitor..............................................................................70
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
1
Open Workflow Monitor...........................................................................70 Connect to the Repository (Workflow Monitor)............................................72 Monitoring the Workflow..........................................................................75 View Session Properties...........................................................................76 View the Session Log..............................................................................78
Lab 7 – Features and Techniques I..................................................81 Lesson 1 – Arrange Objects...........................................................................82 Lesson 2 – Remove Links..............................................................................84 Lesson 3 – Manual Linking............................................................................85 Lesson 4 – Revert to Saved...........................................................................86 Lesson 5 – Link Path.....................................................................................88 Lesson 6 – Autolink by name and position....................................................89 Autolink by name ..................................................................................89 Autolink by name and prefix or suffix........................................................91 Autolink by Position.................................................................................92 Lesson 7 – Moving Ports ...............................................................................95 Up & Down Arrows to Move a Port.............................................................95 Drag-and-Drop Method to Move a Port......................................................95 Lesson 8 – Shortcut from Normal View to Edit Port.......................................96 Lesson 9 – Copy Columns versus Link Columns.............................................98 Copy Columns........................................................................................99 Link Columns.........................................................................................99 Copy / Link Columns...............................................................................99 Lesson 10 – Create Transformation Methods...............................................101 Transformation Toolbar Method...............................................................101 Transformation Menu Method..................................................................102 Lesson 11 – Zoom and scale-to-fit ..............................................................103 Lesson 12 – Designer Options ....................................................................107
Lab 8 – The Debugger......................................................... ...........108 Clear Workspace ..................................................................................110 Close Mapping......................................................................................111 Copy Mapping Setup.............................................................................112 Open Mapping......................................................................................117 Edit Mapping........................................................................................118 Activate the Debugger Toolbar................................................................119 Start Debugger.....................................................................................119 Debug Mapping....................................................................................125 Stop the Debugger................................................................................127 Debugger Breakpoints...........................................................................127
Lab 9 – Flat File Wizard and Filter........................................... .......130 Analyze a Flat File source.......................................................................130 Create Expression Transformation...........................................................137 Create Filter Transformation...................................................................141 Results – Session Properties...................................................................144
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
2
Lab 10 – Heterogeneous Join................................................ .........147 Create Joiner Transformation..................................................................151 Create Aggregator Transformation...........................................................154 Results – Session Properties...................................................................159
Lab 11 – Sorter.................................................................. ............162 Results – Session Properties...................................................................168
Lab 12 – Basic Lookup.......................................... .........................169 Create Lookup Transformation................................................................172 Results – Session Properties...................................................................180
Lab 13 – Deleting Rows..................................... ............................182 Results – Session Properties...................................................................187
Lab 14– Data Driven Inserts and Rejects.......................................188 Create Update Strategy Transformation...................................................189 Results – Session Properties...................................................................193 Extra Credit..........................................................................................194 Results – Extra Credit Session Properties.................................................196
Lab 15 – Incremental Update ..................................................... ...198 Create Source Qualifier ........................................................................201 Create Lookup Transformation - Connected..............................................202 Results – Session Properties...................................................................210
Lab 16 – Features and Techniques II.............................................213 Lesson 1 – The Find feature........................................................................213 Find Next.............................................................................................215 Lesson 2 – View Dependencies....................................................................217 Lesson 3 – Compare Compare Compare
Comparing Objects....................................................................219 Instances Within Mapping or Mapplets.......................................219 Target Definitions....................................................................221 Mapping Instances...................................................................224
Lesson 4 – Overview Window......................................................................225 Lesson 5 – Copying and Pasting Objects ....................................................225 Lesson 6 – Shortcuts...................................................................................226 Create Shortcut to Source......................................................................227
Lab 17 – Router ..................................................... .......................229 Create Router Transformation.................................................................232 Results – Session Properties...................................................................235
Lab 18 – Straight Load.............................................................. .....237 Results – Session Properties...................................................................239
Lab 19 – Conditional Lookup................................................ ..........241 Create Mapping Parameter.....................................................................244 Create Unconnected Lookup Procedure....................................................247 Results – Session Properties...................................................................254
Lab 20 – Heterogeneous Targets...................................................257 Results – Session Properties...................................................................261
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
3
Lab 21 – Mapplets.......................................................................... 264 Create the Mapplet................................................................................265
Lab 22 – Reusable Transformations...............................................271 Add Mapplet.........................................................................................272 Create Persistent-Named Lookup Transformation......................................273 Promote Aggregator to be Reusable........................................................275 Open the Transformation Developer........................................................279 Add Reusable Transformations to Mapping...............................................281 Results – Session Properties...................................................................284
Lab 23 – Dynamic Lookup .......................................... ...................290 Create a Sequence Generator Transformation...........................................293 Create Dynamic Cache Lookup Transformation..........................................294 Results – Session Properties...................................................................298
Lab 24 – Sequential Workflow.......................................................300 INDEX.......................................................................................... ..304
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
4
DOCUMENT CONVENTIONS This guide uses the following formatting conventions and icons: If you see… Normal Text Command text
Object Name
Text entered by student
Information text.
Key Concept Text. Reference text Flagged Step
It means… Normal Text Menu Command or button/screen text Name of tables, files, PowerCenter objects (transformations, workflows, ports, etc.) Text to be typed or selected by the student during the lab Tips and hints for the lab Key concepts and background for the topic being discussed during the lab Reference to other Informatica documents that discuss this topic in depth A reference to the Flagged Step is made later in this guide
LAB GUIDELINES • •
•
Don’t get stuck – ask for help! Note that there are dependencies among the labs – subsequent labs require objects that were created in previous labs. If you are unable to complete a lab during the allotted time, you can obtain a copy of the required objects from the Shared_Student folder. (Your instructor will describe the process for obtaining these objects.) Screen shots are based on columns that are not selected by default. All of the transformation objects used in this course
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
5
have the following columns selected: Name, Datatype and Length/Precision. Others will include Expression.
Informatica PowerMart / PowerCenter 6 Basics – Hands-on Lab Guide
6
Lab 1
THE DESIGN PROCESS The first several labs will provide a step-by-step review of the Informatica Design Process. OVERALL OBJECTIVES • •
•
Become familiar with Informatica PowerCenter-specific terminologies Learn how to navigate the PowerCenter GUI interfaces and menus in the Repository Manager, Designer, Workflow Manager and Workflow Monitor Learn how to create Informatica objects: define and read source table data, provide instructions to perform data reformatting, and define and write data to a target table
LAB 1 – CREATE A FOLDER PURPOSE •
Create a student folder to hold all work performed in this course
OBJECTIVES •
Learn to navigate through the PowerCenter Repository Manager GUI
D U R AT I O N •
15 minutes
Start Repository Manager • • •
Start the Repository Manager: Start | Programs | Informatica PowerCenter Client 6.0 | Repository Manager; or Per the Instructor’s directions
Informatica PowerMart / PowerCenter 6 Basics – Lab 1 – Create a Folder
7
Connect to the Repository
There are several ways to connect to the repository. From the Repository Manager’s Navigator window, try one of the following methods: ° °
°
Double-click on the repository; or Select the repository and select Repository | Connect; or Select the repository and click the Connect toolbar.
icon in the
Any of the above methods will produce the Connect to Repository dialog box:
In the above dialog box, the xx appended to the word student represents the number assigned to you by the Instructor (generally 01 to 12). •
Enter the assigned Username and Password
Informatica passwords are case sensitive. The logins are not. •
Click on the Connect button
Create a Folder
A folder is a logical grouping of metadata in the Repository.
Folders may group metadata according to functionality (for example - CRM Group, Financial or Sales), or by individual developer (for example - Jason, Tracy or Team_1). Creating (and managing) folders is generally handled by an Informatica Administrator, rather than by individual developers.
Informatica PowerMart / PowerCenter 6 Basics – Lab 1 – Create a Folder
8
Refer to Velocity, the Informatica Methodology, for best practices in folder management.
In this course, each student will create his/her
own folder. All PowerCenter objects (and relationships) created in the labs will be stored in this folder. To create the folder: • Click on the Repository PC_Course and select Folder | Create. The Create Folder dialog box appears.
Informatica PowerMart / PowerCenter 6 Basics – Lab 1 – Create a Folder
9
•
Enter the following information: °
°
°
°
° °
Name: Enter Studentxx (xx – represents the assigned student number) Descriptions: Enter information that describes the contents of folder Owner: Select the assigned student login from the pull-down list Group: Select the Students group for permission purposes Shortcut: Leave unchecked Permissions: Leave the default permissions for the Owner and Repository, but remove the Write permission for the Owner’s Group.
A new folder is created and its icon window.
appears in the Navigator
Close the Repository Manager In this course, the Instructor will perform other repository administrative tasks, which would normally be done by the designated PowerCenter Repository Administrator – typically a Technical Leader. • •
Single-click on the PC_Course Repository Disconnect from the repository by doing one of the following: ° ° °
•
Click on the Disconnect icon in the toolbar; or Select Repository | Disconnect; or Right-click on the repository PC_Course, and select Disconnect
Close the Repository Manager by selecting Repository | Exit.
Informatica PowerMart / PowerCenter 6 Basics – Lab 1 – Create a Folder
10
Informatica PowerMart / PowerCenter 6 Basics – Lab 1 – Create a Folder
11
Lab 2
LAB 2 – ANALYZE SOURCE DATA PURPOSE •
Import the Data Definition for the EMPLOYEE relational table
OBJECTIVE • •
Learn to navigate through basic functions of the PowerCenter Designer GUI Learn how to use Designer’s Source Analyzer Tool to work with Source Definitions
D U R AT I O N •
20 minutes
Start the Designer • • •
Start the PowerCenter Designer: Start | Programs | Informatica PowerCenter Client 6.0 | Designer; or Per the Instructor’s directions
Connect to Repository •
Connect to the same repository in the same manner described under the flagged step Connect to the Repository on page 8.
Open Folder
Once you’ve connected to the Repository, there are several ways to open a folder. However, there is some new terminology that must be explained regarding the differences between an Open folder and an Expanded folder. What can challenge new developers is that both the Open and Expanded Folders will appear
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
12
exactly the same in the Navigator window. The important difference is what displays (or does not display) in the Workspace to the right. Open Folder versus Expanded Folder – An Open folder is required in order to add, delete or modify objects. All of the work is performed in the Workspace Window to the right of the Navigator Window (i.e.- where the tools such as the Source Analyzer, Warehouse Designer, Mapplet Designer, etc., are active). Once created in the Workspace, all objects will appear in the Navigator Window.
An Expanded folder looks like an Open folder in the Navigator window, but has no active Workspace window associated with it. Therefore, no object creation or modification is possible. Expanding a folder in the Navigator window does allow you to copy objects from, or create shortcuts to, another folder that is open. How to copy objects and create shortcuts will be discussed later in this course.
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
13
In order for you to add, delete or change any object in a folder, the folder must be open. •
Locate the assigned Studentxx folder and try any one of the following steps: ° ° °
Double-click the Studentxx folder; or Right-click on the Studentxx folder and select Open; or Highlight the Studentxx folder and click on the open folder
icon in the toolbar
Note that the double-click method to open a
folder will only work upon initially connecting to the Repository or after closing (Disconnect icon) the folder. If the folder was previously highlighted or expanded, then the other methods mentioned above must be used to open the folder.
Open the Source Analyzer
The Source Analyzer is the first of PowerCenter’s five Designer tools. It will be used to define the schemas (or structural definitions) for all sources of data. When opening a folder for the first time, the Source Analyzer is opened by default. If it is not the active tool in the Workspace, then try one of the following: • •
In the Designer, highlight the Studentxx folder and select Tools | Source Analyzer; or Click on the first icon on the toolbar just above the Workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
14
Workspace Window Maximize icon
• •
Maximize the window by clicking on the maximize icon Create Workbook tabs at the bottom of the Window by selecting: ° View | Workbook
The Workbook View allows you to easily navigate among open folders.
Application Title Bar Open Folder drop-down list
Workspace (Source Analyzer displayed) Workbook Tab
Displayed in the Application Title Bar are the Tool name (Source Analyzer), the Folder name and the Repository name. The Folder name is also displayed in the Open Folders drop-down list.
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
15
Notice the eight ‘nodes’ within the folder. All
Repository folders contain these same nodes and they can not be deleted or modified.
There are currently no objects defined in the folder above, nor should there be in any of the Studentxx folders created in the previous lab. Once objects are defined, a ‘+’ will appear to the left of any node that contains objects. Clicking on the ‘+’ will expand the node to reveal a list of objects.
Analyze Source Definitions
Source Definitions All source data to be read by the Informatica Server must first have its structure (schema) defined to the PowerCenter Repository. For relational sources, this includes columns, keys, datatypes, precision and scale (decimals). The process of defining the structure of any source data creates a PowerCenter Source Definition object. (Defining other types of source data will be covered later in this course.) Each Source Definition is similar to a parent object, which can be used many times over in the form of child instances. Each child takes on the characteristics of the parent and when the parent object is modified, each child instance is updated automatically. For this lab, the first step in working with PowerCenter objects is to import the table schema definition for the EMPLOYEE database table. • Select Sources | Import from Database. The Import Tables dialog box appears
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
16
a.
Select the ODBC data source from the pull-down list that corresponds to the location of the source tables
b.
Enter the Username and Password given by the Instructor
SDBU may remain as the default Owner name – this will be defined when the Username is entered c.
Note that when tabbing from Username edit
box, the Designer will automatically fill in the Owner name edit box. Typically, when the source is either SQL Server or Sybase, the default owner will not reveal the desired source. Regardless of the database type, if the source is not found try clicking on the All button under the Show owners: section. If the database type is Oracle, this may take several seconds because the supported ODBC driver is not database specific like it is with SQL Server and Sybase; instead, it allows the search to span across an entire database instance with several schemas where a username may have access. •
Click the Connect button.
Once the Connect button changes to read Reconnect, a direct connection to the source database has been established. • • •
In the Select tables box, expand the database Owner name until the TABLES (node) listing appears. Expand the TABLES node. Select the EMPLOYEE table and click on the OK button.
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
17
The new source table definition now appears in the Source Analyzer Workspace.
Source Definition
In addition to the new source object added to the Source Analyzer Workspace, observe the new entry in the Navigator window. Drill down to view the new Source Definition in the Sources node in the assigned Studentxx Folder.
Edit the Source Definition The next step is to enter meaningful end-user metadata so business analysts will be able to understand the data in the source tables.
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
18
•
• •
• •
In the Source Analyzer Workspace, double-click on the header of the EMPLOYEE table Source Definition. The Edit Tables dialog box appears. Select the Table tab In the Description window, enter: This source comes from the CRM system and includes all Sales Representatives from the Sales Department. Select the Columns tab Select the TYPE_CODE column name. In the Description box, type: Human Resource Job Code
Enter Description
The Description field at the bottom of the
dialog box represents each column independently. In other words, for every column in the column section above, a description can be entered in the Description Edit box below. To associate a description with a particular column, simply highlight the column (anywhere on the column line) and add a description. Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
19
This crucial business metadata may also be imported to the Repository from a third party Data Modeling tool (e.g. Erwin) if the object definition is imported via Informatica’s Metadata Exchange (a.k.a. PowerPlug 6.0). •
Click Apply to save comments.
Create Metadata Extensions
Metadata Extensions Metadata may be extended and stored in the repository by creating metadata extensions for repository objects. The Designer allows the creation of metadata extensions for source definitions, target definitions, transformations, mappings, and mapplets. Of those objects, Metadata Extensions may be used to store information that is common across objects of the same type. In addition, there are two different types of metadata extensions: Non-reusable Metadata Extensions are unique to each object and will not appear in other objects of the same type – unless they are copies of the original object. Reusable Metadata Extensions will appear in every object of the same type. For this reason, it would be important to establish the names of many of these objects to be as generic as possible. Subsequently, the values entered for the metadata extensions are unique. It is possible to enter ‘fixed values’ for each metadata extension; however, the values can be overridden and will be stored accordingly. Typically, the Repository Administrator would create Reusable Metadata Extensions to be used by Developers. In this example, every Source Definition created will have an author, creation date and potentially, a last modified date. By creating Metadata Extensions with the names Author, Creation Date and Last Modified Date, all Source Definition objects will have the same Metadata Extensions by name, but different values may then be entered for each.
Create two non-reusable, local Metadata Extensions as follows: • Click on the Metadata Extensions tab •
Click on the Add Metadata Extension icon Enter Author into the Extension Name column of the new entry
•
Select String Datatype with precision (Prec) of 30
•
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
20
•
Add a second Metadata Extension and name it Creation_Date
•
Select String Datatype with precision (Prec) of 10 DO NOT SELECT REUSABLE. Click Apply
•
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
21
Metadata Extensions tab
Add Metadata Extensions icon
Enter: Your Name and the current date
• •
Enter your name for the value of ‘Author’ Enter the current date [using the syntax = mm/dd/yyyy] for the value of Creation_Date
This field is not a date field, it is descriptive only •
Click the OK button to save the new entries to the Source Definition
Clear the Workspace To this point there has been no instruction to save to the repository, hence, the new Source Definition object exists in memory on the client workstation, only. The Workspace serves as a giant clipboard. All of the work performed between saves will only be in memory including adding, deleting or changing objects within the Workspace. Objects may be cleared from the Workspace, but still reside in memory. This can be demonstrated by doing the following: • Right-click on the header of the EMPLOYEE Source Definition in the Workspace window and select Clear from the Workspace; or • Select the object in the Workspace window and choose Edit | Clear; or
In the respective steps above, selecting Delete
from the folder or Edit | Delete will remove the new object from memory and it will no longer exist! The object disappears from the Workspace, yet can be found in the assigned Studentxx folder in the Navigator window (see page 18).
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
22
Since the object is still resident in memory and is visible in the assigned Studentxx folder, the object may be placed, once again, to the Workspace for additional viewing or editing. • Select the EMPLOYEE Source Definition in the Navigator window • Drag-and-drop the object into the Workspace window
Save Work
Save The Designer, unlike the Repository Manager, requires that any changes be manually saved to the Repository. The key concept to grasp is that folders can only be saved one at a time. In order to save the contents of a folder, it must be the active folder in the Workspace. The active folder is the one visible in the Open Folders drop-down list and on the workbook tab (when that view is selected). If changes were made to objects in other folders and not saved, then those folders (one at time) must be the active folder in the Workspace to perform the Save Repository command. In addition, if there is an attempt to close the Designer with unsaved changes to objects, it will notify the end user and provide an opportunity to save, again, one folder at a time.
It is time to save the Source Definition to the
Repository and there are several ways to perform this task. Before proceeding, there is the Repository toolbar that, by default, is not selected upon the PowerCenter client tool installation that looks as follows:
To add it, select Tools | Customize and check the Repository checkbox on the Toolbars Tab. Save the work in the Repository to this point by using one of the following methods: ° Select Repository | Save; or ° Ctrl-s; or °
Click on the Save
icon
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
23
There is a feature to undo changes called Revert to Saved that will be covered later in the course.
In addition, consider the risks where work can be lost (e.g.: the client workstation crashes while playing MP3 music, the database crashes – not uncommon in a development environment or network connectivity halts, etc.) Over time, several objects can be changed and if anything should prevent the client workstation to communicate with the PowerCenter Repository on the database server, all the work will be lost since the last save. So, a best practice would be to SAVE OFTEN to avoid losing changes.
Informatica PowerMart / PowerCenter 6 Basics – Lab 2 – Analyze Source Data
24
Lab 3
LAB 3 – IMPORT TARGET SCHEMA PURPOSE •
Import the Data Definition for the ODS_EMPLOYEE relational table
OBJECTIVE •
Learn how to navigate Designer’s Warehouse Designer Tool
D U R AT I O N •
10 minutes
Open the Warehouse Designer
If the Designer application is not open, refer to the flagged steps Start the Designer and Connect to Repository on page 12 Select any of the Designer tools (at any time) via the Tools menu or using one of the five icons across the top of the Workspace, as seen below •
Open the Warehouse Designer by trying one of the following: ° Select Tools | Warehouse Designer; or ° Click on the second icon on the toolbar just above the Workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 3 – Import Target Schema
25
Import Target Table Definitions
Target Definitions Similar to defining the source in the previous exercise, the target must also be defined and saved to the repository. For relational target tables, columns, keys, datatypes, precision and scale (decimals) are included. Indexes may be defined, but are not required. Special cases will be covered later in this course. Defining the structure of target data – either via importing from the database (schema) or creating it manually -- creates a PowerCenter Target Definition object. Import the structure of the ODS_EMPLOYEE table from the database using Warehouse Designer. • Select Targets | Import from Database…
Informatica PowerMart / PowerCenter 6 Basics – Lab 3 – Import Target Schema
26
The Import Tables dialog box is the same one
used to connect to the source database. Login to the target database in the same manner; refer to the flagged step Analyze Source Definitions on page 16. • •
• • • • •
Select the odbc_target ODBC data source connection from the pull-down list Enter the Username and Password assigned by the Instructor – typically tdbuxx / tdbuxx (where xx is the assigned student number for the username / password) Click on Connect Expand the database owner name Expand the TABLES (node) listing Select the ODS_EMPLOYEE table Click the OK button
Informatica PowerMart / PowerCenter 6 Basics – Lab 3 – Import Target Schema
27
•
Expand the Targets (node), in the Navigator Window. Notice the addition of the ODS_EMPLOYEE definition
•
Save (See flagged step
Save Work on page 23)
Informatica PowerMart / PowerCenter 6 Basics – Lab 3 – Import Target Schema
28
Lab 4
LAB 4 – CREATE A MAPPING PURPOSE •
Create a mapping that represents the dataflow between the EMPLOYEE source and the ODS_EMPLOYEE target
OBJECTIVE • •
Learn to navigate Designer’s Mapping Designer Tool. Learn PowerCenter-specific terminologies as they relate to defining data transformation instructions
D U R AT I O N •
20 minutes
Open Mapping Designer
Mapping A mapping represents the dataflow between sources and targets. When the Informatica Server executes a Session Task within a Workflow (to be discussed later), it uses the instructions configured in the mapping to read, transform, and write data. •
Select the Studentxx folder and open the Mapping Designer by trying one of the following: ° Select Tools | Mapping Designer; or ° Click on the last icon on the toolbar just above the Workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
29
Create Mapping
By default, there is an inactive toolbar that may be desirable for some developers. To add it, select Tools | Customize and check the Mapping/Mapplet checkbox on the Toolbars Tab.
•
Create a new mapping using one of the following methods: ° Select Mappings | Create; or Click the Create Mapping icon from the Mapping/Mapplet toolbar Enter m_ODS_EMPLOYEE_xx for the new mapping name (where xx is the assigned student number) °
•
•
Click on the OK button
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
30
Recent Mappings list
Mappings Node
A new mapping has been created; however, there are no objects present – those will have to be added. Notice that the name of the mapping appears on the ‘Recent Mappings’ pull down list AND in the Mappings node in the Navigator window Add Source and Target Definitions •
In Designer’s Navigator window, expand the Sources node and locate the EMPLOYEE Source Definition
Organizing Source Definitions Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
31
Normally there are several locations and types of sources, e.g. - databases, flat file, ERP sources, etc. needed to populate a Data Warehouse. Note in the Sources node the EMPLOYEE Source Definition is grouped under the name, odbc_source. When the source is imported via an ODBC data source, the name of the ODBC data source name (DSN) is taken as the “database” name. The reason for this is there may be another EMPLOYEE table in another database with a different table structure that may be needed – the name taken from the ODBC DSN makes the Source Definition unique. If the same source table name, EMPLOYEE, were imported via a different ODBC DSN into the same folder, the repository wouldn’t be able to differentiate between the two. This can potentially cause an organizational nightmare and will be very confusing to developers. The “database” name can be changed while in Edit mode, however, renaming several source definitions is a tedious task. This scenario will often occur when there are several developers adding sources to the same folder within the repository. Since ODBC data sources are not shared across workstations, it is vital to select a naming convention to avoid this problem. Hence, an Informatica Best Practice is for those who are responsible for importing Source Definitions to synchronize the selected ODBC DSN naming convention across workstations. Another way to organize sources, among other select objects, is via the Business Components node – this will be discussed later in the course. An important point to remember is no matter how source definitions are organized in the repository, it will not impact how the data is moved from source to target. Refer back to the architecture slides to see how everything connects. At runtime, the connections (there are several options to discuss) are handled from the server’s perspective, not the client workstations, and will be discussed in greater detail when the Workflow Manager is covered. • •
Drag-and-drop the EMPLOYEE Source Definition to the far left side of the Workspace In the Navigator, expand the Targets section (node) and locate the ODS_EMPLOYEE table Target Definition. Select it with the mouse
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
32
Unlike Source Definitions, the ODS_EMPLOYEE Target Definition is not associated with any ODBC DSN. Therefore, there is no need to coordinate the name(s) of the ODBC DSN for those who are responsible for the targets. Most customers, who use PowerCenter, Extract from several disparate sources, Transform and Load data (ETL) into a common data warehouse or data mart. •
Drag-and-drop the ODS_EMPLOYEE Target Definition to the far right side of the Workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
33
Source Qualifier Transformation
Source Qualifier Transformation Notice the object called SQ_EMPLOYEE in the mapping above. This transformation object is called the Source Qualifier and is required for all relational and flat file sources. It was created automatically when an instance of the Source Definition EMPLOYEE was added to the mapping. The Source Qualifier is the “interpreter” for the PowerCenter engine. This is where the file or table read is defined. For relational sources, the Source Qualifier will deliver the SQL (dynamically or via SQL overrides) to the Informatica Server at runtime, for execution according to database type. In turn, the database type dictates or “interprets” the respective native data types that will be delivered to the database engine for extraction. As this course continues, there will be several other attributes of the Source Qualifier to explore and other types of PowerCenter source “interpreters” will be introduced.
The Source Qualifier may be renamed if desired. For this Lab, the default name SQ_EMPLOYEE is adequate; although, it will be renamed to sq_EMPLOYEE to follow Informatica Velocity Best Practices •
Edit the sq_EMPLOYEE by either of the following methods: ° Double-click on the header; or ° Right-click on the header and select Edit ° Click on the Ports tab to view the structure of the data to be read
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
34
The Datatypes in the Source Qualifier are all
PowerCenter Datatypes. Fundamentally, this is how PowerCenter can join sources of disparate database types (e.g.: Oracle to DB2 or Flat File, which is considered as database type in PowerCenter). It’s not quite that simple, however, the Informatica Server must translate the native database datatypes to PowerCenter datatypes before processing. Once the native database Datatype is translated to a PowerCenter datatype, it will remain a PowerCenter Datatype in every transformation object. That is, until it ultimately reaches the Target, where it will be translated to the defined database Datatype or Flat File. •
Click on the OK button to close the sq_EMPLOYEE Source Qualifier
Ports Notice the name of the first column, ‘Port Name’. ‘Port’ is the generic Informatica term used for both table columns and flat file fields.
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
35
Create an Expression Transformation One mapping objective is to concatenate the FIRST_NAME and LAST_NAME on each row prior to loading the target. An Expression Transformation is ideal for this task.
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
36
Passive versus Active Transformations There are two types of transformations: Passive and Active. Passive transformations allow every row of data to pass through it – so if 100 rows go in, then 100 rows will go out to the next transformation. Active transformations may decrease or increase the number of rows passing through. As each transformation object is introduced during the course, it will be labeled as either passive or active. There are internal validation checks to make sure transformations can be linked together based on this very concept. •
Locate the Transformation Toolbar
The default location is at the top right of the Designer. Look for the ‘arrow’ icon just to the right of a set up vertical bars •
Single-click on the double vertical bars and drag to the center of the Workspace
•
Another suggested location is right below the default location of the Standard Toolbar – to move it, drag and drop from the title bar
If at any time this toolbar is not visible, activate it in the same way other toolbars have been activated by selecting Tools | Customize and checking the Transformation checkbox
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
37
If the checkmark is present, then try to maximize the window of the Designer; or open the Customize dialog box, again, and remove the checkmark and click OK. Reopen the Customize dialog box and check the Transformation checkbox, and click OK. •
Click on the Expression transformation Transformation toolbar
•
Drag the pointer, which now appears as crosshairs , into the Workspace window to the right of the Source Qualifier transformation Single-click (or, click-and-drag, which will draw a box, then release)
•
icon in the
Another way to add transformation objects to mappings is selecting Transformation | Create. After the Create Transformation dialog box appears select the type of transformation from the pull-down list; or type the first letter of the transformation name until the desired transformation type appears on the list. Enter a name for the transformation. Click on the Create button and if no additional transformations are required then click on the Done button. •
•
Set the Mapping Designer to copy port names and create link relationships between transformation objects: °
Select Layout | Link Columns;
°
Select the Link Columns
OR
icon in the toolbar
Link the following ports from the sq_EMPLOYEE to the new Expression transformation: EMPLOYEE_ID, FIRST_NAME, LAST_NAME, DATE_HIRED and TYPE_CODE
With the Source Qualifier and the Expression transformations in ‘Normal’ mode, hold down the Ctrl key and select the above ports in the sq_EMPLOYEE. Drag them to an empty line on the Expression transformation. When the mouse
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
38
is released, not only will the port names (including datatypes, precision and scale) will be copied from sq_EMPLOYEE, but links connecting the ports between the two transformations will also be created. • • •
Double-click on the header of the Expression transformation to enter Edit mode Click the Rename button Rename the transformation to exp_ODS_EMPLOYEE
If the transformation was created via the Transformation | Create menu instead of the Transformation Toolbar, the object will have a name •
Click on the OK button to close the dialog box
Modify / Add Ports • •
•
Select the Ports tab Disable the output ports for FIRST_NAME and LAST_NAME by removing the checkmark in the ‘O’ (output) column – this will define the port as input only Click on the Add NAME_out
icon to add a new port and rename it,
Selecting the LAST_NAME port before clicking the ‘Add a new port to this transformation’ icon will cause the new port to be positioned immediately after LAST_NAME. • •
Verify the data type is string and increase the precision to 51 Disable the input port for NAME_out by removing the checkmark in the ‘I’ (input) column – this will define the port as output only
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
39
Add Port icon
New Port, output-only
Create Expression Formula
Notice that expression for the NAME_out
column is no longer grayed out. Once the port is defined as output only (or as a variable ‘V’ port to be discussed later), it requires a user-defined formula in order to be valid. The formula could simply call another port in the transformation by name only. •
Click in the Expression column of the NAME_out port and notice the arrow that appears at the far right Arrow opens Expression Editor
•
Click on the arrow to open the Expression Editor dialog box
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
40
Formula Undo icon Formula Window
NAME_out is currently defined in the ‘Formula:’ window. This name is taken by default, from the name assigned to the port when it was created. This default entry is invalid and must be changed. •
Delete the text NAME_out in the Formula field
•
Select the Ports tab Double-click on the port FIRST_NAME and note its presence in the Formula field
•
Ports Tab
Individual Port Details
Based on which port is selected, the port details appear under the ‘Instance Name:’ section of the Ports tab
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
41
Below the Formula field, locate the Operator
keypad (shown here). It contains mathematic (and other) symbols that can be used in creating formulas
•
Click on the double-pipe icon (concatenation) and it appears just to the right of FIRST_NAME
•
Click on the single-quote icon in the Operator keypad Press the spacebar once and click the single-quote icon again. This will cause a space to be placed between the FIRST_NAME and the LAST_NAME at runtime Click on the double-pipe icon, again Double-click on LAST_NAME. The concatenation formula is complete FIRST_NAME || ' ' || LAST_NAME Click OK. Upon successful parsing of the formula, a dialog will appear proclaiming “Expression parsed successfully”
•
• •
•
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
42
•
Once the formula parses, click OK to continue and return to the exp_ODS_EMPLOYEE transformation in Edit mode
Completed Expression Formula
The formula may be typed in manually rather than following the point-and-click method. If the names of the Functions and Ports are known, it may be faster to type them. If a port name changes, the expression will be automatically changed to incorporate the new port name. If you were to change a port name to a non-existent one from within the formula field prior to changing it in the Port Name dialog box, the expression would fail to parse. • •
Click OK to close the Edit Transformation dialog box Ctrl-s to save The finished Expression transformation will look like the following:
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
43
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
44
Link Target Definition
•
Link the following ports from exp_ODS_EMPLOYEE to ODS_EMPLOYEE (target definition): EMPLOYEE_ID EMPLOYEE_ID NAME_out NAME DATE_HIRED DATE_HIRED TYPE_CODE TYPE_CODE
The mapping is now complete and should look like the following:
•
Ctrl-s to save
Validate the Mapping
Mapping Validation All mappings must pass a set of PowerCenter validation checks before executing it within a Session Task. Every time a repository save is executed, a series of validation checks are performed on what has been changed. If a transformation object is changed within a mapping or mapplet, the object will be checked and then mapping validation checks will occur.
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
45
Since development is iterative, it may be necessary to leave a mapping in an invalid state. All mappings that are invalid will have a red ‘x’ icon just to the left of the mapping name in the Navigator Window.
Validation checks do not automatically seek out invalid mappings to check them. Only when something changes (anything such as links, port names, expressions, etc.) will mapping validation checks occur automatically upon the execution of a repository save. The Mappings | Validate menu command is a manual validation check that can be performed on mappings that are active in the Workspace.
•
View the results of the Validation by locating the Save tab of the Output window, at the bottom of Designer
Output Window’s Save Tab
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
46
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
47
•
Expand the Output window and scroll up in the scrollbar on the right side of the window, so the Validation test detail can be reviewed
Save Time Stamp
First Error
Mapping declared Invalid
If the Validation results shows ‘INVALID’, locate
the last time stamp when the save repository was executed and scan for the first error. The series of validation checks will display all of the errors, however, the first error may cause a domino effect of errors. Fix the first error and save or perform the Mappings | Validate command from the main menu (Note the results will be located in the Output window’s ‘Validate’ tab). Repeat the process until the mapping is valid.
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
48
Validation Time Stamp
No Errors
Mapping declared Valid
Informatica PowerMart / PowerCenter 6 Basics – Lab 4 – Create a Mapping
Validate Tab
49
Lab 5 4
LAB 5 – CREATE A WORKFLOW PURPOSE •
This lab provides instructions to create a simple Workflow with a Start Task and Session Task. Collectively, the goal of creating these objects is essentially to define a run-time version of the mapping created in the previous lab
OBJECTIVE •
Learn to use the Informatica Workflow Manager
D U R AT I O N •
30 minutes
Start Workflow Manager •
Open the Workflow Manager by selecting Start | Programs | Informatica PowerCenter Client 6.0 | Workflow Manager
Connect to Repository •
Connect to the same repository in the same manner described under the flagged step Connect to the Repository on page 8
Open Folder •
Locate the assigned Studentxx folder and try any one of the following steps: ° °
°
Double-click on Studentxx folder and, in the Navigator Window; or Right-click on the assigned Studentxx folder and select Open or Highlight the folder and click on the open folder the toolbar;
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
icon in
50
Open Workflow Designer •
Open the Workflow Designer using one of the following methods: ° Select Tools | Workflow Designer; or °
Click the Workflow Designer Workspace
icon above the
Create Workflow The next step is to create a Workflow to process data. It will contain a Start Task and a Session Task.
A workflow is a set of instructions that tells the Informatica Server how to execute tasks such as sessions, email notifications, and shell commands. After tasks are created in the Task Developer and Workflow Designer, tasks are connected via links to create a workflow. In the Workflow Designer, conditional links and workflow variables are used to create branches in the workflow. The Workflow Manager also provides Event-Wait and Event-Raise tasks. The sequence of task execution can be controlled in the workflow. Worklets (a group of tasks) can be created and inserted inside the workflow. Every workflow contains a Start task, which represents the beginning of the workflow. Refer to “Creating a New Workflow” in the Workflow Administration Guide
This section provides the instructions to create a Workflow first, and then create a Session Task within it. The two objects may be created in reverse; however, a Session Task must be within a Workflow in order to be executed. • •
Select Workflows | Create On the General tab, enter wf_ODS_EMPLOYEE_xx in the Name box (xx represents the assigned student number from 01-12)
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
51
•
Click on the OK button
Note the presence of the Start Task which is added to each new workflow by default •
Select Repository | Save; or Ctrl-s to save
The workflow is incomplete and as a result invalid, but that is to be expected at this point in the exercise • • •
Double-click on the Start Task Click the Rename button Type start_ODS_EMPLOYEE in the Rename Task dialog box
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
52
• •
Click the OK button to save the name Click the OK button to return to the Workflow Designer workspace
The preceding Start Task rename instruction is optional, however, is recommended for clarity purposes when reviewing all of the tasks in the Workflow Monitor.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
53
Edit the Workflow There are several parameters in the new workflow to be set. Edit the workflow by doing one of the following: ° ° •
Click on the Edit Workflow Toolbar; or Select Workflow | Edit
Icon in the Workflow
On the Properties tab, wf_ODS_EMPLOYEE_xx.log into Attribute 2 displays the Workflow Log File Name
Workflow Log File Name
Workflow Log Directory
The workflow log contains basic information (execution start and completion times, status, etc.) for all tasks in the workflow. It is optional and if left blank (default) the Informatica Server writes workflow log messages to the Windows Event Log or UNIX server log. If a log file name is entered, the workflow log messages will be redirected to the designated file name within the Workflow Log File directory on the Informatica Server. Create Metadata Extensions Metadata Extensions were introduced in the flagged step Create Metadata Extensions on page 20. In this lab, the Instructor has already created reusable Metadata Extensions for workflow ‘Creation_Date’ and ‘Workflow_Author’.
The non-reusable Metadata Extensions created in Lab 2 were unique to each folder. With reusable Metadata Extensions, they are available to all workflows across all folders in the repository.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
54
•
Click on the Metadata Extensions tab
Notice the two existing metadata extensions. Each has a checkmark in the ‘Reusable’ column in the right.
Typically, the designated Repository
Administrator(s) creates reusable metadata extensions in the Repository Manager. Depending on how the Metadata Extensions were set up, a default value will either be blank or assigned. If the value entered is different than the default, a new column will appear to allow the developer to revert back to the default value (see below).
•
Enter the current date for the value of Creation_Date [using the syntax as before = mm/dd/yyyy]
•
Enter your name for the value of Workflow_Author Click on OK, to both apply the entries and exit the Workflow Save the new entries to the Repository, select Repository | Save, OR Ctrl-s.
• •
Working with Tasks Now that the Workflow has been created, the next step is to add a Session Task to the Workflow.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
55
By default, there is a toolbar to create tasks that may be desirable for some developers; however, it may be difficult to find. To verify it is present, select Tools | Customize Toolbars and make sure there is a checkmark in the Tasks checkbox.
Very little of the toolbar might be visible as it was with the Transformation Toolbar in Lab 4. Locate the ‘arrow’ icon, which is the first icon in the Tasks Toolbar. Once located, drag-and-drop to the desired location to see all of the task icons available (e.g. horizontally below the Standard toolbar, vertically between windows or floating in the workspace).
Create Session Task
Session Tasks Session tasks represent mappings for the Informatica Server to run. The Informatica Server uses the instructions configured in the session to move data from sources to targets. Among its many properties are parameters for memory usage, log file locations, and source and target read/write instructions. Each session task is associated with one, and only one, mapping; however, one mapping may be used in several session tasks. • •
Click on the Session icon on the Task Toolbar Move the pointer into the Workspace and click
The Mappings dialog box appears listing all
mappings in the folder – valid and invalid – however, only valid mappings are permitted to be selected •
Select the mapping, m_ODS_EMPLOYEE_xx and click OK.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
56
A Session Task appears in the Workspace and it
automatically prefixes the name with an “s_” – the end result is s_m_ODS_EMPLOYEE_xx
An alternate way to create a non-reusable
Session Task is to create it from the menu (similar to the manner described in the flagged step Create the Mapping on page 30). Select Task | Create then select the Session Task type. •
Select Repository | Save, OR Ctrl-s to save
Edit Session Task •
Double-click on the s_m_ODS_EMPLOYEE_xx Session Task and select the Properties tab
Session Log Name
Session Log Directory
The default Session Log File Name,
s_m_ODS_EMPLOYEE_xx, and the default Session Log directory, $PMSessionLogDir, may be overridden, but this is not recommended.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
57
•
Select the Sources tab
Connection Settings tab
•
Native Database Connection Drop-down Listing
Click on the down arrow on the right side of the Value column for the sq_EMPLOYEE listed in the Instance column
Source Native Database Connection
•
Select the native_source connection and click OK
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
58
•
Select the Targets tab
Properties Tab
• •
Native Database Connection Drop-down List
Click on the down arrow on the right side of the Value column for the ODS_EMPLOYEE Target Instance Select native_target_xx (where xx represents the assigned student number from 01-12) and click OK ° Only if the target table database connection is using an Oracle client less than 8.1.7.2 – select the Properties tab at the bottom and select Normal for Target load type
Release note: CR 38672, 36211: Oracle bulk
load session fails or runs in normal mode on some versions of Oracle
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
59
When a session is configured for bulk loading and the target database is on Oracle, the session fails when the Oracle client version is greater than or equal to 8.1.5 and less than 8.1.7.2. When the Oracle client version is less than 8.1.5, the session runs in normal mode. Workaround: Apply the Oracle Threaded Bulk Mode patch (Oracle bug number 1666360), use Oracle client 8.1.7.2 or higher, or run the session in normal mode. • •
Click OK to close the Edit Tasks dialog box Save
Link Workflow Tasks
All tasks in a workflow must be ‘linked’ together for the workflow to be valid.
There are several ways to link tasks: • •
One way is to use the toolbar. Locate the Link icon on the right side of the Task Toolbar Drag the pointer, which now appears as crosshairs , into the Workspace to the task
The cursor remains in ‘link mode’ until toggled off or one of the newly linked objects is single or double-clicked upon. When hovering over a task, the crosshairs change to •
Draw a line between the start_ODS_EMPLOYEE and the s_m_ODS_EMPLOYEE_xx tasks via a drag-and-drop method
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
60
•
Toggle off the ‘link mode’ by clicking again on the Link single click on one of the objects
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
icon or
61
Another method to create the link is to: • First, remove the existing link to practice other linking methods between the two objects. Select the link and hit the Delete Key. Confirm that ONLY the link is to be deleted • To create the link again, hold down the Ctrl key and select both the start_ODS_EMPLOYEE Task and the s_m_ODS_EMPLOYEE_xx Session by single clicking on them or by drawing a box around the two objects (no Ctrl key required when drawing the box) • Select Tasks | Link sequential or right-click anywhere in the workspace except on an object and select Link Sequential. A link appears between the Start Task selected and the Session
Link Sequential or Link Concurrent relates to
how the tasks are processed. If there were two session tasks linked to the Start Task, they would be considered concurrent session tasks would start at the same time. Sequential simply would be one at a time – typically chosen because there dependencies from one session to another. •
Save Refer to “Working with Links” in the Workflow Administration Guide, Chapter 4.
Validate the Workflow The detail of creating the Workflow is now complete. However, it must pass PowerCenter Validation before it can process correctly.
Validating a Workflow Similar to the mapping, a workflow must pass all validation checks prior to running it. For example, if a workflow is created with a Start Task and a Session Task and the two are not linked, the workflow is invalid. The Output window in Workflow Manager displays the results of the validation checks and will point out specifically what (not how) needs to be fixed.
Since a Repository | Save was just executed in the preceding step, the Workflow validation
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
62
checks occurred automatically, just as it does when the mappings are saved in the Designer.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
63
•
Locate the Save tab in the Output Window at the bottom of the Workflow Manager and view the results of the validation checks
Output Window’s Save Tab
If the Validation results shows ‘INVALID’, locate
the last time stamp when the save repository was executed and scan for the first error. The series of validation checks will display all of the errors, however, the first error may cause a domino effect of errors. Fix the first error and save or perform the Workflow | Standard validate command from the main menu (Note the results will be located in the Output window’s ‘Validate’ tab). Repeat the process until the workflow is valid Refer to “Validating a Workflow” in the Workflow Administration Guide.
Informatica PowerMart / PowerCenter 6 Basics – Lab 5 – Create a Workflow
64
Lab 6
LAB 6 – START AND MONITOR THE WORKFLOW PURPOSE •
This lesson provides instructions to start the workflow and monitor the process with the Workflow Monitor.
OBJECTIVE • •
Learn how to start a workflow Learn to use the Informatica Workflow Monitor
D U R AT I O N •
40 minutes Refer to “Running the Workflow” in the Working with Workflows Chapter in the Workflow Administration Guide
Open Workflow Manager •
If the Workflow Manager was closed, re-open it (see flagged step Start Workflow Manager on page 50 for details)
Open Workflow There are two ways to re-open a workflow. To demonstrate both methods, the Workspace must be clear of any workflows: •
Select Workflows | Close – if the menu item to Close is grayed out, then there are no open workflows
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
65
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
66
•
Drag-and-drop the wf_ODS_EMPLOYEE_xx Workflow from the Navigator Window into the Workspace
Notice that the name of the workflow appears
on the ‘Recent Workflows’ pull down list. The Recent Workflows List holds the workflows that have been opened since the last time the folder was closed (disconnected). A folder can be closed in several ways (e.g. Closing the Workflow Manager or Disconnecting from the repository or the folder) – any of which will net the same effect.
Recent Workflows List
Close the Workflow Monitor A Workflow or a specific Task can be started from within the Workflow Manager where it was created. The status of the Workflow and all Tasks associated with it will be monitored from the Workflow Monitor.
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
67
To ensure that the Workflow Monitor Program was not opened inadvertently, please do as follows: • Examine the icons in the Windows Task Bar for the presence of the minimized Workflow Monitor icon • If found, right-click on the icon and select Exit – otherwise, continue
Start Workflow If the workflow is valid, it is ready for execution. In the Workflow Designer, use one of the following methods to start the wf_ODS_EMPLOYEE_xx Workflow: • Select Workflows | Start workflow; or •
Right-click in the Workspace and select Start Workflow; or
•Right-click
on the wf_ODS_EMPLOYEE_xx Workflow in the Navigator window and select Start Workflow; or •
Right-click on the start_ODS_EMPLOYEE task in the wf_ODS_EMPLOYEE_xx Workflow. Select Start Workflow from Task menu option
The menu option Start Task (shown below) is not ideal in this case.
Starts a single Task only
Selecting this menu option starts the start_ODS_EMPLOYEE task, only, and would not
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
68
subsequently run the s_ODS_EMPLOYEE_xx Session Task. Similarly, starting a single Task directly from the Session would run the session but make the Workflow execution incomplete.
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
69
Any of the above options would show the following dialog box:
The workflow and all its tasks will be monitored and examined for processing details in the Workflow Monitor (next section). With the Workflow Monitor closed, as suggested earlier in this lab, the Workflow Monitor icon will automatically appear in the Windows Task Bar.
The Workflow Monitor
Workflow Monitor
-- Initialize vs. Open
There is a difference between the Workflow Monitor program being ‘initialized’ and ‘open’. For example, a virus protection program may be ‘initialized’ and operating in the background (to actively protect the computer), without a client tool appearing as an active program visible on the Desktop. It will be present in the form of a minimized icon on the Windows Taskbar. However, all functional settings cannot be accessed until the program is ‘opened’. Such is the case with Workflow Monitor. Once initialized, it may then be ‘opened’ by the user – without ever accessing the Windows Start menu. Starting the Workflow, wf_ODS_EMPLOYEE_xx from the Workflow Manager automatically initialized the Workflow Monitor. This is a convenient feature that is enabled by default and cannot be turned off. Open Workflow Monitor •
Examine the Windows Task Bar for the presence of the minimized Workflow Monitor icon
If the icon is not found minimized on the taskbar, here are two other methods to open Workflow Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
70
•
Right-click on the PC_Course Repository in the Workflow Manager’s Navigator window and select Run Monitor
•
Select Start | Programs | Informatica PowerCenter Client 6.0 | Workflow Monitor How quickly the Workflow Monitor initializes depends on the speed of network between the Informatica Server and PowerCenter client. A remote connection from the client to the server will likely cause a delay and take a few seconds up to a minute.
To open Workflow Monitor from the minimized taskbar icon, try one of the following methods: • Right-click on it and select Open; or • Double-click on it One of the following two windows will appear: •
Gantt Chart
Navigator window (area)
•
Time window (area)
Task View
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
71
Navigator window
Time window
If the Workflow Monitor does not resemble the above, either the Gantt Chart or Task View, then notify the Instructor before proceeding. The Workflow Monitor needs to be completely disconnected to continue with the instructions going forward. Connect to the Repository (Workflow Monitor) Use any one of the following methods to connect to the repository: •
Select Repository | Connect; or
•
Click the Connect icon in the toolbar; or Double-click on ‘Repository’ in the Navigator window
•
A list of available Repositories will appear in a pull-down list in the Connect to Repository dialog box. This list is local to the client workstation. Any of these methods will produce the Connect to Repository dialog box:
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
72
Repository Select List
Connect to Repository
• •
Select the PC_Course Repository from the Repository pull-down list Tab to the Username box and note the username automatically appears – it was the last username logged into the client workstation
Once the repository is selected, if the Workflow Monitor had been previously used on the client workstation then the last logged in Username, Host Name and Port Number will appear automatically. These are listed in the Windows registry of the local client. The Host/Port combination is communication link where the Server machine is listening for client requests. • • •
Accept the default or enter the assigned Repository studentxx Username and then the Password In the Repository Server box, if there is no default entry, enter the Host Name and Port Number designated by the Instructor Click on the Connect button. The PC_Course Repository shows in the Navigator window. Notice that the disconnect icon and the PC_Server server is displayed; both are signs that the user is connected to the repository
Connect to the Informatica Server In order to view the results of the wf_ODS_EMPLOYEE_xx Workflow, the Server must be connected in either On Line or Off Line mode.
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
73
On Line vs. Off Line Mode In On Line mode, the Workflow Monitor continuously receives current information from the Informatica Server via the Repository Server (e.g. Workflow development testing). With Off Line mode, the Workflow Monitor displays historic information about past Workflow and Task runs by retrieving the information from the Repository (e.g. Production run review). Both Developers and Production Operators will use the Workflow Monitor for Workflow and Task processing on the Informatica PowerCenter Server. Developers, typically, will be in On Line Mode to monitor current activities. They may also act as Production Operators while testing and use it in Off Line Mode to review the previous test runs; Production Operators, of course, will monitor production runs. •
Select the Gantt Chart tab
•
Right-click on Server, PC_Server, and select Connect | On Line. Or double-click on the server and select On Line Double-click on the assigned Studentxx folder to view the previously processed workflows Drill down (double-click on each object) all the way until the session task, s_m_ODS_EMPLOYEE_xx, appears
• •
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
74
Monitoring the Workflow
Workflow
Monitor provides the capability to review server-processing information. For each Session task (the most common), information about rows loaded or failed and run status. In addition, there are two primary types of views, Gantt Chart and Task View. •
Select the Task View
Task Name (icons)
Task View tab
Three objects should appear in the selected folder. Note the run time for each is within seconds of each other: •
The s_m_ODS_EMPLOYEE_xx Session Task --
•
The start_ODS_EMPLOYEE Task --
icon
icon
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
75
•
The wf_ODS_EMPLOYEE_xx Workflow --
icon
Workflow Monitor Time The Workflow Monitor displays Start and End times relative to the time configured on the Informatica Server machine. The PowerCenter client machine interpolates and displays the actual Server run times to the local client time zone. Since the monitoring activity can take place on multiple clients, in multiple time zones, each client will convert the Server time to its local time, e.g. – if the Server runs a Session at 12:00pm EST, a client in EST will display 12:00pm; a client in PST will display the times 3 hours earlier (9:00am); a client in GMT will display the times 6 hours later (6:00pm) than EST. Notice the Status column on the right. The objective is to have all of the tasks show Succeeded. Below, they are grouped together in a view.
Session Status – Success vs. Failure
(vs. Completed with Errors)
A Succeeded status will display if the Informatica PowerCenter Server was able to successfully complete a workflow or task. However, a Session Task may succeed in processing, but may still contain warnings or non-fatal errors. A Failed status may occur if the Informatica Server failed the Workflow or Task due to fatal processing errors. A Session could fail for many reasons (e.g. - read login failure, custom SELECT statement error, or the Server was not able to calculate data due to a NULL condition). A rejected row by the database or discarded by a business rule will not produce a Failed status.
View Session Properties Developers, success or failure, will likely want to review processing details. For example, to see if a Session Task processed and loaded an expected amount of rows. Here’s how to see the results: • Locate the s_m_ODS_EMPLOYEE_xx Session task under the Task Name column
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
76
•
View the Session Properties by doing any one of the following: ° Right-click anywhere on the line selected and select Properties; or ° °
Click on the Session Properties icon; or simply, Double-click on the selected line for s_m_ODS_EMPLOYEE_xx Session
The Properties tab of the s_m_ODS_EMPLOYEE_xx dialog box opens. The Session should display the number of Target Success Rows shown below:
Properties Tab
Expected Results
•
Click on the Transformation Statistics tab. More detail on the number of rows handled by the Server are shown here:
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
77
Target Instance
Transformation Statistics Tab Source Qualifier Instance
Applied / Affected / Rejected Rows Applied rows are rows the Informatica Server successfully produced and applied to the target without errors. Affected rows are generated by the Server and ‘affected to’ (or accepted by) the target. They may be a combination of INSERT, UPDATE and DELETE rows, or multiple rows in a database may be impacted by one UPDATE row. Rejected rows are either those read rows that either the Server dropped during the transformation process, or rows that rejected when writing to the target. Rejected rows may be acceptable and expected. Refer to “Monitoring the Workflow” in Chapter 12 of the Workflow Administration Guide
View the Session Log The Session log is one method to determine what occurred during the system run.
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
78
To view detailed Session information, try one of the following: • •
Right-click on Session in the Task Name column (Time window) and select Get Session Log; or Select the Session anywhere on the line in the Time Window and click on Session Log
icon
The Session Log is copied from the Informatica PowerCenter Server machine as TmpSessionLog_x.log and automatically opens with WordPad on the local client workstation. Multiple logs may be opened on the client, hence the “_x” in the log file name. If errors have occurred, review the Session log to determine what errors occurred. Test and rerun as necessary.
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
79
Processing) Memory (DTM Buffer Pool)
Mapping Name
Session Name
INSERT, UPDATE and DELETE row syntax (if created)
Native Source and Target Database Connections
Session Processing Threads
Informatica PowerMart / PowerCenter 6 Basics – Lab 6 – Start and Monitor the Workflow
80
Lab 7
LAB 7 – FEATURES AND TECHNIQUES I OBJECTIVE This is an Instructor-led “follow along” lesson to show and explain the various Designer features and techniques that can improve efficiency and ease of use. Because these features are introduced together, under the guidance of the Instructor, they do not need to be haphazardly squeezed into the labs. They can be used, when needed, in the subsequent labs. It is recommended not to save any changes at the conclusion of this exercise because it may invalidate the mapping. The following is a list of the more commonly used features and techniques in the Designer tool to assist with mapping development and navigation in the workspace: • • • • • • • • • • • •
Auto Arrange Remove Links Manual Linking Revert to Saved Link Path Autolink by name and position Moving Ports Shortcut from Normal View to Edit Port Copy Columns versus Link Columns Create Transformation Methods Zoom and size-to-fit Designer Options
D U R AT I O N •
40 minutes
Start the Designer Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
81
•
Start the Designer by selecting Start | Programs | Informatica Power Center 6.0 | Designer.
Connect to Repository •
Connect to the PC_Course Repository (refer to flagged step Connect to the Repository on page 8)
Open Folder •
Open the assigned Studentxx Folder (refer to the flagged step Open Folder on page 12)
Open Mapping
•
Open the mapping, m_ODS_EMPLOYEE_xx (refer to the flagged step Open Folder on page 12)
Lesson 1 – Arrange Objects The Designer includes the Arrange feature that will reorganize objects in the workspace in one simple step. This aids in readability and analysis of the mapping flow and can be applied to certain paths through a mapping associated with specific target definitions – this is commonly referred to as a “pipeline”. •
Attempt to Save
If there was nothing to save, don’t fret. This step was just in case the steps above were not needed to open the mapping and a saved wasn’t previously performed. •
In a couple of clicks, this feature can take a mapping that looks like this:
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
82
•
And change it to look like this:
•
Or this:
To display specific pipelines in the Mapping Designer: • Choose Layout | Arrange or right-click in the workspace and select Arrange • Select the target
Select the Iconic option to display arranged pipelines in the Iconic View.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
83
•
Click OK
To restore entire mapping from Iconic View to Normal View, do the following: • • • • •
Choose Layout | Arrange Click the Select All button Uncheck the Iconic checkbox Click OK Save
This is the reset point (Normal View with the
objects aligned) that, when instructed to execute the Revert to saved command, everyone will be at the same starting place. Lesson 2 – Remove Links •
Click-and-drag the pointer over the blue link lines, only, between the exp_ODS_EMPLOYEE and the ODS_EMPLOYEE Target Definition
By default, each link stroked changes in color from blue to red. If any other objects (e.g.: transformations) were selected along with the links, redo the process. •
Press the Delete key to remove the connections and make sure that is all that is deleted
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
84
Lesson 3 – Manual Linking This lesson is to demonstrate the proper technique to manually link ports between transformations
•
Single-click and hold the left mouse button in the white space just to the right of the text for the NAME_out port in the exp_ODS_EMPLOYEE Expression and attempt to link it to the NAME port in the ODS_EMPLOYEE Target Definition
Note the behavior – with absence of the link icon, this indicates the objective to create a link is not in process. In addition, it appears as though several other columns have been selected in the exp_ODS_EMPLOYEE Expression. To successfully create a manual link, the cursor must originate on text in any column visible (e.g. Name, Expression, Datatype, etc.) – not the white space to the right of the text
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
85
•
Make another attempt to create the same link; however, this time, Single-click on text rather than the white space
•
Delete the link to try another manual link method Select only the output ports (Ctrl-click) in the exp_ODS_EMPLOYEE Expression and link them to all of the ports to the ODS_EMPLOYEE Target Definition
•
In order to be successful, the port Datatypes must be compatible. If the ports are out of order, but match by name or some other pattern, consider Autolink as an option.
Lesson 4 – Revert to Saved
Reverting to a Previous Object Version While editing an object in the Designer, if unwanted changes are made there is a way to revert to a previously saved version – undoing the changes since the last save. This works for one transformation object or several depending on the tool.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
86
The Revert to Saved feature works with the following objects: Sources; Targets; Transformations; Mapplets and Mappings. •
Select Edit | Revert to Saved
With mappings, if a Revert to Saved is executed, then all changes must be reverted – it’s not just per selected object. However, if several mappings were changed without a save, only the active mapping in the Workspace would be reverted back. •
• •
To demonstrate that objects can not be reverted back individually, remove the NAME_out and DATE_HIRED ports from the exp_ODS_EMPLOYEE Expression Remove the DATE_HIRED port from the sq_EMPLOYEE Source Qualifier With the sq_EMPLOYEE Source Qualifier being the only object selected, select Edit | Revert to Saved
The same dialog box appears – all changes must revert back.
Individual objects may be reverted back to the previous version without impacting objects around it when using the Source Analyzer, Warehouse Designer and Transformation Developer (to be discussed later) – the Mapplet (to be discussed later) and Mapping Designers require all changes to be reverted back to the previous version. In the tools where individual objects may be reverted back, a Ctrl-Click on each object (or draw a box around those objects) would be required. This technique may be practiced once a second source or target definition is imported.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
87
Lesson 5 – Link Path
Tracing link paths allows the developer to highlight the path of a port either forward or backward through an entire mapping or mapplet.
• •
To make sure all students are on the same page, Revert to Saved Right-click on EMPLOYEE_ID in the sq_EMPLOYEE Source Qualifier sq_EMPLOYEE and Select Link Path | Forward
Notice how EMPLOYEE_ID’s path from
sq_EMPLOYEE all the way to ODS_EMPLOYEE is highlighted in red
•
Right-click on the NAME_out port in the exp_ODS_EMPLOYEE Expression and click Select Link Path | Both
Notice how NAME_out port’s path not only shows where it proceeds to the ODS_EMPLOYEE Target Definition, but also from its origin all the Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
88
way back to the EMPLOYEE Source Definition. Both the FIRST_NAME and LAST_NAME are used in the formula to produce NAME_out, so both links are highlighted in red. •
Right-click on the FIRST_NAME port in the EMPLOYEE Source Definition and click Select Link Path | Forward
Similar to the last example, with the FIRST_NAME going forward it doesn’t stop at the exp_ODS_EMPLOYEE Expression where the port is input only. Because it is used in the formula to calculate the NAME_out, the path continues to the target. Lesson 6 – Autolink by name and position
Autolink by name Developers can automatically link ports by name in the Designer. Use any of the following options to automatically link by name: • Link by name • Link by name and prefix • Link by name and suffix The Designer adds links between input and output ports that have the same name. Linking by name is case insensitive. Link by name when using the same port names across transformations. • • •
Revert to Saved to reset the mapping Remove the links between the exp_ODS_EMPLOYEE Expression and the ODS_EMPLOYEE Target Definition From the Main Menu choose Layout | Autolink
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
89
The Autolink dialog box opens. Another
technique is right-click in the workspace and the same dialog box will open.
By Name
Select the From Transformation from the drop -down arrow then highlight the To Transformations. Using the drop down list, only one transformation may be selected in the From Transformation box and one to many transformations may be selected in the To Transformations section. For objects that contain groups such as Router transformations or XML targets, select the group name from the To Transformations list. •
Click OK
The Designer adds links between input and
output ports that have the same name, regardless of case.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
90
Autolink by name and prefix or suffix
When Autolinking by name, the Designer adds links between ports that have the same name, case insensitive. The Designer also has the ability to link ports based on prefixes or suffixes defined. Adding Suffixes and/or Prefixes in Port Names help identify the ports purpose. For example, a suggested best practice is to use the suffix “_out” when the port is derived from input ports that were modified as it passes through the transformation. Without this feature, Autolink would skip over the names that don’t match and force the developer to manually link the desired ports. To link ports by name and prefix: • • •
Delete the Links between exp_ODS_EMPLOYEE and ODS_EMPLOYEE Choose Layout | Autolink Select the transformations and targets
Only one transformation can be selected in From Transformation listing and multiple transformations can be selected in the To Transformations list. For objects that contain groups such as Router transformations or XML targets, select the group name from the To Transformations list. • •
Select Name radio button Click More to view the options for entering prefixes and suffixes
Note the button toggles to become the Less button
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
91
•
Type _out in selected From Transformation
This singles out all the ports with _out as a suffix in the From Transformation, ignores the suffix and attempts to match the name in the to the To Transformation •
Click OK and the results should look as follows
Autolink by Position
The Autolink by position links the first output port to the first input port, the second output port to the second input port, and so forth. Use this option when creating transformations with related ports when applicable in the same order. Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
92
The Datatype, precision and scale must be
considered when using this Autolink option. In addition, not every port visible in the Normal View has both the input and the output ports selected. In other words, some ports may be required to create an output port (derived port), however, are not needed down stream. A best practice is to uncheck the output port of those ports deemed to be input only. It not only gives the mapping a cleaner look, it simplifies features like Autolink by position. To link ports by position: • • • • •
Revert to Saved to reset the mapping Remove the links between the exp_ODS_EMPLOYEE Expression and the ODS_EMPLOYEE Target Definition Choose Layout | Autolink or right-click in the workspace and select Autolink Select the transformations and targets Select Position
By Position
•
Click OK. The Designer links the first output port to the first input port, the second output port to the second input port, and so forth.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
93
As noted above, ports that have incompatible
data types will not link automatically. Ports with compatible data types will link because it does pass the validation check, however, it is a best practice double-check the results of the Autolink.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
94
Lesson 7 – Moving Ports
There are a couple of ways
to move ports around within a transformation. Moving ports around in a transformation may be required to see which ports are needed for a calculation, or to group them (discussed later), or to simply line them up with how the Target looks to see the flow of the mapping easier. Up & Down Arrows to Move a Port • • •
Revert to Saved to reset the mapping Open the exp_ODS_EMPLOYEE Expression and click on the Ports tab Single-click on the TYPE_CODE port and move it up to the top using the up arrow the tool bar
icon found in the upper right corner of
The results will look as follows:
Drag-and-Drop Method to Move a Port • •
•
Single-click on the number to the left of the DATE_HIRED port Single-click and hold the left mouse button and note the faint square that appears at the bottom of the pointer
While still holding the left mouse button, position the pointer just below the TYPE_CODE port until a red-line appears under it
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
95
•
Release the mouse button and DATE_HIRED will appear below TYPE_CODE
•
Click on the Cancel button to not save the changes
Lesson 8 – Shortcut from Normal View to Edit Port
There is a shortcut to go from the Normal View directly to the port desired in the Edit View – this is especially useful in transformation objects that have dozens of ports.
Prior to completing the next couple of lessons, there is a port to be deleted from exp_ODS_EMPLOYEE transformation in the m_ODS_Employee mapping. This is a great opportunity to demonstrate another very useful technique. From the previous exercise, the TYPE_CODE should exist in the exp_ODS_EMPLOYEE transformation. • • • •
Revert to Saved to reset the mapping Remove the links between the sq_EMPLOYEE Source Qualifier and the exp_ODS_EMPLOYEE Expression Resize or scroll down until the TYPE_CODE port appears in the exp_ODS_EMPLOYEE Expression Double-click on the TYPE_CODE port
This is the shortcut that goes directly to the port in the Edit View
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
96
•
Delete the port by using the
icon
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
97
•
Click on the OK button and the transformation objects should be the same as below
Lesson 9 – Copy Columns versus Link Columns
In the Mapping Designer, copying ports from one transformation to another is a valuable time saver. In lieu of entering the ports manually which includes the name, data type, precision and scale, the drag-and-drop method can be used by the developer to create a port in the desired transformation. The Designer includes a toggle feature to switch between “Copy Columns” and “Link Columns”. With Copy Columns, a port may be dragged and dropped from one transformation to another, capturing the integrity of the Name, Datatype, Precision and Scale. With Link Columns, if the desired port does not exist in the transformation to be linked, it will copy and link the port. If the port does exist, the port can simply be linked. In order for the link to be established between transformations, it must pass a validation check the Designer performs automatically. If the validation check does not pass, the port will be copied, only, and not linked. If only attempting to link ports the link will drop – the message can be found in the status bar at the bottom of the window.
•
Use the mouse to choose the copy columns icon
Copy columns
Link columns
• •
Use the mouse to choose the link columns icon Maneuver the transformations and adjust their sizes to all of the ports
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
98
As a result of the previous exercise, note the
TYPE_CODE port exists in the sq_EMPLOYEE transformation however, it does not appear as a port in the exp_ODS_EMPLOYEE transformation Copy Columns •
Choose the copy columns by selecting Layout | Copy
•
Columns or the Copy Columns icon Use the drag-and-drop method to copy the TYPE_CODE port from the sq_EMPLOYEE Source Qualifier to the exp_ODS_EMPLOYEE Expression
The exp_ODS_EMPLOYEE Expression displays the TYPE_CODE port just copied from the sq_EMPLOYEE Source Qualifier as shown above; however, in this mode no link should have been created. Link Columns •
Choose the link columns by selecting Layout | Link Columns
•
or the Link Columns icon Link the TYPE_CODE port from sq_EMPLOYEE Source Qualifier to the exp_ODS_EMPLOYEE Expression
Copy / Link Columns
There is no mode change here – the Link Columns mode can just link link
ports and copy /
ports as will be demonstrated here.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
99
•
Delete the TYPE_CODE port from the exp_ODS_EMPLOYEE Expression
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
100
•
Use the drag-and-drop method to copy / link the TYPE_CODE from the sq_EMPLOYEE Source Qualifier to the exp_ODS_EMPLOYEE Expression
Lesson 10 – Create Transformation Methods
Developers can create transformations using the following Designer tools: •
Mapping Designer. Create transformations that connect sources to targets. Transformations in a mapping cannot be used in other mappings unless Developers configure them to be reusable
•
Transformation Developer. Create individual transformations, called reusable transformations that Developers can use in multiple mappings
•
Mapplet Designer. Create and configure a set of transformations, called mapplets that Developers can use in multiple mappings
There are two ways to create transformations from scratch. There is the Transformation Toolbar that includes every Informatica transformation and then there is the menu option, Transformation | Create. When using the transformation toolbar developers will be able to click and drop transformations to a desired location in the workspace and it will also give it a default name. The Transformation | Create option forces the developer to select the desired transformation from a drop down list and the name must be typed in before the Designer creates the transformation.
Transformation Toolbar Method
•
Revert to Saved to reset the mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
101
•
On the Transformation toolbar, find the Aggregator Transformation
icon and single-click
Note when the mouse pointer hovers over a transformation icon in the toolbar that the name of the transformation object appears momentarily. In addition, some transformations will be shadowed meaning the transformation object is not available in the selected tool (e.g. Transformation Developer, Mapplet or Mapping Designer). •
Drag the mouse into the workspace
Note the arrow changes to crosshairs while in the workspace. •
Pick a location where the transformation should appear and single-click again
The selected transformation appears in the desired location of the workspace and the crosshairs change back to the arrow. The object has a default name that should be changed for easy identification during a debug run.
Transformation Menu Method •
Select Transformation| Create
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
102
•
Select the Aggregator from the drop down list
•
Select Create and give it the name agg_TargetTableName Click on the Done button and the new transformation appears in the workspace
•
Lesson 11 – Zoom and scale-to-fit
A mapping may be difficult to view in the workspace and the developer may want to change the magnification to see it better. Below are the features and techniques to adjust the magnification. •
Revert to Saved to reset the mapping
Zoom Out by using the Toolbar Option
There are features to change the magnification
of the contents of the workspace. Use the toolbar
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
103
or the Layout menu options to set zoom levels. The toolbar has the following zoom options: Zoom in based on rectangle
Drop down list
Scale to fit
Zoom in by 10% button
Zoom out by 10% button
Zoom in 10% on button •
Uses a point selected as the center point from which to increase the current magnification in 10 percent increments
Zoom out 10% on button. Uses a point selected as the center point from which to decrease the current magnification in 10 percent increments • • •
Click on the Zoom Out 10% on button on the toolbar Click anywhere in the workspace and the mapping will zoom out by 10% each time the mouse is clicked Keep clicking until the mapping is small enough to fit within the window
Zoom In by using the Toolbar Option
Zoom in 10% on button increases the current
magnification of a rectangular area selected. Degree of magnification depends on the size of the area selected, workspace size, and current magnification
• •
Click on the Zoom In 10% on button on the toolbar Click anywhere in the workspace and the mapping will zoom in by 10% each time the mouse is clicked.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
104
A developer can also zoom in or out by right
clicking in the mapping designer and clicking on zoom in or zoom out by the same 10% increments
Scale to Fit
Scale all objects to fit in the workspace •
Toggle off the Zoom In 10% on button
•
Click on the Scale to Fit
icon in the toolbar
Two other options are to 1) select the menu
option, Layout | Scale to Fit or 2) Right-click anywhere in the workspace and choose Scale to Fit. Layout Menu Options •
The layout menu has the following Zoom options
Zoom in Based on Rectangle
Zoom in based on rectangle increases the
current magnification of a rectangular area selected. Degree of magnification depends on the size of the area selected, workspace size, and current magnification.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
105
Zoom Center
Zoom Center maintains the center point of the workspace and zooms in or out by 10 percent increments. Zoom Point
Zoom Point uses a point selected as the center point from which to zoom in or out by 10 percent increments. Zoom Rectangle
Zoom Rectangle increases the current
magnification of a rectangular area selected. Degree of magnification depends on the size of the area selected, workspace size, and current magnification. Zoom Normal
Zoom Normal. Sets the zoom level to 100 percent.
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
106
Zoom Percent
Zoom Percent. Select the desired percent and Zoom in or out by that percent Lesson 12 – Designer Options
As mentioned in Lab Guidelines, screen shots
are based on columns that are not selected by default. All of the transformation objects used in this course have the following columns selected: Name, Datatype and Length/Precision. Others will include Expression. • • • •
•
Select Tools | Options Click on the Table Tab For the Source and Target, select Key Types, Name, Datatype, Length/Precision For the Source Qualifier, Update Strategy, Filter, Lookup, Joiner, Router, Mapplet and Sorter select Name, Datatype, Length/Precision For the Expression and Aggregator select Name, Expression, Datatype, Length/Precision
Informatica PowerMart / PowerCenter 6 Basics – Lab 7 – Features and Techniques I
107
Lab 8 4
LAB 8 – THE DEBUGGER PURPOSE •
Troubleshoot a mapping that is not producing correct data results. A fellow developer (student) has produced the same m_ODS_EMPLOYEE_xx mapping created in Lab 4. The mapping is valid. However, when the mapping is executed it is not producing correct results. Obtain a copy of the mapping then run a debug session to determine where the mapping errors are
OBJECTIVES •
Learn to use and apply the Informatica Debugger
D U R AT I O N •
20 minutes
Review the Problem The mapping created by the other developer produces database results as follows:
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
108
The results of the mapping should be the same as the expected results obtained in Lab 4:
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
109
The missing TYPE_CODES in the results above are Nulls. To verify, use the Preview Data feature on the ODS_EMPLOYEE target within the Designer to see the values. Prepare Designer These first steps will be to setup the Mapping Designer. Some of the steps may not be necessary because the Designer may still be open from the previous exercise. • •
Open Designer and connect to the Repository PC_Course Repository Open the assigned Studentxx folder
Clear Workspace
At the beginning of each new development
project (or Lab) it is a good practice to clear any unneeded objects from the Source Analyzer or Warehouse Designer Workspace (e.g. - from a previous project that will no longer be needed). If Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
110
the objects are not cleared, they may simply be unnecessary clutter in the Workspace. If the Workspace in both tools is already clear, skip the following four steps. •
The last tool present (Source Analyzer, Warehouse Designer, Mapping Designer, etc.) will have opened. If the Source Analyzer is not present in the work area, select it via the toolbar
•
Right-click and select Clear All to clear the Workspace
•
Change to the Warehouse Designer Clear the Workspace via the same method
•
Close Mapping A good practice to set up for any new development assignment (in this case, a lab) is to close the previously opened mapping. •
Change the Workspace to Mapping Designer as described in the flagged step Open the Mapping Designer on page 29Recent Mappings List
•
Locate the Recent Mapping list immediately above the Mapping Designer Workspace. If the mapping name is present in the Recent Mappings drop-down list, then it is still open. If not, skip to the Copy Mapping Step
The Recent Mapping list holds the mappings that have been opened since the last time the folder was closed (disconnected). A folder can be closed in several ways (e.g. Closing the Designer or Disconnecting from the repository or the folder) – any of which will net the same effect. •
Close the mapping by using one of the following methods: ° Select Mappings | Close; or °
Click on the Close Mapping
icon in the Toolbar
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
111
If a mapping is not closed prior to building a
new mapping or viewing another in the Workspace, the Designer will prompt whether or not to close the current one. If any changes were made to the mapping and not saved before opening or creating another, the Designer will NOT prompt to save changes until the folder is closed. Once a repository save is performed, any mappings that were opened in the Workspace and changed will be saved – it does not have to be performed individually. Anything saved will appear in the Output window.
Copy Mapping Setup The first step will be to obtain access to the errant mapping and make a copy of it to the assigned Studentxx folder. • Make sure the Workbook tab is selected from View | Workbook. It should look depressed. When depressed, the bottom of the Workspace will display all of the open folders – the only folder that should be opened is the assigned Studentxx •
Locate the Shared_Student folder in the Navigator window and expand it by clicking on the ‘+’ sign to the left of the folder name
If the Shared_Student folder appears as another Workbook tab next to the assigned Studentxx folder:
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
112
This tab should not be visible
Then the folder was opened in error per the instructions Please close (disconnect) by right clicking on the Shared_Student folder in the Navigator window and select Disconnect. Opening the folder when there is no intent to add or change anything within it is not a best practice because it is easy to perform work in the wrong folder (Workspace) if not careful. Refer to the Key Concept “Open Folder versus Expanded Folder”, in the flagged step Open Folder on page 12. •
Continue to drill down to the Mappings node in the same fashion as the folder and select the m_ODS_EMPLOYEE_BAD mapping
Copy Mapping – Edit | Copy or Ctrl-c Method •
Copy the mapping by doing one of the following: ° Select Edit | Copy; or
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
113
°
Ctrl-C
The m_ODS_EMPLOYEE_BAD mapping is in the Windows clipboard •
Paste the mapping by selecting the assigned Studentxx folder and doing one of the following: ° Select Edit | Paste; or ° Ctrl-V
The Designer displays the message to confirm the copy. Before proceeding, there is one other technique to practice. Copy Mapping – Mappings | Copy As… Method • •
Select the No button to practice another method to copy mappings Verify the Studentxx folder is active in the Workspace (or by viewing the Open Folder List – see below)
•
Select the Mapping Designer
•
•
In the Navigator Window, single-click the m_ODS_EMPLOYEE_BAD mapping from the Shared_Student folder Select Mappings | Copy As…
•
Type in the name, m_ODS_EMPLOYEE_BAD_xx, and click OK
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
114
The mapping is copied to the Studentxx Folder. There is one more method to try, so adding the mapping will not be saved at this point. •
Click on the assigned Studentxx Folder
•
Click on the Disconnect icon Click on the No button to not save changes
•
Copy Mapping – Drag and Drop Method •
Verify the Studentxx folder is active in the Workspace (or by viewing the Open Folder List – see below)
•
Select the Mapping Designer
•
In the Navigator Window, drag the m_ODS_EMPLOYEE_BAD mapping from the Shared_Student folder to the Workspace (DON’T DROP YET) – note the shortcut icon
•
as it enters the Workspace Click and hold the Ctrl key and note the copy icon while in the Workspace
•
Drop the mapping
This exercise requires the Ctrl key because the object is coming from a shared folder – one that allows shortcuts. Whenever an object is dragged from a shared folder to another, it is assumed that a shortcut is desired. An analogy is the Windows standard when using Explorer to copy a file – if the file is an executable (.exe), then it assumes a shortcut is desired. The Ctrl key, in both cases, changes it from a shortcut to a copy. Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
115
As previously stated, the key here is pay close attention to the object that is anchored to the arrow.
If the anchor is dark arrow pointing to the upper right, then the action will create a shortcut. While holding the Ctrl key, the anchor turns into a “+” sign which indicates a copy.
Regardless of which action is taken, the Designer provides a dialog box to confirm:
So, if is says ‘Create a shortcut…’, then try again •
After selecting Yes to confirm the copy, two more decision dialog boxes appear. Select Reuse for both:
And…
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
116
When copying a mapping, the mapping itself is
not the only object that appears in the target folder’s mapping node. Any object that exists in the original mapping that was pulled from other nodes such as the Sources and Targets nodes (other nodes to be discussed later) will be checked for duplicate names in the corresponding nodes where the mapping is being copied; hence, the decision boxes. • •
Collapse the Shared_Student folder (click on the ‘-‘ to the left of the folder name in the Navigator window) Save
Open Mapping In order to run the Debugger, the mapping to be debugged must be open in the Mapping Designer. It may already be open based on how it was copied into the assigned Studentxx folder from the previous step. Check for the name m_ODS_EMPLOYEE_BAD in the recent mapping list:
If not listed in the recent mapping list, then do as follows:
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
117
•
•
In the Navigator window, locate the m_ODS_EMPLOYEE_BAD mapping (just copied) in the mapping node of the Studentxx folder Click and hold the mouse button, then drag-and-drop the mapping into the Workspace
Notice the “Open this mapping” message in the status bar. Look to the status bar for other useful information as these sorts of tasks are being performed. Edit Mapping Change the name of the m_ODS_EMPLOYEE_BAD mapping to include the student assigned number to keep it unique – consequently it will have a unique session name and session log by default. • Select Mappings | Edit. The Edit Mappings dialog box opens
• •
Change the mapping name to m_ODS_EMPLOYEE_BAD_xx (xx represents the assigned student number) Save
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
118
Arrange Mapping • •
•
Right-click in the Workspace Select Arrange.
Select the ODS_EMPLOYEE target instance listed, click the Iconic checkbox and click the OK button
Activate the Debugger Toolbar Before proceeding, there is the Debugger toolbar that, by default, is not selected upon the PowerCenter client tool installation that looks as follows:
To add it, select Tools | Customize and check the Debugger checkbox on the Toolbars Tab.
Start Debugger •
Start the Debugger by doing one of the following: ° °
Click on the Start Debugger icon; or Select Mappings | Debugger | Start Debugger; or
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
119
°
•
Press the F9 key
Click on the Next button. The next Debug Mapping Wizard page appears
Server Select List
Instruct the Debugger Server to Create a Debug Session
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
120
• • •
Leave PC_Server as the Server to use for the debug Session – the default Server displayed on the pull-down list In the Session box, leave the default, Create a debug session instance for this mapping, radio button selected Click on the Next button
The Debugger spawns a server process, hence
the selection of the PC_Server server. Since no session has been created to represent this new mapping, the Debug Mapping Wizard has the ability to create one on the fly. This is especially useful when testing complex expressions and logic before trying to create a Workflow with a Session task.
Since the Session is being created on the fly, the Debugger needs to setup minimum parameters required for execution.
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
121
•
Select the same native connections used for the run of the Workflow in the flagged step Edit Session Task on page 57 ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select the Properties tab at the bottom and select Normal for Target load type
•
Click on the Next button to proceed Leave the default settings on this page of the Wizard and click on the Next> button
•
•
On the last page of the Wizard (not shown), leave the check in the Discard target data checkbox.
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
122
This option will take the data to the database to check for errors, but will not commit the rows •
Click on the Finish button and wait for the ‘spinning’ (initializing) of the Debugger proceeding
icon to stop before
Look in the upper right corner of the Debugger. The initialization process may take several seconds.
Rotate position of the Instance windows if desired via drag and drop method
Debugger Server Log
•
Real-time Session Log
Target Instance Window: Final data rows will appear
Instance Window: Data Rows will appear
Click on the Session Log tab to watch the Real-Time log creation
When running the Debugger, there are two very
important logs, the Debugger and Session Logs, to view in the Output window. They may be hidden and some size adjustments may need to be made
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
123
to the Output window to see it. Move the log window using the Double-vertical bars on its left side, and stretch it to make that Debugger and Session Log tabs visible. If that doesn’t do the trick, then the scroll bar may be in the way – that, too, may need adjustment. Click on the Session Log tab to watch the Real-Time log creation:
Furthermore, developers have a choice to position the windows in any of the three slots or undocked. Click up near the header (title) of the window and drag to the left or right to position them or hold the Ctrl key to prevent docking:
The Debugger toolbar will come become active when in debug mode.
Next Instance vs. Step to Instance Once a starting point (transformation) for the debug process is selected, the Next Instance icon allows the user to ‘follow’ a row through each transformation in the mapping. The Debugger will then proceed to the next row and follow it through each transformation.
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
124
Think of the two squares in the Next Instance icon as representing transformations, and a row stepping from one transformation to the next. The Step to Instance icon allows the user to examine one transformation at a time, following one-row-after-another through the same transformation. Think of the picture in the Step to Instance icon representing ‘steps’ – as if stepping down to the next row, in the same transformation.
Debug Mapping There are errors in the mapping. Select a transformation to serve as a starting point for the debugging: • Select the sq_EMPLOYEE Source Qualifier in the mapping. •
Click on the Next Instance icon (or F10) to start the movement row through the Debugger
Notice the large Yellow arrow that appears on
the Source Qualifier, sq_EMPLOYEE. This shows the focus of the debug action.
•
•
•
The smaller green arrow displaying a flashing yellow on the bottom of the Source Qualifier will continue to flash as long as rows are being read Continue to press F10 and notice the focus of the debug change as row-by-row, each row moves from the sq_EMPLOYEE to the exp_ODS_EMPLOYEE Examine the data in the Instance window and notice the pulldown list can be used to see a row in either of the two transformations in the mapping
Flipping back and forth between the
sq_EMPLOYEE and the exp_ODS_EMPLOYEE shows how the data has been transformed. If it appears to be different records, then the Next
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
125
Instance F10 needs to be pressed again to see the same row as it moves into the exp_ODS_EMPLOYEE – then flip back and forth to see how the record is transformed. • •
Try the ‘Step to Instance’ (Ctrl-F10) and watch each row through one transformation Compare the data from one transformation to the next and determine where the mapping errors are
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
126
Stop the Debugger Once a decision has been made on where data mapping has gone wrong, stop the Debugger: •
Click on the ‘Stop the Debugger’ for confirmation:
icon. The Designer will ask
•
Click on Yes. The Debugger Server will shutdown and return to the normal Mapping Designer Workspace
Examine the mapping to see if the anticipated errors exist. It is not necessary to correct the mapping for this exercise. It was introduced to encourage its use for the remaining exercises to assist the learning process.
Debugger Breakpoints •
Restart Debugger using exactly the same selections all the way to initialization (refer to the flagged steps Start Debugger on page 119)
•
Click on the Edit Breakpoint icon Select the sq_EMPLOYEE for the Instance Name at the top of the Edit Breakpoints dialog box Click on the Add button in the Breakpoints: section towards the bottom of the dialog box
• •
• • •
Click on the Add a new column to this table icon in the Condition section in the middle of the dialog box Select ‘<
>’ in the Ports Drop down list Select ‘ISNULL’ as the Operator
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
127
•
Double check the parameters above and click OK to save the breakpoint
•
Click the Continue
button
The Debugger will run through all of the rows until the first occurrence of a NULL is present. Note that two rows pass to the Target and then stops. With the sq_EMPLOYEE as the Instance, note the TYPE_CODE is NULL – there is a check in the NULL checkbox on the left. •
Click the Continue button, again
Ten rows enter the target and the debugger stops. Again, the TYPE_CODE is NULL. •
Click the Continue button, once again
NULL is not found in any other row – all rows continue to the target. • •
Click the Continue button, yet again for a final time, and the Debugger stops Edit the Breakpoints
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
128
Note the defined breakpoint is still present.
Breakpoints may be edited even when the Debugger is not running. When a repository save is performed, any breakpoints defined will remain associated with the mapping until they are deleted. •
Add additional breakpoints and experiment
Breakpoints may also be deactivated without being deleted – there is a checkbox to the left of the breakpoint. To see all of the Breakpoints, make sure ‘All Instances’ is selected at the top of the Edit Breakpoints Dialog box – from there it will be easier to determine which Breakpoints are needed for the debug run.
Informatica PowerMart / PowerCenter 6 Basics – Lab 8 – The Debugger
129
Lab 9
LAB 9 – FLAT FILE WIZARD AND FILTER PURPOSE •
Load a Time ODS Table. This table will contain filtered time data based on an Instructor-defined date
OBJECTIVE • •
Perform Source Analysis of a Flat File using the Flat File Wizard Learn to use the Filter Transformation
D U R AT I O N •
30 minutes
SUMMARY
Analyze a Flat File source •
In Designer, open the folder and open the Source Analyzer by clicking the Source Analyzer Source Analyzer
Icon or by selecting Tools |
Flat File Wizard Wherever a source flat file is located (Windows or UNIX) its definition must be imported into the repository via the Source Analyzer in the Designer tool. The definition for Flat Files may be imported into the Repository from local, mapped, or NFS mounted drives. If a flat file is on UNIX, then it is suggested to copy it via FTP to either a local or mapped drive (Windows-based operating
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
130
system). When the Session Task in the Workflow runs, the file will be read from the UNIX directory in which it is physically located. Once the file definition is in the repository, it can be discarded from the local or mapped drive because the Informatica PowerCenter Server will not look to client workstation for location information. When running a Session Task, always think about the location of the file from the Informatica PowerCenter Server’s perspective – never from the client workstation. • •
• •
Select Sources | Import From File Change the Files of type: to All Files (*.*), locate the directory (Instructor will provide location, and import the Flat File, TIME.txt.
Select Delimited Click on Import Field Names From First Line
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
131
Import field names from first line
The first row in the TIME.txt flat file contains
the column names. When the option is checked, it automatically starts the import at row 2 and assumes that future extracts will start at row 2. This, of course, can be changed if necessary without using the wizard (e.g. If the first extract contained column names – which is helpful to the developers who need to know – but future extracts do not, then this can be changed back to import from the 1st row.) • •
Click Next Make sure that Comma is set under the Delimiters section and the text Qualifier should be set to Double Quotes
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
132
• •
Click Next By importing the field names from the first line (Step 1), the port names have already been created. No other modifications are necessary
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
133
The Wizard takes a best guess at the Datatype, length and width based on the first 500 rows.
Refer to “Importing Flat Files” in the Designer Guide.
•
Click Finish
The Source Definition should look as follows. If any discrepancies exist, edit the Flat File definition and make the necessary changes. Otherwise, for practice, delete the definition, save the repository (Ctrl-s) and repeat the Flat File Wizard Steps 1 3.
•
Save the Repository by Ctrl-s or clicking Repository | Save
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
134
Import Target Table •
• • • • • •
•
Open the Warehouse Designer by clicking the Warehouse Designer icon on the toolbar Click on Targets | Import From Database Connect to your Target ODBC (tdbuxx - where xx is your student number) Expand the TDBU schema and TABLES folder Select ODS_TIME Click OK The ODS_TIME Target Transformation should look as follows. If any discrepancies exist, let your Instructor know
Save the Repository by Ctrl-s or clicking Repository | Save
Create Mapping •
Create a new mapping called m_ODS_TIME_xx (refer to the flagged step Create the Mapping on page 30)
When creating a new mapping, the Designer will prompt whether or not to close the current mapping in the workspace. Click on the Yes button when the prompted. Add Source and Target Transformations •
From the FlatFile database name under the Sources node, drag-and-drop TIME Source definition, into the Mapping Designer Workspace
Of course, the FlatFile database is not really a database, however, this is the chosen terminology in how Source Definitions are grouped. Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
135
•
From the Targets node, drag-and-drop ODS_TIME into the Mapping Designer Workspace
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
136
Create Expression Transformation
•
Create an Expression Transformation (refer to the flagged step Create the Expression Transformation on page 36)
Going
forward, when there is a picture below a Step Title (e.g. Create Expression Transformation), it reflects how the object should appear upon completion of the step. When transformations are linked, it will show the previous transformation and which columns are linked, input/output ports, etc. The bullets provide the details necessary to complete the entire step. Furthermore, the learning process should be in full swing and the steps will become more vague expecting that tasks have been learned. As the instructor stated, the rule of thumb is the first time step has been introduced in detail, the second time there is reference back to it and finally, the third time and beyond it is expected to have been learned. •
Select all ports within the sq_TIME transformation, then dragand-drop the cursor to the Expression Transformation just created
If the links do not appear, chances are Copy
Columns rather than Link Columns was selected. See flagged step Copy Columns and Link Columns on page 98 • • •
Double-click on the header bar of the Expression Transformation to go into edit mode On the Transformation Tab, to the right of the EXPTRANS name, click on the Rename button Type in exp_ODS_TIME and click the OK button
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
137
• •
Click on the Apply button Click on the Ports tab
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
138
•
Highlight the third port, Quarter, then add the following ports, each by clicking the Add Port
button
PORT DATATYPE LENGTH PORT TYPE DATE_ID date/timestamp 19 (default) Output Only MONTH_out string 3 Output Only QUARTER_out string 2 Output Only YEAR_out string 4 Output Only An expression will be needed for each of the above ports. Open the expression editor and add expressions for each port as follows (See flagged step Create an Expression on page 40).
As mentioned earlier, formulas can be typed in manually, however, it’s recommended to use the tabs in the Expression Editor’s Navigator Window to build them. Select the required functions (TO_DATE and SUBSTR) and ports (Date and Quarter) from their respective tabs when building these four separate expressions. This is a best practice, especially, when building complex expressions with nested functions using multiple ports. It will make debugging expressions easier. PORT DATE_ID MONTH_out QUARTER_out YEAR_out • •
FORMULA TO_DATE(DATE,’DD-MON-YYYY’) SUBSTR(DATE,4,3) SUBSTR(QUARTER,6,1) SUBSTR(QUARTER,1,4)
Change the ports DATE and QUARTER to input ports only Examine the Ports Tab of the exp_ODS_TIME Expression – it should resemble as follows:
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
139
•
Click on the OK button
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
140
Create Filter Transformation
•
Click on the Filter Transformation toolbar
•
Drag the pointer, which now appears as crosshairs , into the Workspace window to the right of the exp_ODS_TIME Expression Single-click (or, click-and-drag, which will draw a box, then release) Drag and drop the following ports from exp_ODS_TIME to the FILTRANS window: DATE_ID DAY_OF_WEEK MONTH_out QUARTER_out YEAR_out Double-click on the Filter Transformation Under the Transformation tab click the Rename button and enter fil_ODS_TIME
• •
• • •
•
icon on the Transformation
Click on the Ports tab and remove the ‘_out’ from each of the three ports (before = aft): MONTH_out = MONTH QUARTER_out = QUARTER YEAR_out = YEAR Click on the Properties tab
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
141
•
Click on the arrow on line 1 (Filter Condition) to the far right in the Value column to open the Expression Editor Click here to open Expression Editor
• • •
• • •
To specify the Filter Condition, highlight TRUE and then press the delete key to remove the default condition Click on the Ports Tab in the Navigator Window Double-click on DATE_ID under the fil_ODS_TIME transformation Click the greater than or equals button on the Operator Keypad Click the Functions Tab, expand the Conversion folder and double-click on To_Date To complete the expression, type '12/04/1999', 'MM/DD/YYYY' between the parentheses. The final expression should look as follows: DATE_ID >=TO_DATE('12/04/1999', 'MM/DD/YYYY')
• •
Click on the Validate button to validate the filter condition Click OK to save changes
Link Target Transformation
•
Use the Autolink by Name or Position feature to link the transformation objects. Select fil_ODS_TIME for the From Transformation and ODS_TIME for the To Transformations. Click OK. All ports should be linked between these two transformations (shown above)
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
142
•
Validate the mapping by selecting Mapping |Validate
Remember to view the Validate Tab in the Output window for the results. •
Save
Completed Mapping
Create Workflow
Complete the following steps (refer to Start Workflow Manager on page 50) ° Start Workflow Manager ° Connect to Repository ° Open Folder •
Create Workflow, wf_ODS_TIME_xx, via Workflows | Create
•
Edit wf_ODS_TIME_xx Workflow: ° Properties tab, add the wf_ODS_TIME_xx.log as the Workflow Log File Name ° Metadata Extensions tab, add creation date and your name in the respective reusable metadata extensions Edit Start Task:
•
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
143
Rename Start task to start_ODS_TIME ° Create Session Task called s_m_ODS_TIME_xx Edit s_m_ODS_TIME_xx session ° Task – Sources/Properties tab, verify Source filename is time.txt ° Task – Targets/Connections tab, select native_target_xx for ODS_TIME target table ° Targets (top)/Properties (bottom), check Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2 - Targets/Properties tab, select Normal for Target load type Link start_ODS_TIME to s_m_ODS_TIME_xx Session Task Save, Validate and Start Workflow from Selected Task, start_ODS_TIME Monitor and Review the results for s_m_ODS_TIME_xx in the Workflow Monitor °
•
• • •
Results – Session Properties •
By using the date 12/04/1999, the ODS_TIME_xx table should contain 1489 rows.
Expected Results
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
144
Results – Session Transformation Statistics
Results – Preview Data Use the Preview Data option in the Designer to preview Target Data • Return to the Designer and either go to: ° Mapping Designer; or ° Warehouse Designer • Right-click on ODS_TIME target definition and select Preview Data…
The above dialog box is similar to the ones used to connect to the source and target database. In this case the connection needs to be made to the target database. Login to the target database in the same manner; refer to the flagged step Analyze Source Definitions on page 16.
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
145
• • •
Select the odbc_target ODBC data source connection from the pull-down list Enter the Username and Password assigned by the Instructor – should be tdbuxx / tdbuxx (username / password) Click on Connect and the results appear in the bottom half of the dialog box. The default number of returned rows is 100, however, for the exercises going forward, only the first few rows will be shown in the Results sections.
Informatica PowerMart / PowerCenter 6 Basics – Lesson 9 – Flat File Wizard and Filter
146
Lab 10
LAB 10 – HETEROGENEOUS JOIN PURPOSE •
The purpose of this lab is to populate a fact table by joining a flat file and a relational table. The flat file has orders to vendor’s data for products and supplies. The product table has data on the make and model name, the vendor that supplies the product, and the cost. We want to load an ODS table with the costs summarized by date, product and vendor. This will be the raw data that will be used to populate the fact table.
OBJECTIVE • •
Learn how to perform a heterogeneous join using the Joiner transformation Learn how to aggregate data using the Aggregator Transformation
D U R AT I O N •
35 minutes
SUMMARY SOURCES: PRODUCT table, ORDER flat file TARGET: ODS_ORDER_AMOUNT table
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
147
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
148
Import Sources
• •
• •
Clear Source Analyzer Workspace (Right-click anywhere in the Workspace and select Clear All) Continue to work in the assigned student folder and import tab delimited flat file, ORDER.txt. The first row includes the field names, so check the Import Field Names From First Line box (see flagged step Analyze a Flat file source on page 130) Import relational table, PRODUCT Save Repository
Import Target • • •
Clear Warehouse Designer Workspace (Right-click anywhere in the Workspace and select Clear All) Import the relational database target, ODS_ORDER_AMOUNT Save Repository
Create Mapping •
Create a mapping called m_ODS_ORDER_AMOUNT_xx
No more hints on how to create a mapping –
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
149
Add Sources and Target
•
Add both order and product source definitions with their respective Source Qualifiers
Make sure each source has its own Source
Qualifier; if they do not, then they will have to be created manually. Refer to Lab 4 on page 29. •
Add target definition, ODS_ORDER _AMOUNT
•
Save the mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
150
Click on the Navigator window and Output
window toggle buttons which are located next to folder name as seen below, to minimize these two windows.
Navigator Window Toggle Button
Output Window Toggle Button
This will expand the Workspace, which will make it easier to work with all of the mapping objects. The Output Window will automatically reappear when information is added (e.g. Saving the repository or Mapping Validation)
Create Joiner Transformation
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
151
While
a Source Qualifier transformation can join data originating from a common source database, the Joiner transformation joins two related heterogeneous sources residing in different locations or file systems. The combination of sources can be varied. The following sources are allowed: •
Two relational tables existing in separate databases
•
Two flat files in potentially different file systems
•
Two different ODBC sources
•
Two instances of the same XML source
•
A relational table and a flat file source
•
A relational table and an XML source •
Create Joiner Transformation and name it jnr_ODS_ORDER_AMOUNT
The
Joiner Transformation can only join two disparate (heterogeneous) sources. If three or more heterogeneous sources are needed to complete the mapping, then multiple Joiner transformations will be needed and they will be “nested” – the results from one Joiner will be passed on to the next and so forth. In addition, the Joiner needs a minimum of two ports, one from each source to create a join condition. Those two ports must be compatible data types and length to for the join condition to be successful. One of the two sources in each Joiner must be deemed the Master and the other, by default, will become the Detail. The Master is designated by the “M” port attribute, and is essentially cached to memory. If not enough memory is allocated on the Properties tab (discussed later in the course) then the spillover will cache to disk and ultimately degrade performance. To determine which source should be the Master, it is recommended to choose the smaller (total bytes; not necessarily rows) of the two since it will take less time and use less system resources to cache. To calculate which source will cache less between the two, factor in the number of ports, their respective data types, precision and scale and THEN, multiply that by the expected number of rows.
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
152
Ports PRODUCT_CODE VENDOR_ID PRICE COST ORDER_DATE PRODUCT QUANTITY
Port Precision (Width) for: PRODUCT ORDER 10 10 10 10 11 10 1
Total Port Width Total Rows
40 80
Port Width x Rows 3200 •
• • •
22 200 4400
Add the following ports to jnr_ODS_ORDER_AMOUNT from: ° sq_PRODUCT (PRODUCT_CODE, VENDOR_ID, PRICE, COST) ° sq_ORDER (ORDER_DATE, PRODUCT, QUANTITY) Edit jnr_ODS_ORDER_AMOUNT – Ports tab, select the ports from the Product as the Master On the ports tab, increase size of PRODUCT port to a precision of 10. On the Condition tab, add one condition where PRODUCT_CODE = PRODUCT by clicking on the Add a new condition
•
button
On the Properties tab, confirm the Join Type is Normal There are several different Join Types available to choose. There is the Normal Join (inner), Master Outer Join (left), Detail Outer Join (right), and Full Outer Join. See Defining the Join Type in Chapter 6 of the Transformation Guide.
•
Save the mapping
Create Expression Transformation
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
153
• • • • •
Create an Expression Transformation called exp_ORDER_DATE Copy/Link ORDER_DATE port from the jnr_ODS_ORDER_AMOUNT to exp_ORDER_DATE Rename ORDER_DATE port to ORDER_DATE_in Add a new port called ORDER_DATE_out with the Datatype, date/time Create an expression as follows: TO_DATE(ORDER_DATE_in, 'DD-MON-YYYY')
Create Aggregator Transformation
The
Aggregator transformation allows aggregate calculations, such as averages and sums. The Aggregator transformation is unlike the Expression transformation, in that it can be used to perform calculations on groups. The Expression transformation permits calculations on a row-by-row basis only. •
Create an Aggregator Transformation ( agg_ODS_ORDER_AMOUNT
icon) and name it
•
Add the ORDER_DATE_out port from the exp_ORDER_DATE to the agg_ODS_ORDER_AMOUNT
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
154
•
Add the following ports from jnr_ODS_ORDER_AMOUNT to agg_ODS_ORDER_AMOUNT and arrange them in this specified order following the ORDER_DATE_out port: PRODUCT_CODE, VENDOR_ID, PRICE, COST, QUANTITY
It is important to specify how to group the data
to do aggregate calculations. The group by ports can be input, output, or variable ports. To define a group, check the GroupBy checkbox for each port required on the Ports tab in the aggregator. The order of the ports from top to bottom determines the group by orders, so move them around if necessary. In addition, when grouping data the last row of each group is output, unless otherwise specified in an Aggregator Expression. The “Expression” section of the port will be enabled, turn white and logic can be inserted in the Expression Editor section of the port for output and variable ports •
Rename ORDER_DATE_out to ORDER_DATE
•
Edit agg_ODS_ORDER_AMOUNT – Ports tab, group the data by: ° ORDER_DATE ° PRODUCT_CODE ° VENDOR_ID On the Ports tab, append _in to the QUANTITY port. The end result will be QUANTITY_in. In addition, make this an Input only port by turning off the Output port On the Ports tab, create an output port called QUANTITY_out with a Datatype of “integer” Build an expression should result in the sum of the QUANTITY_in
•
• •
SUM(QUANTITY_in)
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
155
•
Save the mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
156
Link Target Definition
•
Use the Autolink feature to link agg_ODS_ORDER_AMOUNT to ODS_ORDER_AMOUNT. All of the port names are the same as the target except QUANTITY_out. Try By Name with _out in the From Transformation Suffix for practice and click on the OK button (see flagged step Autolink by Name on page 89)
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
157
Completed Mapping
•
Save the mapping and check for validation information on the Save tab in the output area. If valid, proceed to the Workflow Manager.
Create Workflow
•
Create Workflow called wf_ODS_ORDER_AMOUNT_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
158
• • • • • • •
• • •
Add wf_ODS_ORDER_AMOUNT_xx.log as the Workflow Log File Name Add creation date and your name in the respective reusable metadata extensions Rename Start task to start_ODS_ORDER_AMOUNT Create Session Task called s_m_ODS_ORDER_AMOUNT_xx Select native_source connection for sq_PRODUCT source instance Verify Source filename is order.txt (Properties tab) Select native_target_xx connection for ODS_ORDER_AMOUNT target instance ° Check property Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link start_ODS_ORDER_AMOUNT to s_m_ODS_ORDER_AMOUNT_xx Save, Validate and Start Workflow wf_ODS_ORDER_AMOUNT_xx Monitor and Review the results for s_m_ODS_ORDER_AMOUNT_xx in the Workflow Monitor
Results – Session Properties
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
159
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
160
Results – Session Transformation Statistics
Results – Preview Data
Only the first few rows are shown.
Informatica PowerMart / PowerCenter 6 Basics – Lab 10 – Heterogeneous Join
161
Lab 11
LAB 11 – SORTER PURPOSE •
The purpose of this lab is to populate an ODS table by joining a flat file and a relational table. It’s the same table as the previous lab, however, assuming there is a large amount of data broken down into several groups, the data will flow through the mapping much faster through the Aggregator if the data is sorted. Since the data is not coming, solely, from a relational source, grouping the data in the Source Qualifier will serve very little purpose. The addition of the Sorter Transformation and a property in the Aggregator has the potential to significantly enhance performance.
OBJECTIVE • • •
Reiterate the Copy Mapping Process Learn how to use the Sorter Transformation Learn the purpose of the Aggregator Transformation’s Sorted Input property
D U R AT I O N •
20 minutes
SUMMARY SOURCES: PRODUCT table, ORDER flat file TARGET: ODS_ORDER_AMOUNT table
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
162
Copy Mapping The first step will be to copy the previous mapping • Copy the mapping m_ODS_ORDER_AMOUNT_xx (See flagged step Copy Mapping on page 112) and name it m_ODS_ORDER_AMOUNT_sorter_xx • Open m_ODS_ORDER_AMOUNT_sorter_xx mapping in the Workspace • Save Create Sorter Transformation
• • •
Create a Sorter transformation and name it srt_ODS_ORDER_AMOUNT Add ORDER_DATE_out port from exp_ORDER_DATE Add QUANTITY, PRODUCT_CODE, VENDOR_ID, PRICE and COST ports
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
163
When inserting another transformation between
transformations that are already linked it is easier to get a visual of what ports need to go where. So, rather than removing the input links from the agg_ODS_ORDER_AMOUNT transformation, add the links to the one that needs them, first. • •
Rename ORDER_DATE_out to ORDER_DATE Check the key check box for the ORDER_DATE, PRODUCT_CODE, VENDOR_ID ports and make sure the ports are in that order
The intent is to match the grouping in the Aggregator •
Save
Remove Links •
Remove all input links to agg_ODS_ORDER_AMOUNT
This will be the location where the Sorter Transformation will be inserted Add Links
• • •
Autolink by name srt_ODS_ORDER_AMOUNT to agg_ODS_ORDER_AMOUNT Manually link QUANTITY from srt_ODS_ORDER_AMOUNT to QUANTITY_in in agg_ODS_ORDER_AMOUNT Save
Set Sorted Input Property
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
164
Since the Sorter Transformation sorts the data, the Aggregator has a property to keep the data flowing called Sorted Input. It makes no difference if the database does the sorting or the Sorter Transformation, however if the property is checked the, Aggregator expects the data to be sorted. How this property keeps the data flowing is after the last row of a particular group (defined by the Aggregator GroupBy checkboxes) enters it’s respective memory cache, the aggregation for that group begins and a new memory cache is created for the next group and so on. Without this property, each and every row would have to enter the memory cache before aggregation would occur. This potentially could produce a very large bottleneck. •
Check the Sorted Input checkbox on the Properties tab in the agg_ODS_ORDER_AMOUNT Transformation
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
165
Completed Mapping
Create Workflow
• • • • • • •
•
Create Workflow called wf_ODS_ORDER_AMOUNT_sorter_xx Add wf_ODS_ORDER_AMOUNT_sorter_xx.log as the Workflow Log File Name Rename Start task to start_ODS_ORDER_AMOUNT_sorter Create Session Task called s_m_ODS_ORDER_AMOUNT_sorter_xx Select native_source connection for sq_PRODUCT source instance Verify Source filename is order.txt (Properties tab) Select native_target_xx connection for ODS_ORDER_AMOUNT target instance ° Check property Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link start_ODS_ORDER_AMOUNT_sorter to s_m_ODS_ORDER_AMOUNT_sorter_xx
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
166
• •
Save, Validate and Start Workflow wf_ODS_ORDER_AMOUNT_sorter_xx Monitor and Review the results for s_m_ODS_ORDER_AMOUNT_sorter_xx in the Workflow Monitor
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
167
Results – Session Properties 103 rows loaded (same as previous lesson) Results – Session Transformation Statistics
Results – Preview Data
Only the first few rows are shown.
Informatica PowerMart / PowerCenter 6 Basics –Lab 11 – Sorter
168
Lab 12
LAB 12 – BASIC LOOKUP PURPOSE •
To load an ODS table with only those customers who have one or more completed, non-cancelled, invoices on record
OBJECTIVE •
To demonstrate the use of a Lookup transformation where a Source Qualifier SQL override or a Joiner may not be the best solution
D U R AT I O N •
40 minutes
SUMMARY SOURCE: TARGET:
INVOICE table ODS_CUSTOMER_ACTIVE table
Import Source Table
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
169
• •
Clear Source Analyzer Workspace – Right-click anywhere in the workspace and select Clear All Continue to work in the assigned student folder and import the relational source, INVOICE
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
170
Import Target Table
•
Import the target table ODS_CUSTOMER_ACTIVE
Create Mapping •
Create a mapping called m_ODS_CUSTOMER_ACTIVE_xx
Add Source and Target •
Add INVOICE Source Definition with its respective Source Qualifier
•
Add ODS_CUSTOMER_ACTIVE target definition
Use the Navigator and Output Widow toggle buttons, again, to maximize the Workspace real estate.
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
171
Create Filter Transformation
•
•
•
Create a Filter Transformation called fil_ODS_CUSTOMER_ACTIVE (see flagged step Create Filter Transformation on page 141) Add the following ports from sq_INVOICE to fil_ODS_CUSTOMER_ACTIVE: CUST_NO, DATE_CLOSED and CANCELED Edit fil_ODS_CUSTOMER_ACTIVE ° Click on the Properties tab and add the following Filter Condition: NOT (ISNULL (DATE_CLOSED)) AND CANCELED = 0
Only those invoices that have a date in the DATE_CLOSED column and were not cancelled will meet the filter condition Click Validate or OK to validate the condition Close the Edit Transformations window Save °
• •
Create Lookup Transformation
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
172
Connected vs. Unconnected Lookups In this step a connected Lookup is needed in order to bring back specific existing Customer data where a Joiner or SQL override will not work correctly. Like a number of other transformations, Lookups can be configured to be ‘connected’ or ‘unconnected’.
•
Add a Lookup box appears:
Transformation to the mapping and a dialog
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
173
•
•
Since the CUSTOMER table has not yet been analyzed or imported as a Source or a Target, select the Import button (as shown above) Enter the logon information needed to connect to the odbc_source location (see flagged step Analyze Source Definitions on page 16)
•
Expand the SDBU Tables list and select the table CUSTOMER
•
Click OK and a Lookup transformation window will appear in the workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
174
•
Rename LKPTRANS to lkp_CUSTOMER and exit – an error message appears stating: ”The lookup condition is empty…” (See below)
If any edits are made to the Lookup Transformation and no condition has been entered prior to clicking on the Apply or OK buttons, the following error will appear:
There is no problem with clicking on Yes to continue, however, the mapping will not be valid until a condition is entered. A similar error message would occur with the Joiner (previous exercise) and Router (discussed later) Transformations – as these, too, require valid conditions.
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
175
•
Copy/Link CUST_NO
from
fil_ODS_CUSTOMER_ACTIVE to an empty or output port in lkp_CUSTOMER – the icon will have the plus sign tagged at the bottom as it enters the Lookup Transformation
• •
Double-click on the header of the Lookup transformation. On the Condition tab choose Add a new condition button (see picture below)
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
176
The Designer will automatically choose the ports to compare based on the sequential order of Lookup ports and Input ports. •
Select CUST_ID and CUST_NO for the condition
The
Lookup, like the Joiner and the Aggregator discussed earlier, has the ability to cache the rows to memory (and disk). There are several caching options to be discussed later in the course; however, when a lookup is connected (every source row passes through), it is a Best Practice to choose one of many caching options to obtain the best performance. •
Change the Location Information on the Properties tab to $Source
In this example, since the source and lookup tables come from the same Database (Schema), the $Source variable may be used. This means that when native database connection is chosen for the Source Instance in the Session Task, the Lookup will use the same one.
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
177
Essentially, the $Target variable does the same thing except the location of the lookup table would have to be in the same location as the target. Choosing either of the two variables is a Best Practice because it provides the most flexibility. However, when neither is the correct Database Location then it must be hard-coded. Note that in the listing (see Select Database dialog box below) just below $Source and $Target variables are all of the available database connections. odbc_source represents the Data Source Name (DSN) from where the Lookup table was imported – the Informatica PowerCenter Server will not recognize this as a valid database connection and must be changed
Click here to open Select Database Dialog Box
Select $Source variable
•
After selecting the Location Information, close the Edit Transformations Dialog Box and Save
Link Target Definition
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
178
• •
Autolink by name the lkp_CUSTOMER to ODS_CUSTOMER_ACTIVE Save the mapping
Completed Mapping
Create Workflow
• • • • • •
Create Workflow called wf_ODS_CUSTOMER_ACTIVE_xx Add wf_ODS_CUSTOMER_ACTIVE_xx.log as the Workflow Log File Name Rename Start task to start_ ODS_CUSTOMER_ACTIVE Create Session Task called s_m_ODS_CUSTOMER_ACTIVE_xx Select native_source connection for sq_INVOICE source instance Select native_target_xx connection for ODS_CUSTOMER_ACTIVE target instance ° Check property Truncate target table option
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
179
Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link start_ ODS_CUSTOMER_ACTIVE to s_m_ODS_CUSTOMER_ACTIVE_xx Save, Validate and Start Workflow wf_ODS_CUSTOMER_ACTIVE_xx Monitor and Review the results for s_m_ODS_CUSTOMER_ACTIVE_xx in the Workflow Monitor °
• • •
Results – Session Properties
Results – Session Transformation Statistics
The number of Failed or Rejected Rows indicates how many were rejected by the database. In this example, duplicate rows were rejected because of a unique constraint violation. In general, building mappings, their respective workflows and testing
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
180
them are an iterative process. It is not recommended to leave a mapping or workflow in this state; however, there are several other features that have not been covered to apply best practices. The number listed in the Target Success Rows is accurate, however, there are ways to build mappings to identify rows that will be rejected by the database before they get there; hence reducing the Target Failed Rows to zero. Some of those features will be addressed in this basic course and others will be left for the advanced course. Results – Preview Data
Only the first few rows are shown.
Informatica PowerMart / PowerCenter 6 Basics – Lab 12 – Basic Lookup
181
Lab 13
LAB 13 – DELETING ROWS PURPOSE •
To delete rows from an ODS table where customers do not have valid type codes, i.e., they have not been properly ‘typed’ or are marked INACTIVE
OBJECTIVE •
To demonstrate how to run a session that only deletes rows
D U R AT I O N •
25 minutes
SUMMARY SOURCE: TARGET:
INVOICE table ODS_CUSTOMER_ACTIVE table
Copy mapping • •
• •
Expand the Mappings node in the assigned folder in the Navigator window Select the m_ODS_CUSTOMER_ACTIVE_xx and hit Ctrl-C and the Ctrl-V (See flagged step Copy Mapping on page 112 for other methods of copying mappings) Click yes to copy the mapping Add ‘_del’ to the mapping name before the student number, xx:
Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
182
•
Open the new mapping in the Workspace
Add Second Filter Transformation •
Delete the links between the lkp_CUSTOMER and the ODS_CUSTOMER_ACTIVE before adding a new filter
Click-and-drag the mouse pointer over the blue link lines, only, between the lkp_CUSTOMER and the ODS_CUSTOMER_ACTIVE and note the lines change to red. If any other objects (e.g.: transformations) were selected along with the lines, redo the process ° Press the Delete key to remove the connections Add a Filter Transformation to the mapping and give it the name fil_ODS_CUSTOMER_ACTIVE_delete (See flagged step Create Filter Transformation on page 141 °
•
Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
183
•
•
Add the following ports from lkp_CUSTOMER to fil_ODS_CUSTOMER_ACTIVE_delete: CUST_ID, CUST_NAME, FIRST_CONTACT, CUSTOMER_STATE, CUSTOMER_ZIP, CUSTOMER_TYPE and DISCOUNT_RATE Edit fil_ODS_CUSTOMER_ACTIVE_delete – Properties tab, add the following condition: ISNULL(CUSTOMER_TYPE) OR UPPER(RTRIM(CUSTOMER_TYPE)) = 'INACTIVE'
The UPPER function takes the string value and changes it to upper case.
In UNICODE mode, RTRIM removes both single- and double-byte spaces from the end of a string. In ASCII mode, RTRIM removes only single-byte spaces. • •
Click on Validate or OK to validate the filter condition Close the Edit Transformations window and Save
Link Target Definition
Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
184
• •
Autolink by name the fil_ODS_CUSTOMER_ACTIVE_delete to ODS_CUSTOMER_ACTIVE Save
Completed Mapping
Create Workflow
• • • • • •
Create Workflow called wf_ODS_CUSTOMER_ACTIVE_del_xx Add wf_ODS_CUSTOMER_ACTIVE_del_xx.log as the Workflow Log File Name Add reusable metadata extensions: creation date and your name Rename Start task to start_ODS_CUSTOMER_ACTIVE_del Create Session Task called s_m_ODS_CUSTOMER_ACTIVE_del_xx Select Task – Properties - Treat source rows as delete
The above is a key property in the Session Task that must be set to get the expected results. In this case, the objective is to delete rows, hence the property will be changed to delete • •
Select native_source connection for sq_INVOICE source instance Select native_target_xx connection for ODS_CUSTOMER_ACTIVE target instance ° do not Truncate
Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
185
Since the table is suppose to have rows to
delete, do not Truncate the table as suggested in previous Session Tasks. Select Normal for Target load type property Link start_ ODS_CUSTOMER_ACTIVE_del to s_m_ODS_CUSTOMER_ACTIVE_del_xx Save, Validate and Start Workflow wf_ODS_CUSTOMER_ACTIVE_del_xx Monitor and Review the results for s_m_ODS_CUSTOMER_ACTIVE_del_xx in the Workflow Monitor °
• • •
Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
186
Results – Session Properties
Results – Session Transformation Statistics
If more than 3 rows (shown above as Affected
Rows) were deleted, then fix the logic in the mapping or session. Before rerunning this workflow, the workflow from the previous exercise must be executed. In summary: 1) Fix logic in the m_ODS_CUSTOMER_ACTIVE_del_xx mapping 2) Rerun wf_ODS_CUSTOMER_ACTIVE_xx workflow to repopulate the 28 rows; and finally 3) Rerun wf_ODS_CUSTOMER_ACTIVE_del_xx workflow Informatica PowerMart / PowerCenter 6 Basics – Lab 13 – Deleting Rows
187
Lab 14
LAB 14– DATA DRIVEN INSERTS AND REJECTS PURPOSE •
In the previous labs filtering out all of the INACTIVE customers created the ODS_CUSTOMER_ACTIVE table. In the Customers table CUSTOMER_TYPE is either set to INACTIVE or ACTIVE. Because Customers may change from ACTIVE to INACTIVE, Labs 12 and 13 would have to be run every time ODS_CUSTOMER_ACTIVE is loaded. The use of an Update Strategy eliminates the need for two mappings to accomplish the same task.
OBJECTIVE •
Create and configure an Update Strategy. Then do inserts and rejects based on whether the customer type is ACTIVE or INACTIVE
D U R AT I O N •
30 minutes
SUMMARY SOURCE: TARGET:
INVOICE table ODS_CUSTOMER_ACTIVE table
Copy Mapping •
Copy the mapping m_ODS_CUSTOMER_ACTIVE_xx and name it m_ODS_CUSTOMER_ACTIVE_upd_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
188
Create Update Strategy Transformation
Use an Update Strategy Transformation when keeping history information in a table or updating a table to keep most current information. In an Update Strategy a record can be flagged to be updated, inserted, deleted, or rejected using the following constants: DD_UPDATE, DD_INSERT, DD_DELETE, and DD_REJECT, respectively.
•
Delete the links between the lkp_CUSTOMER Lookup and ODS_CUSTOMER_ACTIVE Target Definition
•
Create an Update Strategy Transformation and name it upd_ODS_CUSTOMER_ACTIVE Add the Following ports from lkp_CUSTOMER to upd_ODS_CUSTOMER_ACTIVE: CUST_ID, CUST_NAME, FIRST_CONTACT, CUSTOMER_STATE, CUSTOMER_ZIP, CUSTOMER_TYPE and DISCOUNT_RATE
•
Update Strategy Expression Pseudo Code IF (CUSTOMER_TYPE= ‘INACTIVE’ OR NULL THEN REJECT record ELSE INSERT record) IIF statement logic: Condition CUSTOMER_TYPE = ‘INACTIVE’ or NULL (BLANK)
Action Description REJECT The CUSTOMER_TYPE is ‘INACTIVE’ or NULL (BLANK), it will be rejected.
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
189
CUSTOMER_TYPE=’ACTIVE’ INSERT The CUSTOMER_TYPE is ‘ACTIVE’. The row will be inserted.
Informatica has a set of constants used in the Update Strategy to flag the rows that pass through. They are:
DD_UPDATE flags row for update; DD_INSERT flags row for insert; DD_DELETE flags row for delete; and DD_REJECT flags row for reject. The ‘DD’ stands for Data Driven. DD_DELETE, DD_INSERT, DD_REJECT and DD_UPDATE are not listed in the functions tab, so they must be manually entered in the formula. •
Edit upd_ODS_CUSTOMER_ACTIVE – properties tab, add the following expression: IIF(UPPER(RTRIM(CUSTOMER_TYPE))='INACTIVE' or ISNULL(CUSTOMER_TYPE), DD_REJECT, DD_INSERT)
• • •
Click Validate button to validate the formula Click OK button Save the repository
Link Target Definition
•
Autolink by name the upd_ODS_CUSTOMER_ACTIVE Update Strategy to ODS_CUSTOMER_ACTIVE Target Definition
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
190
•
Save
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
191
Completed Mapping
Create Workflow
• • • • •
• •
• • •
Create Workflow called wf_ODS_CUSTOMER_ACTIVE_upd_xx Add wf_ODS_CUSTOMER_ACTIVE_upd_xx.log as the Workflow Log File Name Rename Start task to start_ ODS_CUSTOMER_ACTIVE_upd Create Session Task called s_m_ODS_CUSTOMER_ACTIVE_upd_xx Verify Task – Properties - Treat source rows as Data Driven (this is the default entry when an Update Strategy Transformation is present) Select native_source connection for sq_INVOICE source instance Select native_target_xx connection for ODS_CUSTOMER_ACTIVE target instance ° Truncate Table ° Select Normal for Target load type property Link start_ ODS_CUSTOMER_ACTIVE_upd to s_m_ODS_CUSTOMER_ACTIVE_upd_xx Save, Validate and Start Workflow wf_ODS_CUSTOMER_ACTIVE_upd_xx Monitor and Review the results for s_m_ODS_CUSTOMER_ACTIVE_upd_xx in the Workflow Monitor
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
192
Results – Session Properties
Results – Session Transformation Statistics
The above results still do not resemble the Best Practice – that is, the Database “unique constraint errors” and the Rejected Rows are still present. There will be other lessons on managing rows; however, additional topics need to be covered. There is, however, a simple solution to avoid them – see the Extra Credit for more details.
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
193
Results – Preview Data
Extra Credit To avoid the database unique constraint violations, there is a very simple solution. Try the following this exercise to eliminate them.
Consider there are 200 invoices and 30 customers. The objective was to identify active customers. With the Invoice Table as the source and Customer table as the lookup, there may be several invoices per customer. Based on the criteria, it only takes one invoice to pass through the lookup. Since several are passing through, the database rejects them as unique constraint violations. What transformation can take several rows and group them together as one? It’s the Aggregator, of course.
•
Using the same mapping, m_ODS_CUSTOMER_ACTIVE_upd_xx, insert an Aggregator called agg_ODS_CUSTOMER_ACTIVE between the fil_ODS_CUSTOMER_ACTIVE Filter and the lkp_CUSTOMER Lookup Procedure
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
194
•
•
•
•
Copy/Link the CUST_NO port from the fil_ODS_CUSTOMER_ACTIVE Filter to agg_CUSTOMER_ACTIVE Aggregator Link the CUST_NO port from the agg_CUSTOMER_ACTIVE Aggregator to the CUST_NO port in the lkp_CUSTOMER Lookup Procedure Edit the agg_CUSTOMER_ACTIVE Aggregator ° Check the Groupby checkbox for the CUST_NO in the Ports tab ° Check the Sorted Input checkbox in the Properties tab Edit the sq_INVOICE Source Qualifier – in the Properties tab, increase the value for the Number of Sorted Ports from 0 to 1
By setting the value to 1, at session runtime,
the Source Qualifier will dynamically build SQL selecting the first active output port and use it to build the Order by clause. If the value were set to 2, it would build the Order by clause by the first 2 active output ports, etc. The generated SQL would be as follows: SELECT INVOICE.CUST_NO, INVOICE.DATE_CLOSED, INVOICE.CANCELED FROM INVOICE ORDER BY INVOICE.CUST_NO • • • •
Save the mapping Validate the s_m_ODS_CUSTOMER_ACTIVE_upd_xx Session Task Save the workflow Restart the wf_ODS_CUSTOMER_ACTIVE_upd_xx Workflow
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
195
Results – Extra Credit Session Properties
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
196
Results – Extra Credit Session Transformation Statistics
It is, indeed, the same 25 rows that loaded, however, more efficient and much cleaner results.
Informatica PowerMart / PowerCenter 6 Basics – Lab 14– Data Driven Inserts and Deletes
197
Lab
15 LAB 15 – INCREMENTAL UPDATE PURPOSE •
The PRODUCT and VENDOR source tables will be used to create an ODS PRODUCT table that has information about the equipment and supplies that the company resells to customers. This will update the ODS Product table to keep the most current product information. Only equipment and supplies that are currently available should be in the table, so all records that are discontinued will be taken out of the ODS Product table.
OBJECTIVE •
Create and configure a Lookup and Update Strategy Transformations to do an incremental update on the PRODUCT table
D U R AT I O N •
40 minutes
SUMMARY SOURCES: VENDOR, PRODUCT TARGETS: ODS_PRODUCT
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
198
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
199
Import Source Tables
•
Right-click anywhere in the Work Area and select Clear All Continue to work in the assigned student folder and import the relational sources, PRODUCT and VENDOR
•
Click on the Replace button to replace the PRODUCT table
•
Import Target Table
•
Clear Warehouse Designer workspace – Right-click anywhere in the Work Area and select Clear All
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
200
•
Import the relational Target, ODS_PRODUCT
Create Mapping •
Create a mapping called m_ODS_PRODUCT_xx
Add Sources and Target Definitions • •
Add PRODUCT and VENDOR Source Definitions Add ODS_PRODUCT Target Definition
Create Source Qualifier •
If present, delete the Source Qualifiers
There is a primary key – foreign key relationship between the two source tables. To take advantage of the relationship, only one Source Qualifier is necessary. The Source Qualifier will pass the “where clause” to the database at runtime to join the tables.
There is a feature to prevent the Source Qualifier from entering the mapping when sources are added, however, it is easier to delete unwanted Source Qualifiers that enter the mapping and create anew than to manipulate turning the feature on and off. The feature is located Tools | Options - Format tab – Tool: Mapping Designer -Create source Qualifier checkbox. •
Create new Source Qualifier selecting both sources (PRODUCT and VENDOR) and name it sq_PRODUCT_VENDOR
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
201
Create Lookup Transformation - Connected For more information on Connected vs. Unconnected Lookups refer to flagged step Create a Lookup on page 172.
•
Add a Lookup box appears
Transformation to the mapping and a dialog
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
202
• • • •
Choose Target button and select the ODS_PRODUCT table Copy/Link PRODUCT_CODE from the sq_PRODUCT_VENDOR to the Lookup Transformation Rename the Lookup Transformation lkp_PRODUCT Rename the port PRODUCT_CODE1, which came from sq_PRODUCT_VENDOR, to PRODUCT_CODE_src
Appending _src to a port states it came from
the source and not the lookup. It will make more sense when it is used in a transformation later in the exercise. Never change the name of a port that has the Lookup (L) port checked. Lookup ports are the ports that will be used in the SQL select statement sent to the database. If column is renamed, then a database error will occur at session runtime. •
Turn off the ‘O’ (output) attribute for: VENDOR_ID, VENDOR_NAME, VENDOR_STATE, PRODUCT_NAME, CATEGORY, MODEL, PRICE, and FIRST_CONTACT
Another Best Practice with the Lookup Transformation is to be discerning when selecting the columns to be cached. The less cached, the better to prevent wasting system resources that would likely cause performance degradation. There are two approaches: Delete the unnecessary port(s) – this is not recommended while in development mode;
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
203
Uncheck the Output attribute – this prevents columns from being unnecessarily cached. This can be done for those ports that are not needed for the condition, and will not be used downstream in another transformation. If you later determine that a port is needed, recheck the Output attribute, and the port will be cached and available.
Uncheck these eight (8) Output ports to exclude them from the SQL Select Statement
•
Add the Lookup condition: PRODUCT_CODE = PRODUCT_CODE_src
•
Click OK button and save the repository
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
204
Create Update Strategy Transformation For more information about an Update Strategy see flagged step Create an Update Strategy on page 189.
• • •
•
Create an Update Strategy Transformation and name it upd_ODS_PRODUCT Copy/Link PRODUCT_CODE and PRODUCT_CODE_src from the lkp_PRODUCT to the upd_ODS_PRODUCT Copy/Link VENDOR_ID, CATEGORY, PRODUCT_NAME, MODEL, PRICE, DISCONTINUED FLAG, VENDOR_NAME, FIRST_CONTACT, and VENDOR_STATE from the sq_PRODUCT_VENDOR to upd_ODS_PRODUCT Rename the PRODUCT_CODE port, which came from the lkp_PRODUCT Lookup Procedure, to PRODUCT_CODE_lkp
Similar to the rename of the Product_Code_src, once the ports are passed from the Lookup downstream to another transformation, it can be renamed. Renaming ports in this fashion will make expression building easier because the port’s origin will be known. However, to reiterate the point, never change the name of a lookup port in the Lookup Transformation or, for that matter, in the Source Qualifier. Both of these transformation objects generate SQL select statements passed to the database so the column
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
205
names must match to avoid database errors at runtime.
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
206
•
Click on the Properties tab. Add the following expression:
IIF(ISNULL(PRODUCT_CODE_lkp), IIF(DISCONTINUED_FLAG = 0, DD_INSERT, DD_REJECT), IIF(DISCONTINUED_FLAG = 0, DD_UPDATE, DD_DELETE)) Below is logic used in IIF statement: Condition LKP_PRODUCT_CODE ISNULL DISCONTINUED_FLAG = 0 LKP_PRODUCT_CODE NOT ISNULL DISCONTINUED_FLAG = 0 LKP_PRODUCT_CODE NOT ISNULL DISCONTINUED_FLAG != 0 LKP_PRODUCT_CODE ISNULL DISCONTINUED_FLAG != 0
Action Description Insert The record was not found in lookup table and the product has not been discontinued, so the record will be inserted into target table. Update The record was found in lookup table and the product has not been discontinued, so the record will update the target table. Delete The record was found in lookup table and the product has been discontinued, so the record will be deleted from target table. Reject The record was not found in lookup table and the product has been discontinued, so the record will be rejected from target table.
Pseudo Code IF(record doesn’t exist in target table) THEN IF DISCONTINUED_FLAG = 0 THEN INSERT record ELSE REJECT record) ELSE (record exists in target table) THEN IF DISCONTINUED_FLAG = 0 THEN UPDATE record ELSE DELETE record)
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
207
•
Just below the expression, the Forward Rejected Rows Checkbox is checked by default – uncheck it to keep rejected rows from being forwarded
With the Forward Rejected Rows selected, the Informatica Server flags the rows for reject and writes them to the session reject file. When not selected, the Informatica Server drops rejected rows and writes them to the session log file. Link Target Definition
•
•
Connect PRODUCT_CODE_src, VENDOR_ID, CATEGORY, PRODUCT_NAME, MODEL, PRICE, VENDOR_NAME, FIRST_CONTACT, and VENDOR_STATE from the from the upd_ODS_PRODUCT to the ODS_PRODUCT Target Definition Save
Completed Mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
208
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
209
Create Workflow
• • • • • • •
• • •
Create Workflow called wf_ODS_PRODUCT_xx Add wf_ODS_PRODUCT_xx.log as the Workflow Log File Name Rename Start task to start_ODS_PRODUCT Create Session Task called s_m_ODS_PRODUCT_xx Verify Task – Properties - Treat source rows as Data Driven Select native_source connection for sq_PRODUCT_VENDOR source instance Select native_target_xx connection for ODS_PRODUCT target instance ° do not Truncate ° select Normal for Target load type property Link start_ODS_PRODUCT to s_m_ODS_PRODUCT_xx Save, Validate and Start Workflow wf_ODS_PRODUCT_xx Monitor and Review the results for s_m_ODS_PRODUCT_xx in the Workflow Monitor
Results – Session Properties
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
210
In this case, the CMN_1079 WARNING message on the last line of the dialog box above is just that, a warning. Since this was the first run of the session and the lookup was on an empty target table, then the message can be ignored. The session will never fail on this message. If, on the other hand, the lookup on the target table is expected to have rows in it from a previous run of the session task, then there is a problem to investigate. Results – Session Transformation Statistics
If the Forward Reject Rows checkbox were checked, 3 rows would have been rejected:
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
211
Results – Preview Data ODS_PRODUCT
Only the first few rows are shown.
Informatica PowerMart / PowerCenter 6 Basics – Lab 15 – Incremental Update
212
Lab
16 LAB 16 – FEATURES AND TECHNIQUES II OBJECTIVES This is the second Instructor-led “follow along” lesson to show and explain the various Designer features and tools that can improve efficiency and ease of use. Below is a list of lessons to be covered: • • • • • •
Find (Search Options) View Object Dependencies Comparing Objects and Mappings Overview Window Copy Objects across Mappings Shortcuts
D U R AT I O N •
40 minutes
Lesson 1 – The Find feature The Designer includes the Find Next and Find in Workspace tools to help you find columns or ports in repository objects, or strings in the output window. This can be very helpful in any number of scenarios including: • Tracing the impact of change through a mapping; • Debugging an invalid port; • Recognizing a miscalculated port value; or • Detecting a mapping logic flow error. Find in Workspace
The Find in Workspace tool searches all of the transformations in the workspace for port or transformation names. It can search for column or port names or table names matching the search string. The search can span across all names in Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
213
the workspace, or across the business name of a table, column, or port. The search can be case sensitive and match on a whole word. • •
•
Open the m_ODS_EMPLOYEE_xx Mapping In any of the Designer tools, click the Find in Workspace button or choose Edit- Find in Workspace In the Find What box type type_code and click on the Find Now button
The choices are to search for Field names or
Table names matching the search text. If Fields is chosen, the Designer searches for column or port names matching the search text. If Tables is selected, the Designer searches for table names matching the search text
By default, it is not case sensitive, however there are additional checkboxes to select, which will force it to do so.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
214
The Designer saves the last 10 search strings to be chosen in the drop -down list from both the Find in workspace and Find next (to be discussed next) options. •
Double-click on the exp_ODS_EMPLOYEE result
In the Normal View, this positions the mapping where the TYPE_CODE port in the exp_ODS_EMPLOYEE Expression is located in the upper left corner of the Workspace. In other words, it would scroll down to the port’s location, highlight it, and position the object in upper left corner of the Workspace. In the Iconic View, Find in Workspace does the same thing, however, the transformation object would have to be opened to the Normal View to see the port. Find Next
The Find Next tool can search for a column or
port name (or any set of characters in a name) in: Transformations, Mapplets, Source and Target Definitions. Unlike Find in Workspace, the Find Next tool can search one object at a time – it cannot search multiple objects at the same time. The Find Next works in each Designer tool. In addition, this is the ONLY way to do a full text search in any of the Tabs in the Output Window (e.g. Save, Validate, Debugger, etc.), which is extremely useful when debugging.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
215
•
Select a single transformation or click in the output window before performing the search – otherwise, the Find Box will remain grayed out.
To find a column or port name: • Select the transformation, mapplet, source or target definition • Enter the text to search for, in the Find box on the Standard toolbar. For example enter EMPLOY to locate an EMPLOYEE column
Find Next button
Find Box
The search is not case-sensitive.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
216
•
Select Edit | Find Next or click the Find Next Function key, F3, to search for the string
icon or
The Designer finds the first occurrence of the
search string. This is the only way to find text in the Output Window.
•
Press Enter again to search for the next occurrence of the search string
In the Designer, it searches each port or column name in the selected transformation and stops at the end of the transformation (see status bar at the bottom of the screen for messages) In the Output window, it starts at the top of the log, searches to the bottom and loops back to the top as it finds the entered string. Lesson 2 – View Dependencies
View Dependencies is a quick and powerful way to do a dependency analysis on virtually any object in the repository. This is especially powerful because it can be executed from the Designer tool A simple example is a source needs to be changed. View Dependencies will show all of the mappings associated with the source. In turn, View Dependencies on those mappings impacted by the source change identifies the session tasks associated with the mappings. • •
Open the Navigator Window and expand the Targets Node in the assigned Student folder Right-click on the ODS_CUSTOMER_ACTIVE Target Definition and select View Dependencies…
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
217
The folder must open (active) in the workspace for this feature to work
•
Try the same steps on any Mapping and note the dependency to a Session Task
•
Try expanding down to the Source Instance in a mapping such as, m_ODS_CUSTOMER_ACTIVE_xx, and View Dependencies… on INVOICE.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
218
Note the dependencies on other mapping
objects. This demonstrates this features versatility because it is available on virtually every object.
Lesson 3 – Comparing Objects The Designer allows two repository objects of the same type to be compared to identify differences between them. There are limitations between each of Designer Tools; however, this is a very powerful feature. The Designer allows for the comparison of sources, targets, transformations, mapplets, mappings, instances or mapping/mapplet dependencies in detail. The comparison can be based on two objects simply within the same mapping, or on two objects across open folders, or across repositories (e.g. Development, Test, Production).
Use the following tools to compare sources, targets, or transformations:
Source Analyzer - Compare two sources, two source shortcuts, or a source with a source shortcut. Warehouse Designer - Compare two targets, two target shortcuts, or a target with a target shortcut. Transformation Developer - Compare two reusable transformations, two reusable transformation shortcuts, or a reusable transformation with a reusable transformation shortcut. Mapping Developer (Mapplet Developer, too) Compare two source instances, two transformation instances, two target instances, or two mapplet instances. Instances of different types cannot be compared (e.g. source instance with a transformation instance) Compare Instances Within Mapping or Mapplets •
Open mapping m_ODS_ORDER_AMOUNT_sorter_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
219
•
Right-click on jnr_ODS_ORDER_AMOUNT Joiner and select Compare Objects…
•
Select srt_ODS_ORDER_AMOUNT as Instance 2 and click on the Compare button Click through the different tabs to see the differences
•
All of the differences will appear in red. On the Ports tab note the ports names that are highlighted have identical names and are shown in the same row. Also note that the ORDER_DATE port has a different Datatypes and Precision and it is further highlighted in red.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
220
Compare Target Definitions •
Import ODS_EMPLOYEE_ROOKIE and the DIM_REP tables
ODS_EMPLOYEE_ROOKIE and DIM_REP will be used in the later labs; however, they are good examples to demonstrate the feature. These additions should not be saved after this lesson. • •
Expand the Targets Node and pull ODS_EMPLOYEE into the Warehouse Designer Right-click on ODS_EMPLOYEE, select Compare Objects… and a different dialog box will appear than the last
The key difference is this dialog box has a
Browse button that will allow searching through all of the open folders for objects of the same type.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
221
•
Click on the Browse button and drill down through the assigned Student folder to the Targets Node to select ODS_EMPLOYEE_ROOKIE
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
222
•
Click on the Compare button and view the Columns tab
Differences would appear in red
Again, all differences would appear in red. In this case, the difference is the Table Name. • •
Close and try the same steps with the DIM_REP table – compare ODS_EMPLOYEE to DIM_REP Click on the Compare button and view the Columns tab
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
223
Note that columns that do match by name
appear on different lines. There is nothing in red, indicating differences. Only when the column names are the same would differences appear in red. Compare Mapping Instances The ability to compare an entire mapping is the Grand Finale of comparisons in the Designer Tool. There is too much detail to cover all of the options, so this will just demonstrate the basics. • • • • • •
Disconnect from the Studentxx folder and do not save the changes from the previous lesson Open Studentxx folder Open the m_ODS_ORDER_AMOUNT_xx Mapping Select Mappings | Compare from the main menu Click on the Browse button for Mapping 2 and select the m_ODS_ORDER_AMOUNT_sorter_xx Mapping Click on Compare button
Again, the differences are highlighted in red. Key differences will be listed on the Summary Tab such as in this case: Instances: Instance agg_ODS_ORDER_AMOUNT is different across the two mappings. Instance srt_ODS_ORDER_AMOUNT is present only in mapping m_ODS_ORDER_AMOUNT_sorter_xx. • • •
Click on the Instances Tab and in the Transformations Instances, select agg_ODS_ORDER_AMOUNT Instance Click on the Compare Instances… button Click through the tabs to find the difference
The only difference should be on the Properties tab – the Sorted Input flag was set to True for in the m_ODS_ORDER_AMOUNT_sorter_xx mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
224
Lesson 4 – Overview Window This feature assists with maneuvering around mappings with many transformation instances. •
Open m_ODS_ORDER_AMOUNT_xx Mapping
The m_ODS_ORDER_AMOUNT_xx Mapping should be in the Recent Mappings drop down list. • •
• •
Arrange the mapping to the Normal View Select View | Overview Window
Drag the box around to reposition the mapping in the workspace Draw a box around a couple of transformations in the workspace and note the changes in the Overview Window
Objects selected in the space will be highlighted in red in the Overview Window.
It will also always remain on top of any client application, not just the Designer, until closed. Lesson 5 – Copying and Pasting Objects
This feature is new with version 6 and allows the developer to copy and paste objects from mappings and mapplets within the same folder. One of the primary advantages of using this feature is the ability to select more than one object at a time and paste the objects into a mapping, mapplet or both.
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
225
Keep the following concepts in mind when copying and pasting objects within a folder: - Objects can be copied from any mapping or mapplet in a folder to any new, existing mapping or mapplet in the same folder - Multiple objects can be copied from the workspace - Selected objects can be viewed by choosing View | Overview window. Selected objects will appear in color. Objects cannot be copied from a shortcut to a mapping or mapplet nor is it allowed to paste objects into a shortcut to a mapping or mapplet
• • • • • •
Open m_ODS_ORDER_AMOUNT_sorter_xx Mapping Arrange the mapping to the Iconic View Draw a box around the Expression, Sorter and Aggregator Press Ctrl-C to copy Create a new mapping via Mappings | Create and select the default name Click in the workspace and press Ctrl-V to paste
Lesson 6 – Shortcuts
One of the primary advantages of using a shortcut is maintenance. If a change is needed to all instances of an object, make the change to the original repository object and all shortcuts to it will automatically inherit those changes. In contrast, if there are multiple copies of an object, each will need to be updated. A word of caution, changes can invalidate mappings and sessions. For example, if there is a shortcut to a reusable transformation and simple Precision change is made to a port, each and every mapping with a shortcut to that object will become invalid, which in turn, invalidates the session tasks using those mappings. The key is to plan and use the Dependency Analysis Features to determine the impact of the change and plan accordingly. If the change is needed immediately, then temporarily make a copy of the object until the shortcut can be updated
Shortcuts have the following advantages over copied repository objects:
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
226
- Allows the developer to maintain a common repository object in a single location - Inherits all changes - Restricts developers to a set of predefined metadata by incorporating the shortcuts into their work instead of developing repository objects independently - Share complex mappings, mapplets, or reusable transformations across folders and repositories – only if the repository where the shared folder is located is a Global Repository - Saves space in the repository by keeping a single repository object and using shortcuts to that object, instead of creating copies of the object in multiple folders or multiple repositories
As with copying objects, the destination folder must be open with the appropriate tool active.
Shortcuts cannot be created to objects from nonshared folders. Create Shortcut to Source •
Expand the Shared_Student folder
•
Click-and-drag EMPLOYEE Source Definition into the mapping that is open in the workspace
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
227
As the cursor enters the workspace, the icon should look as follows as:
If it does not, then start over and make sure the object is being dragged in from the Shared_Student Folder. •
Click on Yes to create the shortcut – make sure it doesn’t say copy
`
The Source Definition and the Source Qualifier both reflect that the source is a shortcut.
To copy an object from a Shared Folder rather than to shortcut it, hold the CTRL key before dropping the object. Do Not Save Changes •
Disconnect from the Studentxx folder and do not save the changes from the previous lesson
Informatica PowerMart / PowerCenter 6 Basics – Lab 16 – Features and Techniques II
228
Lab 17
LAB 17 – ROUTER PURPOSE •
To develop a mapping where a row can be mapped to any one of or all three possible Targets
OBJECTIVE •
Learn the functionality of the Router Transformation when multiple filtering conditions are required in a mapping
D U R AT I O N •
30 minutes
SUMMARY SOURCE: ODS_EMPLOYEE TARGETS: ODS_EMPLOYEE_SALES, ODS_EMPLOYEE_NON_SALES, ODS_EMPLOYEE_ROOKIE
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
229
Import Source Table
The ODS_EMPLOYEE table is the first ODS table to be used in any exercise as a source – so, what once was a target is now a source, too. That means there will be two instances of this table in the folder -- both a Source Definition and Target Definition. In contrast to the Automatic Target Creation (the next step) where a source can be pulled into the Warehouse Designer and it becomes an instant (automatic) target; a Target Definition cannot be copied to the Source Node. So, the most viable solution is to import the Source Definition from the target database (schema) into the Source Analyzer. •
Import the relation source, ODS_EMPLOYEE. The key in this step is to import from the target database
Import Target Tables
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
230
• • •
Clear Warehouse Designer workspace Import the relational Targets, ODS_EMPLOYEE_SALES, ODS_EMPLOYEE_NON_SALES ODS_EMPLOYEE_ROOKIE was imported in the previous exercise
Create Mapping •
Create a mapping called m_ODS_EMPLOYEE_TYPE_xx
Add Source and Target Definitions •
Add ODS_EMPLOYEE Source Definition with its respective Source Qualifier
•
Add ODS_EMPLOYEE_SALES, ODS_EMPLOYEE_NON_SALES and ODS_EMPLOYEE_ROOKIE Target Definitions
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
231
Hold down the Ctrl key as each table is selected so that all three tables can be moved at the same time
Create Router Transformation
A Router Transformation is similar to a Filter Transformation because both transformations allow you to test data based on a condition; but that’s where the similarities end. The Filter test rows passing through it against one condition and drops the rows that do not match. A Router Transformation tests each row against one or more conditions, called groups, and flags the row true allowing it to pass through that group. Each row will be tested against each and every group, so it can be dispersed across many groups if the conditions are met. Another advantage the Router has over the Filter is if the row does not meet any of the defined group conditions, it will route the row to the Default Group. This feature alone makes it great transformation for error handling.
•
Create a Router Transformation and name it rtr_ODS_EMPLOYEE_TYPE Copy/Link all of the ports from sq_ODS_EMPLOYEE to rtr_ODS_EMPLOYEE_TYPE Edit the rtr_ODS_EMPLOYEE_TYPE Transformation and click on the Groups tab Click on Add a new group button
•
Rename NEWGROUP to SALES
• • •
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
232
• • •
Add a condition for TYPE_CODE equal to ‘SALES’ (e.g. TYPE_CODE = 'SALES') Add two more groups (using the same method) by the names: NON_SALES and ROOKIES For NON_SALES, check for TYPE_CODE of ‘ADMIN’ or ‘FIN’ or ‘MGR’.
•
For ROOKIES check where their hire date is less than 365 days from today’s date. Use the following condition: DATE_DIFF (SESSSTARTTIME, DATE_HIRED, 'dd') < 365
•
Click OK to close the Router edit window
The Router gives each row an opportunity to meet every group condition, so a row can exit the Router multiple times. Only when the row does not meet any condition does it slip all the way to the Default Group. The row can exit the Default group if the ports are linked to another transformation or simply terminate by not linking them. Link Target Definitions •
Link the ports from the various groups within the rtr_ODS_EMPLOYEE_TYPE Router to their corresponding columns in the three target definitions
Try Autolink to see the way groups are
displayed and since the port names will have numbers associated with them in the Router Transformation it will either be a link by position or link by suffix – try both ways since there are three groups and target definitions. •
Save
Completed Mapping
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
233
Create Workflow
• • • • •
Create Workflow called wf_ODS_EMPLOYEE_TYPE_xx Add wf_ODS_EMPLOYEE_TYPE_xx.log as the Workflow Log File Name Rename Start task to start_ODS_EMPLOYEE_TYPE_upd Create Session Task called s_m_ODS_EMPLOYEE_TYPE_xx Select native_target_xx connection for sq_ODS_EMPLOYEE source instance
Note that native_source is not the chosen
database connection. The ODS_EMPLOYEE source table was populated in an earlier lesson and is located in the target database (schema), native_target_xx •
Select native_target_xx connection for each of the target instances ° Check property Truncate target table option for each target object (3 attribute changes)
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
234
Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property for each target (3 attribute changes) Link start_ODS_EMPLOYEE_TYPE to s_m_ODS_EMPLOYEE_TYPE_xx Save, Validate and Start Workflow wf_ODS_EMPLOYEE_TYPE_xx Monitor and Review the results for s_m_ODS_EMPLOYEE_TYPE_xx in the Workflow Monitor °
• • •
Results – Session Properties
Results – Session Transformation Statistics
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
235
Results – Session Properties ODS_EMPLOYEE_SALES:
ODS_EMPLOYEE_NON_SALES
ODS_EMPLOYEE_ROOKIES
Informatica PowerMart / PowerCenter 6 Basics – Lab 17 – Router
236
Lab
18 LAB 18 – STRAIGHT LOAD PURPOSE In the exercise that follows this one, there is a table, ODS_LINE_ITEM that will have a foreign key relationship with the table in this exercise, ODS_INVOICE. Ultimately, the aforementioned tables will be joined together in another exercise. OBJECTIVE Learn how to create a simple pass through mapping D U R AT I O N •
10 minutes
SUMMARY SOURCE: TARGET:
INVOICE ODS_INVOICE
Import Target A Mapplet to be developed during a later lab will need two sources, ODS_INVOICE and ODS_LINE_ITEMS. ODS_INVOICE has not been populated in any previous exercise
•
Import ODS_INVOICE Target Definition
Create Mapping Informatica PowerMart / PowerCenter 6 Basics – Lab 18 – Straight Load
237
• • •
Create mapping called m_ODS_INVOICE_xx Add INVOICE Source Definition with its respective Source Qualifier Add ODS_INVOICE Target Definition
There is no need to apply additional logic to the load. Hence, this is a straight through mapping meaning it will go directly from source to target. • •
Link the sq_INVOICE to ODS_INVOICE Save repository
Completed Mapping
Create Workflow
• • • • • •
Create a Workflow called wf_ODS_INVOICE_xx Add wf_ODS_INVOICE_xx.log as the Workflow Log File Name Rename Start task to start_ODS_INVOICE Create Session Task called s_m_ODS_INVOICE_xx Select native_source connection for sq_INVOICE source instance Select native_target_xx connection for ODS_INVOICE target instance ° Check property Truncate target table option
Informatica PowerMart / PowerCenter 6 Basics – Lab 18 – Straight Load
238
Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link start_ODS_INVOICE to s_m_ODS_INVOICE_xx Save, Validate and Start Workflow wf_ODS_INVOICE_xx Monitor and Review the results for s_m_ODS_INVOICE_xx in the Workflow Monitor °
• • •
Results – Session Properties
Results – Preview Data ODS_INVOICE
Informatica PowerMart / PowerCenter 6 Basics – Lab 18 – Straight Load
239
Informatica PowerMart / PowerCenter 6 Basics – Lab 18 – Straight Load
240
Lab
19 LAB 19 – CONDITIONAL LOOKUP PURPOSE •
Read from a large source that has a few rows of invalid data. To validate specific data, look up the value in another table. This is only necessary for about one row out of ten
OBJECTIVE •
Build a mapping to load an ODS table that contains line item data that will ultimately be used to load the fact table in the data mart
D U R AT I O N •
30 minutes
SUMMARY SOURCES: INVOICE, LINE_ITEM and PRODUCT TARGETS: ODS_LINE_ITEM
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
241
Import Source Tables
•
Continue to work in the assigned student folder and import the relational source, LINE_ITEM. The Sources PRODUCT and INVOICE tables have already been imported for use in previous mappings and should be in the Sources node.
Import Target Table
•
Import the target table ODS_LINE_ITEM
Create Mapping •
Create a mapping called m_ODS_LINE_ITEM_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
242
Add Sources and Target • •
Add the PRODUCT, LINE_ITEM and INVOICE Source Definitions Add the ODS_LINE_ITEM Target Definition
Create Source Qualifier
•
If present, delete all of the Source Qualifiers (see flagged step Create Source Qualifier on page 201 for more information)
•
Create a new Source Qualifier Transformation selecting all three sources ( PRODUCT, INVOICE and LINE_ITEM) and name it sq_PRODUCT_INVOICE_LINE_ITEM
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
243
Create Mapping Parameter
A Mapping Parameter represents a constant value that can be defined before running a session task. A mapping parameter retains the same value throughout the entire session. By creating these, hard coding values within logical expression can be avoided. See Mapping Parameters and Variables Chapter in the Designer Guide for additional information.
•
Select Mappings | Parameters and Variables…
•
Click the Add button In the Name column, type in $$MIN_RATE
•
“$$” is the required prefix for User Defined Mapping Parameter and Variable object names
•
Hit the Tab key to jump to the Type column and click on the down-arrow or type the letter ‘p’ to select Parameter Hit the Tab key to jump to the Datatype column and click on the down-arrow or type the letter ‘d’, twice, to select decimal. If you bypass decimal, continue to cycle through all of the Datatypes that begin with ‘d’ until decimal is found Set Precision to 10 and Scale to 2
•
Click in the Initial Value: edit box and key in .00
•
Validate the initial value by clicking on the Validate the far right of the edit box Click the ADD button, again In the Name column, type $$MAX_RATE
• •
• •
•
Select Parameter as the Type Select decimal as the Datatype Set Precision to 10 and Scale to 2
•
Set Initial Value: .35 and validate
• •
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
button at
244
•
Click OK to continue
These Parameters will be utilized in an expression later in this lesson.
Create an Expression Transformation
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
245
• •
Create an Expression Transformation named exp_ODS_LINE_ITEM Copy/Link the following ports from the sq_PRODUCT_INVOICE_LINE_ITEM to the exp_ODS_LINE_ITEM transformation: INVOICE.CUST_NO, LINE_ITEM.LINE_ITEM_NO, LINE_ITEM.INVOICE_NO, LINE_ITEM.PRODUCT_CODE, LINE_ITEM.QUANTITY, LINE_ITEM.DISCOUNT, PRODUCT.PRICE, and PRODUCT.COST ports
INVOICE_NO and PRODUCT_CODE have two entries in the sq_PRODUCT_INVOICE_LINE_ITEM. The duplicates are appended with the number “1”. For this exercise, both ports are to be sourced from the LINE_ITEM table – hence it is listed above as LINE_ITEM.INVOICE and LINE_ITEM.PRODUCT_CODE. The bottom line is, choose the ports as they are listed in the bullet and ignore the names in the picture, as they may be different based on how the sq_PRODUCT_INVOICE_LINE_ITEM was created.
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
246
• • •
•
Rename the ports to exclude the number “1” where it was appended (e.g. INVOICE_NO and PRODUCT_CODE) DISCOUNT port should be Input port, only Insert a new port called DISCOUNT_RATE_out (after the DISCOUNT port)
Click OK
Create Unconnected Lookup Procedure In this step an Unconnected Lookup will be utilized as a function within the mapping. As with a number of transformations, Lookups can be configured to be ‘connected’ or ‘unconnected’.
Unconnected Lookups A Connected Lookup takes every row passed to it, processes it based on the lookup condition and passes it downstream to the next transformation. It should only be used when the majority or all of the rows are required to pass through it or when more than one port (return value) is required from the Lookup. On the other hand, an Unconnected Lookup is called upon conditionally. It is ideal to use when the majority or all of the rows do not need a return value (ports) from it. This conditional Lookup is called unconnected because no physical links are present – it stands alone, unconnected. To use it, it must be called upon by a formula in some other transformation; typically in a Filter or Update Strategy Transformation. Several ports may be passed to it to complete the condition, however, only one value may return designated by a new port attribute, the ‘R’ (return) port (see below).
The ‘R’ port is only required in when the Lookup is unconnected. If checked while connected, it will be ignored. Only one port can be selected, hence it acts like a radio button. Another common usage of an unconnected lookup is when it needs to be called several times from different expressions – again, considering all of the limitations mentioned above. In summary, use an Unconnected Lookup when:
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
247
1) Every row does NOT need a return value from the Lookup, in other words, can be called conditionally; and 2) Only one port is required (‘R’ port) for the return value
•
Create a Lookup Transformation and choose ODS_CUSTOMER_ACTIVE from the Target – the ports will look as follows:
•
Rename the transformation lkp_ODS_CUSTOMER_ACTIVE Add a port (input only) called CUST_ID_in with the same Datatype, Precision and Scale as CUST_ID Uncheck the Output ‘O’ ports for CUST_NAME, FIRST_CONTACT, CUSTOMER_STATE, CUSTOMER_ZIP and CUSTOMER_TYPE
• •
This prevents those ports from being selected for the Lookup Cache, however, keeps them available in case they are needed later. See Key Concept under the Create Lookup Transformation - Connected step on starting on page 202. •
Click the Return Port attribute for the DISCOUNT_RATE
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
248
•
On the Properties tab, change the Location Information to native_target_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
249
• •
On the Conditions tab, add the condition: CUST_ID = CUST_ID_in Save repository
Edit Expression Add an expression that includes a formula that calls the Lookup Transformation, hence the name of the Lab, Conditional Lookup
The reason for not doing this step during the
Create Expression step is the Lookup Transformation, for which will be called in a formula, did not exist. This demonstrates a logical and iterative approach to development. •
Build the following formula in expression of the DISCOUNT_RATE_out port:
For this exercise, try not to type the expression in manually. This entire expression can be created via the point-and-click method. Below are detailed instructions to accomplish this task, which is the best practice considering the new expressions and variables to be used in the formula.
For connected transformations, if the point-and-click method is used to enter port names in an expression, the Designer updates that expression when port names are changed in the transformation. For example, a valid expression is written that determines the difference between two dates, Date_Promised and Date_Delivered. Later, Date_Promised port name is changed to Due_Date. If the expression is developed using the point-and-click method, the Designer changes the Date_Promised port name to Due_Date in the expression. If, however, the expression was written manually, it must be updated manually to maintain a valid expression.
•
From the Functions tab, expand the Special category and double-click on the IIF function. Notice the function appears in the Formula work area with parenthesis in place.
There is no need to reposition the cursor until instructed
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
250
•
• • • • • • • • • • •
Expand the Test category and double-click on the ISNULL function. Again, note that this function is inserted automatically within the original parenthesis From the Ports tab and double-click on the DISCOUNT port Position the cursor between the two right parentheses and click on the OR button from the Operator Keypad Add the DISCOUNT port again From the Operator Keypad, click the less-than (<) button From the Variable tab, expand the Mapping Parameters folder and double-click on $$MIN_RATE From the Operator Keypad, click on the OR button Return to the Ports tab, add the DISCOUNT port, yet again From the Operator Keypad, click the greater-than (>) button Return to the Variable tab, add the $$MAX_RATE to the expression Insert a ‘,’ (comma) from the operator keypad Return to the Functions tab and expand the Lookup category at the bottom of the list and double-click on lkp_ODS_CUSTOMER_ACTIVE
This activates the Lookup transformation as an expression •
Return to the PORTS tab and add CUST_NO port
This establishes the input argument to the Lookup transformation
•
Position the cursor between the two right parentheses and insert a (,) (comma) Return to the Ports tab, add the DISCOUNT port Your function should look like this:
•
IIF (ISNULL (DISCOUNT) OR DISCOUNT < $$MIN_RATE OR DISCOUNT > $$MAX_RATE, :LKP.lkp_ODS_CUSTOMER_ACTIVE(CUST_NO), DISCOUNT) Validate and save to the repository
• •
Link Target Definition
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
251
• •
Link exp_ODS_LINE_ITEM to ODS_LINE_ITEM Save
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
252
Completed Mapping
Create Workflow
• • • • • •
•
Create Workflow called wf_ODS_LINE_ITEM_xx Add wf_ODS_LINE_ITEM_xx.log as the Workflow Log File Name Rename Start task to start_ODS_LINE_ITEM Create Session Task called s_m_ODS_LINE_ITEM_xx Select native_source connection for sq_PRODUCT_INVOICE_LINE_ITEM source instance Select native_target_xx connection for ODS_LINE_ITEM target instance ° Check property Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property In the Transformations Tab of the s_m_ODS_LINE_ITEM_xx Session Task: ° Scroll down to where the Object Name is lkp_ODS_CUSTOMER_ACTIVE
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
253
° °
Scroll down further to the Location Information Attribute Verify the Value to native_target_xx
The Transformations Tab was designed to
override properties when needed. It should not be considered a substitute for what really should have been in place in the mapping. If the value for the location information does not reflect native_target_xx as was suggested in the Create Unconnected Lookup Procedure step, then it needs to be updated here to run successfully. A best practice is to update the mapping with the correct value for future session tasks that may be created against it.
This property allows the location of the lookup
to be somewhere other than where the Source and Target are located. It is required to set this property because the Lookup is Unconnected. • • •
Link start_ODS_LINE_ITEM to s_m_ODS_LINE_ITEM_xx Save, Validate and Start Workflow wf_ODS_LINE_ITEM_xx Monitor and Review the results for s_m_ODS_LINE_ITEM_xx in the Workflow Monitor
Results – Session Properties
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
254
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
255
Results – Session Transformation Statistics
Results – Preview Data ODS_LINE_ITEM
Only the first few rows are shown.
Informatica PowerMart / PowerCenter 6 Basics – Lab 19 – Conditional Lookup
256
Lab 20
LAB 20 – HETEROGENEOUS TARGETS PURPOSE •
To alter the mapping created in the Router lab to send rows not loaded into any of the three target tables to a flat file for inspection
OBJECTIVE •
To demonstrate the ability to populate targets of different types
D U R AT I O N •
15 minutes
SUMMARY SOURCE: ODS_EMPLOYEE TARGETS: ODS_EMPLOYEE_SALES, ODS_EMPLOYEE_NON_SALES, FF_EMPLOYEE_INVALID_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
257
Create Flat File Target Definition
•
Drag the ODS_EMPLOYEE Source Definition into the Warehouse Designer workspace
•
Click on the Rename button and name it FF_EMPLOYEE_INVALID_xx Change the Database type to Flat File (on the Table tab) In the Flat File information (also on the Table tab), select Delimited and click on the Advanced… button and verify the defaults: Comma delimited with no optional quotes
• •
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
258
•
On the Columns tab, change the Key Type to NOT A KEY for the EMPLOYEE_ID column
•
Click OK
Copy Mapping •
Copy the m_ODS_EMPLOYEE_TYPE_xx mapping and call it m_ODS_EMPLOYEE_INVALID_TYPES_xx
Edit Mapping •
Delete the ODS_EMPLOYEE_ROOKIE Target Definition from the mapping
Deleting a Target Definition from the mapping
does not delete it from the Target Node – it only deletes it from the mapping • • •
Edit rtr_ODS_EMPLOYEE_TYPE – Groups tab, - delete the ROOKIES group Add the FF_EMPLOYEE_INVALID_xx target to the mapping Leave the rest of the mapping intact
Link Flat File Target Definition
• •
Link the rtr_EMPOLYEE_TYPE Default Group to the FF_EMPLOYEE_INVALID_xx Target Definition Save
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
259
Completed Mapping
Create Workflow
• • • • •
Create Workflow called wf_ODS_EMPLOYEE_INVALID_TYPES_xx Add wf_ODS_EMPLOYEE_INVALID_TYPES_xx.log as the Workflow Log File Name Rename Start task to start_ODS_EMPLOYEE_INVALID_TYPES Create Session Task called s_m_ODS_EMPLOYEE_INVALID_TYPES_xx Select native_target_xx connection for sq_ODS_EMPLOYEE Source Instance
Note that native_source is not the chosen database connection. The ODS_EMPLOYEE source table was populated in an earlier lesson and is located in the target database (schema), native_target_xx •
Select native_target_xx connection for ODS_EMPLOYEE_NON_SALES and ODS_EMPLOYEE_SALES Target Instances
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
260
Check property Truncate target table option for both target objects (2 attribute changes) ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property for each target (2 attribute changes) No changes required for FF_EMPLOYEE_INVALID_xx Target Instance Link start_ODS_EMPLOYEE_INVALID_TYPES to s_m_ODS_EMPLOYEE_INVALID_TYPES_xx Save, Validate and Start Workflow wf_ODS_EMPLOYEE_INVALID_TYPES_xx Monitor and Review the results for s_m_ODS_EMPLOYEE_INVALID_TYPES_xx in the Workflow Monitor °
• • • •
Results – Session Properties
Results – Session Transformation Statistics
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
261
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
262
Results – Preview Data To preview the data on Windows-based Operating System servers: • Right-click on FF_EMPLOYEE_INVALID_xx and select Preview Data… • Click on the button to the right of the Flat File Name
•
Drill down to the directory given by the Instructor and select ff_employee_invalid_xx.out
•
Click on the Open button
•
Click on the Open button, again.
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
263
Lab 21
LAB 21 – MAPPLETS PURPOSE The purpose of this lab is to introduce the concept of Mapplets. Mapplets are a group of transformations that are saved as a set. This set then becomes reusable, and can be used in any mapping. The mapplet created here will be used in next lab. OBJECTIVE Learn the functionality and characteristics of Mapplets D U R AT I O N •
30 minutes
SUMMARY SOURCE: TARGETS:
ODS_INVOICE, ODS_LINE_ITEM out_PROFIT_PERCENT
Import Source Tables
• •
Import ODS_LINE_ITEM and ODS_INVOICE as sources Add the Foreign Key/Primary Key relationship from ODS_LINE_ITEM.INVOICE_NO to ODS_INVOICE.INVOICE_NO
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
264
In the mapplet, the two source definitions above will be joined via the Source Qualifier. There are times when the relationship between tables is not present and needs to be manufactured in the Source Analyzer. This will have no impact on the database, however, in order for the Source Qualifier to dynamically build the “where clause”, the link (relationship) must be present. Create the Mapplet
A Mapplet is a reusable object created in the Mapplet Designer. It contains a set of transformations that allows transformation logic to be used in multiple mappings. The Mapplet Designer is a component of the PowerCenter/PowerMart Designer tool. Once saved, Mapplets are stored in their own node called ‘Mapplets’, visible in the Navigator window and can be used in any mapping by dragging and dropping the mapplet into the mapping from the Navigator window. Creating Mapplets can save time by eliminating the need to create the same set of transformations again and again. They also enforce standards. From a graphical user interface (GUI) perspective, there are very few differences between the Mapplet Designer and the Mapping Designer – they are virtually mirror images of each other. The primary differences are which transformation objects can be used and there are limitations -- the majority, of which, exist within the Mapplet Designer. Furthermore, most objects can be copied between them. This is key because mapplets usually come to fruition as a result of mappings. If, after a mapping has already been created, it has been determined that the complexity and reusability warrants the creation of a Mapplet, it is easy to copy those key objects without building them from scratch.
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
265
• •
Select Tools | Mapplet Designer or Designer Icon Choose Mapplets | Create and name it mplt_PROFIT_PERCENT
Informatica PowerMart / PowerCenter 6 Basics – Lab 20 – Heterogeneous Targets
Mapplet
266
•
Add Sources and Create Source Qualifier
•
Add the ODS_LINE_ITEM and ODS_INVOICE Source Definitions If they exist, delete the two Source Qualifiers and create another using both the ODS_LINE_ITEM and ODS_INVOICE Sources – one Source Qualifier should remain. Rename the Source Qualifier sq_ODS_INVOICE_ODS_LINE_ITEM
•
•
Informatica PowerMart / PowerCenter 6 Basics – Lab 21 – Mapplets
267
Create Aggregator
• •
•
•
Create Aggregator with the name agg_PROFIT_PERCENT Copy/Link ports from the sq_ODS_INVOICE_ODS_LINE_ITEM to the agg_PROFIT_PERCENT as shown above The ports, QUANTITY, DISCOUNT, PRICE and COST should be Input, only Create Port called PROFIT_out (decimal 10,2) with the expression SUM(((PRICE*((100-DISCOUNT)/100))-COST)*QUANTITY) Group by REP_NO, CUST_NO, DATE_OPEN and PRODUCT_CODE
Create Output Transformation (available in Mapplet Designer only)
Informatica PowerMart / PowerCenter 6 Basics – Lab 21 – Mapplets
268
Informatica PowerMart / PowerCenter 6 Basics – Lab 21 – Mapplets
269
Target Definitions are not allowed in a Mapplet.
Hence the existence of the Output Transformation, which will be the link that transports the data from a mapplet to the mappings that use it. Each Output Transformation created in a mapplet displays as an output group, similar to the Router, when viewed in a mapping.
•
Select Transformation| Create - Output Transformation or
•
Use the
Output transformation icon on the designer toolbar
The Output Transformation is only available in the Mapplet Designer. • •
• • •
Name the Output Transformation out_PROFIT_PERCENT Copy/Link all of the ports from the agg_PROFIT_PERCENT Aggregator to the out_PROFIT_PERCENT Output Transformation Rename the PROFIT_out port to PROFIT Save the mapplet View the Mapplet Node – the location of the newly created mapplet, mplt_PROFIT_PERCENT in the Navigator window
As stated in the purpose of this lab, the mapplet created will be used in next lab. The key is when building a Session Task a mapplet cannot be selected. A mapplet must be included within a mapping in order for a Session Task to use it. Completed Mapplet
Informatica PowerMart / PowerCenter 6 Basics – Lab 21 – Mapplets
270
Lab 22
LAB 22 – REUSABLE TRANSFORMATIONS PURPOSE •
The mapplet created in the previous exercise supplies a profit value for each sales rep, customer, product, and date. In this lab, the data will be distributed among four ODS tables, one that will hold the data on a daily level, which is how it is coming directly from the mapplet, and the other three will hold the same data “summarized” or aggregated on a monthly, quarterly and yearly basis, respectively. To accomplish the further aggregation required (monthly, quarterly and yearly), three general-purpose Reusable Aggregator transformations will be created. The Aggregators are considered “general purpose” (generic) because each can take any numeric data – regardless of the meaning of that data – and summarize it on a monthly, quarterly and yearly level.
OBJECTIVES • • •
Promote Transformation to be Reusable Create a mapping that uses a mapplet and Reusable Transformations Use Persistent Named Lookups, mapplets as inputs, as well as, object copy and paste
D U R AT I O N •
75 minutes
SUMMARY SOURCE: TARGETS:
mplt_PROFIT_PERCENT (Sources within mapplet) Shortcuts to ODS_PROFIT_DAILY, ODS_PROFIT_WEEKLY, ODS_PROFIT_MONTHLY and ODS_PROFIT_YEARLY
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
271
Create Mapping •
Continue to work from the assigned folder and create mapping called m_ODS_PROFIT_PERIODIC_xx
Add Mapplet •
Add the mplt_PROFIT_PERCENT, created in the previous exercise, to the mapping from the Mapplets Node
Create Shortcuts to Target Definitions • •
Expand the Target Node in the Shared_Student folder Create shortcuts in the new mapping from the following tables: ° ODS_PROFIT_DAILY ° ODS_PROFIT_MONTHLY ° ODS_PROFIT_QUARTERLY ° ODS_PROFIT_YEARLY
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
272
Create Persistent-Named Lookup Transformation
The Lookup transformation has several performance enhancement properties related to its cache – specifically size and refresh parameters. properties is called Lookup cache persistent.
One of those
If a lookup is performed more than once, either within the same session or in another, and the data does not change for several runs then consider using this property. If the intent is to use this property across sessions, then another property, Cache File Name Prefix needs to be selected to share the cache
If at some point in the various workflows and session tasks that use the Lookup Cache the data needs to be refreshed, another property can be used to force the persistent cache to be refreshed called Recache from the Database.
There is an entire chapter dedicated to Lookup Cache properties including a tips section. Informatica highly recommends studying all of the Lookup Cache Properties to gain the best performance. Refer to Lookup Caches Chapter in the Transformation Guide.
•
Create a Lookup against the ODS_TIME table in the assigned target schema and name it lkp_ODS_TIME
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
273
• •
Edit the lkp_ODS_TIME Lookup Procedure and select the Lookup cache persistent property Enter lkp_ODS_TIME_xx in the Cache File Name Prefix property
It is very important to include the assigned student number in the lkp_ODS_TIME_xx because by default, all of the name caches will be written to the same directory after the session task runs. • •
Link the DATE_OPEN port from the mplt_PROFIT_PERCENT transformation and rename it DATE_OPEN_src Create the Lookup condition DATE_ID = DATE_OPEN_src
Create Aggregator This mapping presents a good opportunity to create three reusable (general purpose) transformations for other mappings to use – ultimately this will reduce duplication of effort and ensure accuracy. Often it is easier to get a visual perspective on what is being built in the Mapping Designer, first, and then promote a specific transformation to be reusable.
• • •
Create an Aggregator and name it agg_3_ID_SUM_BY_MTH Add REP_NO, CUST_NO, PRODUCT_CODE and PROFIT from the mptl_PROFIT_PERCENT Mapplet Link MONTH and YEAR from lkp_ODS_TIME
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
274
Since this Transformation is going to be reused in other mappings, Informatica recommends assigning generic port names considering names from other transformations may be different.
This general-purpose reusable transformation will SUM an input value grouped by three input identifiers and by a time slice derived from an input date. It will pass out the three Ids, the time slice, and the associated summarized value. • • • • •
Rename the ports as shown in the picture above Group the ports: ID1, ID2, ID3, MONTH_in, YEAR_in Change Profit_in Port to Input Only Create Port called Profit_out with the same Datatype, Precision and Scale as Profit_in Add the following formula to the Profit_out port SUM(PROFIT_in)
Promote Aggregator to be Reusable
Reusable Transformations Standard transformations exist within a single mapping. Reusable transformations can be used in multiple mappings or mapplets. Rather than re-creating a transformation with the very same logic several times, create a reusable transformation. Wherever the reusable transformation is needed, add an instance of it to the mapping. The Designer stores each reusable transformation as metadata separate from any mapping that uses the transformation – similar to a shortcut. Once promoted to be reusable, the transformation will appear in the Transformations Node and only to be edited in the Designer’s Transformation Developer. One of the Transformation Developer purposes is for editing reusable transformations. Changes made on the Ports Tab of a reusable transformation are inherited by all its instances and automatically reflects those changes. Instances of the reusable transformation do not inherit changes on the Property Tab (with the exception of the Sequence Generator Transformation) – only modifications to Transformation, Ports, Condition, Groups and Metadata Extension Tabs are passed.
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
275
. •
Edit agg_3_ID_SUM_BY_MTH
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
276
•
On the Transformation tab, check Make Reusable check box – immediately the Designer warns that this action is irreversible
•
Click on Yes to continue
Several things just occurred: the Transformation Type now includes (Reusable) in the name and the Make Reusable Checkbox disappeared.
•
Click on the Ports tab and note the ports are not editable – they have been completely grayed out; on the other hand, the attributes displayed on the Properties Tab are editable
The Reusable Transformation was added to the Transformations Node. Once the repository is saved, it can only be edited in the Transformation Developer.
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
277
•
Save the repository
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
278
Open the Transformation Developer •
Switch over to the Transformation Developer
Copy Reusable Aggregators •
• • •
Expand the Transformations Node in the Navigator window and open the agg_3_ID_SUM_BY_MTH Reusable Transformation in the Transformation Developer using the drag-and-drop method Rename the transformation re_agg_3_ID_SUM_BY_MTH – “re” representing reusable Save the repository From the Transformation Node in the Navigator Window, copy re_agg_3_ID_SUM_BY_MTH by selecting it (single-click), press Ctrl–C and then Ctrl–V
•
Click on the Yes button and change the name to re_agg_3_ID_SUM_BY_QTR
•
Repeat the steps to create a third reusable transformation and name it re_agg_3_ID_SUM_BY_YEAR
Edit Reusable Aggregators
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
279
• •
Edit the re_agg_3_ID_SUM_BY_QTR – change the MONTH_in port to QUARTER_in Edit the re_agg_3_ID_SUM_BY_YEAR – remove the MONTH_in port
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
280
Add Reusable Transformations to Mapping
•
• • •
Add the re_agg_3_ID_SUM_BY_QTR and re_agg_3_ID_SUM_BY_YEAR Reusable Transformations to the m_ODS_PROFIT_PERIODIC_xx Mapping Link QUARTER and YEAR from lkp_ODS_TIME to re_agg_3_ID_SUM_BY_QTR Transformation Link YEAR from lkp_ODS_TIME to re_agg_3_ID_SUM_BY_YEAR Transformation Link REP_NO, CUST_NO, PRODUCT_CODE and PROFIT from mplt_PROFIT_PERCENT to both re_agg_3_ID_SUM_BY_QTR and re_agg_3_ID_SUM_BY_YEAR (see diagram above)
Link Target Definitions • • •
Link mplt_PROFIT_PERCENT Mapplet directly to the Shortcut_To_ODS_PROFIT_DAILY target definition Link the Reusable Aggregators based on Month, Quarter and Year to their corresponding Target Definitions Save
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
281
Completed Mapping
Create Workflow
• • • • •
Create Workflow called wf_ODS_PROFIT_PERIODIC_xx Add wf_ODS_PROFIT_PERIODIC_xx.log as the Workflow Log File Name Rename Start task to start_ODS_PROFIT_PERIODIC Create Session Task called s_m_ODS_PROFIT_PERIODIC_xx Select native_target_xx connection for mplt_PROFIT_PERCENT.sq_ODS_INVOICE_ODS_LINE_IT EM Source Instance
Note that native_source is not the chosen
database connection. The ODS_INVOICE and ODS_LINE_ITEM Source tables were populated
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
282
in earlier lessons and is located in the target database (schema), native_target_xx
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
283
• •
•
• • •
Select native_target_xx connection for all four (4) Target Instances Potentially change a total of 8 properties (4 minimum) on the Targets | Properties Tab: ° Check property Truncate target table option for all 4 Shortcuts to the ODS Target Objects (4 attribute changes) ° Select Normal for Target load type property for each target (4 attribute changes) In the Transformations Tab of the s_m_ODS_PROFIT_PERIODIC_xx Session Task: ° Scroll down to where the Object Name is lkp_ODS_TIME ° Scroll down further to the Location Information Attribute ° Change the Value to native_target_xx Link start_ODS_PROFIT_PERIODIC to s_m_ODS_PROFIT_PERIODIC_xx Save, Validate and Start Workflow wf_ODS_PROFIT_PERIODIC_xx Monitor and Review the results for s_m_ODS_PROFIT_PERIODIC_xx in the Workflow Monitor
Results – Session Properties
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
284
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
285
Results – Session Transformation Statistics
Results – Preview Data ODS_PROFIT_MONTHLY
After previewing data for the Monthly (shown above), note the missing data – that is, the NULLs. There’s no need to review Quarterly and Yearly Profit tables because we can assume those are bad, too. After troubleshooting, it has been determined that lookup, lkp_ODS_TIME, has only returned dates starting in 1999 and the source data goes all the way back to 1997. EXTRA CREDIT •
Update the m_ODS_TIME_xx mapping by changing the year from 1999 to 1996
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
286
Of course this is not the best solution, however, this demonstrates how to do a quick fix and see results. • •
Save the mapping and rerun s_m_ODS_TIME_xx Rerun the workflow wf_PROFIT_PERIODIC_xx and check the results
Why are the results the same? Since the lookup property, Persistent Cache, on the Properties tab for lkp_ODS_TIME is turned on, it did not recache the new values from the ODS_TIME table. The Recache from Database, on the same Properties tab, needs to be selected to update the cache with the new records. This can be accomplished both in the mapping and in the session task. •
Turn on the property, Recache from Database, located on the Transformations tab of the session task, s_m_ODS_PROFIT_PERIODIC_xx
This demonstates how a session property,
originally set in the mapping, can override the mapping and not impact it and other sessions tasks using same mapping. This is an excellent example of how to use Recache from Database. •
After setting the property, rerun wf_PROFIT_PERIODIC_xx and note the differences in the results. The number of rows doesn't change, but all of the columns should no longer be blank
Results All the numbers should be the same as the above results. The difference is in the data – there should no longer be any Nulls. Results – Preview Data Only the Monthly table will be shown for results. ODS_PROFIT_MONTHLY
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
287
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
288
QUESTION Based on the fact there were no calculations being performed by the reusable transformations re_agg_3_ID_SUM_BY_MTH and re_agg_3_ID_SUM_BY_QTR because the lkp_ODS_TIME provided it, could these transformation been combined into one?
Informatica PowerMart / PowerCenter 6 Basics – Lab 22 – Reusable Transformations
289
Lab 23
LAB 23 – DYNAMIC LOOKUP PURPOSE •
Build a mapping to load the DIM_PRODUCT table when source rows do not exist on the target
OBJECTIVE • •
Utilize Informatica’s Dynamic Caching feature when looking up source information against the target table Use a Sequence Generator transformation
D U R AT I O N •
30 minutes
SUMMARY SOURCE: TARGET:
ODS_PRODUCT DIM_PRODUCT
Import Source and Target Table •
Import ODS_PRODUCT as a Source Definition
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
290
Note that this source will be found using the odbc_target DSN. • •
Import DIM_PRODUCT as a Target Definition Save the Repository
Create Mapping • • • • •
• •
Create a mapping called m_DIM_PRODUCT_xx Add the ODS_PRODUCT Source Definition and its’ corresponding Source Qualifier Rename Source Qualifier to sq_ODS_PRODUCT Add the DIM_PRODUCT Target Definition Create Shortcut to Reusable Transformation
Expand the Transformations Node in the Shared_Student folder Create a shortcut to the re_exp_DECODE_CATEGORY Transformation in the m_DIM_PRODUCT_xx Mapping and confirm the creation of a shortcut
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
291
•
Open the re_exp_DECODE_CATEGORY transformation and click on the Ports tabs
As explained in the previous lab, the Ports tab
of the Reusable Transformation cannot be edited from the mappings using them. In addition, Reusable Transformations can only be edited within the Transformation Developer. Certain attributes on the Properties tab can be modified. What can be changed differs between each type of transformation. •
Link CATEGORY port from the sq_ODS_PRODUCT to the CATEGORY_in port of the re_exp_DECODE_CATEGORY
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
292
Create a Sequence Generator Transformation
•
Create a Sequence Generator seq_DIM_PRODUCT
Transformation and name it
Create Expression Transformation
• •
•
•
Create Expression Transformation and name it exp_DIM_PRODUCT Copy/Link PRODUCT_NAME, CATEGORY, MODEL and PRICE ports from the sq_ODS_PRODUCT Transformation to exp_DIM_PRODUCT Copy/Link PRODUCT_DEPARTMENT Port from the Shortcut_To_re_exp_DECODE_CATEGORY transformation to exp_DIM_PRODUCT Copy/Link NEXTVAL from the seq_DIM_PRODUCT transformation to exp_DIM_PRODUCT
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
293
• • •
• •
In the exp_DIM_PRODUCT, make the PRODUCT_NAME and MODEL ports Input Only Ports Rename NEXTVAL port to PRODUCT_SEQUENCE Add a port after PRODUCT_SEQUENCE port rename it PRODUCT_NAME_MODEL
Add the following expression to PRODUCT_NAME_MODEL PRODUCT_NAME || ‘ ‘ || MODEL Save repository
Create Dynamic Cache Lookup Transformation
Dynamic Cache Lookups In this step a lookup will be utilized to determine if source rows exist in the target table. When the Dynamic Cache property is set on a Lookup Transformation, all rows sent to the Target will also update the lookup cache. Therefore, the Target and the lookup cache will be in sync. Refer to “Working with a Dynamic Lookup Cache ” in the “Lookup Caches” Chapter in the Transformation Guide
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
294
•
•
•
• •
Create lookup called lkp_DIM_PRODUCT based on the target table, DIM_PRODUCT, which will be in the assigned target schema, tbuxx From the exp_DIM_PRODUCT transformation, copy/link CATEGORY, PRICE, PRODUCT_DEPARTMENT, PRODUCT_SEQUENCE and PRODUCT_NAME_MODEL to the lkp_DIM_PRODUCT Rename all of the ports that entered lkp_DIM_PRODUCT from the exp_DIM_PRODUCT Expression by replacing the “1” with “_in” Uncheck the Output Port “O” for each port that entered lkp_DIM_PRODUCT from the exp_DIM_PRODUCT Expression Turn on the Dynamic Lookup Cache property:
Turning on Dynamic Lookup Cache changes the
Ports Tab of the Lookup. A port called NewLookRow is inserted in the first position and the Associated Port column becomes active. •
Add the following conditions:
Note that for each condition, the Associate Port is filled in and grayed out on the Ports tab. • • •
Change the PRODUCT_SEQUENCE Datatype to Integer Remain on the Ports tab, click on the Associated port for PRODUCT_SEQUENCE and select PRODUCT_SEQUENCE_in Save the repository
Create Filter Transformation
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
295
• •
•
Create a Filter transformation called fil_DIM_PRODUCT Copy/Link the NewLookupRow, PRODUCT_SEQUENCE PRODUCT_CATEGORY, PRODUCT_NAME_MODEL, PRICE, PRODUCT_DEPARTMENT Ports from the lkp_DIM_PRODUCT transformation to fil_DIM_PRODUCT transformation Add the following condition on the Properties tab of the filter: NewLookupRow = 1
The Dynamic Lookup assigns the value 1, 2, or 0 to the NewLookupRow port to indicate if it inserts or updates the row in the cache, or makes no change, respectively. Link Target Definition
• •
Use the Autolink feature By Name to link the fil_DIM_PRODUCT to DIM_PRODUCT Target Definition Save the repository
Completed Mapping
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
296
Create Workflow • • • • • •
• • •
Create a Workflow called wf_DIM_PRODUCT_xx Add wf_DIM_PRODUCT_xx.log as the Workflow Log File Name Rename Start task to start_ DIM_PRODUCT Create Session Task called s_m_DIM_PRODUCT_xx Select native_target_xx connection for sq_ODS_PRODUCT source instance Select native_target_xx connection for DIM_PRODUCT target instance ° Check property Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link start_ DIM_PRODUCT to s_m_DIM_PRODUCT_xx Save, Validate and Start Workflow wf_DIM_PRODUCT_xx Monitor and Review the results for s_m_DIM_PRODUCT_xx in the Workflow Monitor
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
297
Results – Session Properties
Results – Session Transformation Statistics
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
298
Results – Preview Data
Informatica PowerMart / PowerCenter 6 Basics –Lab 23 – Dynamic Lookup
299
Lab 24
LAB 24 – SEQUENTIAL WORKFLOW PURPOSE •
Build a workflow to load the remaining dimensions table, and then the fact table to complete the course.
OBJECTIVE • •
Create shortcuts to mappings Setup a workflow with sequential loading
D U R AT I O N •
30 minutes
SUMMARY
Create Mapping Shortcuts
To complete the mini Data Warehouse or
DataMart, is to load the five required dimension tables, DIM_REP, DIM_CUSTOMER, DIM_VENDOR, DIM_INVOICE, and DIM_TIME and the fact table, FACT_SALES. The six mappings that load the tables have been created and reside in the Shared_Student folder. •
Create shortcuts and rename them (Mappings | Edit) replacing the xx to include the assigned student number from the following mappings: ° m_DIM_CUSTOMER_xx ° m_DIM_INVOICE_xx ° m_DIM_REP_xx ° m_DIM_TIME_xx ° m_DIM_VENDOR_xx
Informatica PowerMart / PowerCenter 6 Basics –Lab 24 – Sequential Workflow
300
° •
m_FACT_SALES_xx
Save
Create Workflow
If this training environment was a true
Development, QA or Production environment with enough processing power and memory to handle what is about to be created by each and every student, then a few of the dimension tables could be loaded concurrently. Since a typical training environment has limited resources, the following workflow will be loaded sequentially.
• • • •
Create a Workflow called wf_LOAD_DW_xx Add wf_LOAD_DW_xx.log as the Workflow Log File Name Rename Start task to start_LOAD_DW Create Session Task for each mapping shortcut with the names of: ° s_Shortcut_To_m_DIM_TIME_xx ° s_Shortcut_To_m_DIM_REP_xx ° s_Shortcut_To_m_DIM_VENDOR_xx
Informatica PowerMart / PowerCenter 6 Basics –Lab 24 – Sequential Workflow
301
s_Shortcut_To_m_DIM_CUSTOMER_xx ° s_Shortcut_To_m_DIM_INVOICE_xx ° s_Shortcut_To_m_FACT_SALES_xx Select native_source connection for s_Shortcut_To_m_DIM_VENDOR_xx Source Instance Select native_target_xx connection for each of the remaining Source Instances Select native_target_xx connection for each of the Dimension and Fact Target Instances ° For multiple practice loads, check property Truncate target table option ° Only if the target table database connection is using an Oracle client less than 8.1.7.2, select Normal for Target load type property Link sequentially from the start_ LOAD_DW in the following order: ° s_Shortcut_To_m_DIM_TIME_xx ° s_Shortcut_To_m_DIM_REP_xx ° s_Shortcut_To_m_DIM_VENDOR_xx ° s_Shortcut_To_m_DIM_CUSTOMER_xx ° s_Shortcut_To_m_DIM_INVOICE_xx ° s_Shortcut_To_m_FACT_SALES_xx Save, Validate and Start Workflow, wf_LOAD_DW_xx Monitor and Review the results for each of the session tasks in the Workflow Monitor °
• • •
•
• •
Results – Session Transformation Statistics for DIM_TIME Table
Results – Session Transformation Statistics for DIM_REP Table
Results – Session Transformation Statistics for DIM_VENDOR Table
Informatica PowerMart / PowerCenter 6 Basics –Lab 24 – Sequential Workflow
302
Results – Session Transformation Statistics for DIM_CUSTOMER Table
Results – Session Transformation Statistics for DIM_INVOICE Table
Results – Session Transformation Statistics for FACT_SALES Table
Informatica PowerMart / PowerCenter 6 Basics –Lab 24 – Sequential Workflow
303
INDEX Aggregator Transformation............ Purpose..................................154 Autolink......................................... by Name...................................89 by Name & Prefix or Suffix..........91 by Position................................92 Columns......................................... Copy & Link..............................98 Debugger....................................... Next Instance..........................124 Start......................................119 Stop.......................................127 Designer......................................... Find.......................................213 Find in Workspace....................213 Find Next................................216 Start........................................12 Using Shortcuts. 217, 219, 225, 226 Expression..................................... Create formula..........................40 Expression transformation............. Purpose....................................37 Filter Transformation..................... Create / Edit.............141, 145, 163 Flat File.......................................... Analyze..................................130 Flat File Wizard.............................. Purpose..................................130 Folder Create........................................8 Expanded.................................13 Open........................................12 Joiner Transformation.................... Create....................................151 Purpose..................................152 Lookup Transformation.................. Cache File Name Prefix.............273 Connected and Unconnected.....173 Create Connected.............172, 232 Create Unconnected..........247, 254 Dynamic Cached Lookup...........294 Persistent................................273 Recache from Database............273 Mapping......................................... Copy...............................111, 112 Create..............................30, 201
Debug....................................125 Edit........................................118 Validation.................................45 Mapping Designer.......................... Open........................................29 Mapping Parameter........................ Purpose..................................244 Using.....................................251 Mapplet.......................................... Create....................................265 Description..............................265 Mapplet Designer.....................265 Metadata Extensions...................... Create......................................54 Non-reusable............................20 Purpose....................................20 Reusable..................................20 Output Transformation................... Create....................................270 Ports Definition..................................35 Modify or Add............................39 Repository...................................... Connect to..................................8 Revert to Saved.........................86 Save........................................23 Repository Manager....................... Close.......................................10 Start..........................................7 Router Transformation................... Purpose..................................232 Session........................................... Properties.................................76 Status......................................76 Session Log.................................... View........................................78 Session Task.................................. Create......................................56 Edit..........................................57 Shortcut......................................... Create....................................227 Source Analyzer............................. Open........................................14 Source Qualifier Transformation.... Purpose....................................34 Sources..........................................
Informatica PowerMart / PowerCenter 6 Basics – Index
304
Edit definition............................18 Organize...................................31 Analyze...................................16 Targets........................................... Import Definitions......................26 Task Link.........................................60 Transformation.............................. Aggregator..............................154 Create....................................101 Expression................................36 Passive vs. Active......................37 Reusable.................................275 Sequence Generator.................293 Source Qualifier.........................34 Transformation Developer.............. Open......................................279 Purpose..................................275 Update Strategy Transformation.... Create....................................189 Forward Rejected Rows.............208 Warehouse Designer...................... Open........................................25 Workflow....................................... Edit..........................................54 Log..........................................54 Start........................................68 Validate....................................62 Workflow Manager......................... Start.................................50, 130 Workflow Monitor........................... Close.......................................67 Gantt Chart...............................70 Initialize...................................70 Purpose....................................75 Task View.................................71 Workspace..................................... Clear................................22, 110 Zoom and scale-to-fit..................... How to............................103, 107
Informatica PowerMart / PowerCenter 6 Basics – Index
305