Large Objects With Db2 Zos And Os390

Front cover

with DB2 for z/OS and OS/390 Define LOBs, see how they work, and see how to store them Manage LOBs in operational environments Use LOBs in applications and DB2 Extenders

Paolo Bruni Patric Becker Michael Dewert Bob Riehle

ibm.com/redbooks

International Technical Support Organization Large Objects with DB2 for z/OS and OS/390 June 2002

SG24-6571-00

Note: Before using this information and the product it supports, read the information in “Notices” on page xiii.

First Edition (June 2002) This edition applies to Version 7 of IBM DATABASE 2 Universal Database Server for z/OS and OS/390 (DB2 UDB for z/OS and OS/390 Version 7), program number 5675-DB2. © Copyright International Business Machines Corporation 2002. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

Contents Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv June 2002, First Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xv November 2004, First Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix Chapter 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Summary of considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Chapter 2. Large objects with DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Object orientation with DB2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Large objects with DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 The new data types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3.1 LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3.2 The ROWID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 3. Creating LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Data conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Storing LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 How to store them in DB2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Creating a table containing LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Creating the LOB table spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Creating the auxiliary tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.5 The auxiliary index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.6 Adding a LOB column to an existing table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 The CURRENT RULES special register . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Impact on CREATE and DROP LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Impact on cursors fetching LOB values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Feeding a LOB column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 LOAD with LOB columns smaller than 32 KB . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Inserting LOBs via host application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13 14 16 16 19 22 28 29 30 31 31 33 33 34 34

Chapter 4. Using LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 SQL semantics with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Getting to know LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Handling BLOBs, CLOBs, DBCLOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Manipulating a LOB without retrieving it . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43 44 46 47 52 52

© Copyright IBM Corp. 2002

iii

4.3.2 Unloading a LOB via application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Finding the nth occurrence of a string. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 LOBs are different DB2 objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Storage of LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.1 Locks with simple reads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Using ISOLATION (UR) for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.3 Locks with INSERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.4 Locks with DELETE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.5 Locks with UPDATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.6 Some general information about locking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Details about ROWID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

54 57 58 60 62 63 64 65 65 67 67 68

Chapter 5. DB2 Extenders and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 AVI Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Desktop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 WLM environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.4 Administration tasks to enable a table to use extenders. . . . . . . . . . . . . . . . . . . . 5.3 XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 XML Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 WLM application environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 General host requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.4 Workstation requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 XML and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73 74 74 76 76 76 77 81 83 83 83 84 84

Chapter 6. Data administration with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.1 The DB2 Catalog and LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.1 Some catalog definitions for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6.1.2 LOBs defined in DB2 catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.2 Utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2.1 LOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2.2 DSNTIAUL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.2.3 UNLOAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.2.4 COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.2.5 REPORT TABLESPACESET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.2.6 RUNSTATS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.2.7 REORG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.2.8 CHECK LOB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2.9 CHECK DATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.10 CHECK INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.2.11 DSN1COPY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.2.12 REPAIR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.2.13 RECOVERY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.3 Recovery scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.4 Operational scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Chapter 7. Performance with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 LOB materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Using data spaces for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Buffer pools and group buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 LOBs performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Logging with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

Large Objects with DB2 for z/OS and OS/390

119 120 123 125 127 128

7.3.2 7.3.3 7.3.4 7.3.5 7.3.6 7.3.7

LOBs processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LOBs read performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Comparing SQL accounting profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LOBs write performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IFCIDs enhancements for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LOBs recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129 130 131 132 133 136

Appendix A. Sample jobs output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 DDL for a LOB environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Sample of DB2 PM accounting trace output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Using file reference variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137 138 139 144

Appendix B. Unicode implementation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 B.1 Generate the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 B.2 Activate the UNICODE table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Appendix C. Recent maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 C.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 C.2 LOB related maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Appendix D. Additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Locating the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . System requirements for downloading the Web material . . . . . . . . . . . . . . . . . . . . . . . How to use the Web material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

155 155 155 156 156

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks collections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

161 161 161 162 162 162

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

Contents

v

vi

Large Objects with DB2 for z/OS and OS/390

Figures 2-1 2-2 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 5-1 5-2 5-3 5-4 6-1 6-2 6-3 6-4 6-5 6-6 6-7 7-1 7-2 7-3 7-4 7-5 7-6 7-7 7-8

© Copyright IBM Corp. 2002

Assigning a LOB locator for a LOB value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 LOB structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Mixed client /server environment with different data types . . . . . . . . . . . . . . . . . . . . 14 Unicode considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Association between base table and auxiliary table . . . . . . . . . . . . . . . . . . . . . . . . . 17 Base table with two LOB columns and the two associated LOB table spaces . . . . . 17 Partitioned base table containing two LOB columns . . . . . . . . . . . . . . . . . . . . . . . . . 18 Association between base table, ROWID, auxiliary table, and LOB table space . . . 19 Catalog description for table BOOK_BASE_TABLE . . . . . . . . . . . . . . . . . . . . . . . . . 21 Applying changes to a LOB table space created with LOG NO. . . . . . . . . . . . . . . . . 24 SYSIBM.SYSCOLUMNS contents for TBNAME= BOOK_AUX_TABLE . . . . . . . . . . 29 Index keys for an auxiliary index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Secondary chain of locators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Primary chain of locators containing secondary chains . . . . . . . . . . . . . . . . . . . . . . . 40 Concurrent LOB access using LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Accessing a LOB without a locator reference using ISOLATION (CS) . . . . . . . . . . . 56 Processing a LOB using a locator reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 LOB value spanned over pages using chunks and non-chunks . . . . . . . . . . . . . . . . 60 LOB data pages chunked together via space map and LOB map pages . . . . . . . . . 61 Lock escalation on LOBS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Scan lock sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Scan lock sequence using uncommitted read . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Insert lock sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Delete lock sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Locks acquired by mass delete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Lock sequence when updating a LOB column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Extender architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 XML columns with side tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 XML column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 XML collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Non-fragmented LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Recovery environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Recovering the auxiliary table with full image copy . . . . . . . . . . . . . . . . . . . . . . . . . 113 Rebuilding the index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Delete row from base table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Take a full image copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 DB2 materialization overview picture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 A SELECT application and LOB materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 An INSERT application and LOB materialization. . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Panel DSNTIP7 showing storage LOB values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Separation of LOB buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 LOB group buffer pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Read performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Write performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

vii

viii

Large Objects with DB2 for z/OS and OS/390

Tables 2-1 3-1 3-2 3-3 3-4 3-5 3-6 3-7 4-1 5-1 5-2 5-3 5-4 5-5 5-6 6-1 6-2 6-3 6-4 6-5 6-6 7-1 7-2 C-1 C-2

© Copyright IBM Corp. 2002

Typical average size for large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Choosing a LOB page size that minimizes getpages . . . . . . . . . . . . . . . . . . . . . . . . 26 Choosing a LOB page size for LOBs all of the same size . . . . . . . . . . . . . . . . . . . . . 26 Primary and secondary quantity with LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Summary of data set, partition, and partitioned table space sizes. . . . . . . . . . . . . . . 27 LOB table space created using CURRENT RULES STD . . . . . . . . . . . . . . . . . . . . . 32 Auxiliary index created using CURRENT RULES STD . . . . . . . . . . . . . . . . . . . . . . . 32 Where large objects come from and how . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Casting large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 UDTs used with AVI Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 DEFAULT UDTs after enabling the XML SERVER . . . . . . . . . . . . . . . . . . . . . . . . . . 81 XML Extender storage functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Storage UDFs used with XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Extract UDFs used with XML Extenders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 User defined types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Impact of logging on LOB tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 SYSIBM.SYSLOBSTATS columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Allowed keywords and options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Resetting AUXW flags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Resetting AUX CHECK pending status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Object status after different recovery scenarios. . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Two subsystem parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Current IFCID providing information regarding LOBs . . . . . . . . . . . . . . . . . . . . . . . 133 Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 DB2 V7 LOB related APARs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

ix

x

Large Objects with DB2 for z/OS and OS/390

Examples 2-1 2-2 3-1 3-2 3-3 3-4 3-5 3-6 3-7 3-8 3-9 3-10 3-11 3-12 3-13 3-14 3-15 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 4-13 4-14 4-15 4-16 5-1 5-2 5-3 5-4 5-5 5-6 5-7 5-8 5-9 5-10 6-1 6-2 6-3 6-4 6-5

© Copyright IBM Corp. 2002

Host variable definitions for LOB locators in COBOL . . . . . . . . . . . . . . . . . . . . . . . . What the DB2 precompiler makes of LOB locators . . . . . . . . . . . . . . . . . . . . . . . . . . DDL for a base table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DDL for a LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DDL for an auxiliary table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DDL for an auxiliary table containing data of one base table partition. . . . . . . . . . . . DDL for an auxiliary Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying a database for LOBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding a ROWID column. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding a LOB column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LOAD a base table containing a LOB columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tieing rows together in one variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . host variable declaration for a LOB column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What the DB2 precompiler makes of your host variable declaration . . . . . . . . . . . . . Inserting a single LOB value using one host variable . . . . . . . . . . . . . . . . . . . . . . . . Pseudo-code inserting LOBs > 32 KB with one locator chain . . . . . . . . . . . . . . . . . . Pseudo-code inserting LOBs > 32 KB with multiple locator chains . . . . . . . . . . . . . . Syntax for FREE locator and HOLD locator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Delete Chapter 8 of book CLOB using locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Updating a part of a CLOB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inserting a text at a particular position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unloading LOB data using one host variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unloading a LOB using locators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finding a specific occurrence of a string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DDL for a table containing a ROWID column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Inserting a row in CUSTOMER table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ROWID value of a table created with ROWID column. . . . . . . . . . . . . . . . . . . . . . . . DSN1PRNT of ROWID in hex value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ALTER TABLE adding a ROWID column . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ROWID value of a table where a ROWID column was added . . . . . . . . . . . . . . . . . . DSN1PRNT of ROWID in hex value after adding and updating a row . . . . . . . . . . . ROWID values of updated and inserted columns . . . . . . . . . . . . . . . . . . . . . . . . . . . DSN1PRNT of updated and inserted ROWIDs in hex value . . . . . . . . . . . . . . . . . . . Sample DMBWLM1 procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DMBENVAR DD card input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of ENABLE SERVER command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the table for an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of ENABLE TABLE command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example Enable Column Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Storing an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Browsing an image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . WLM procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Retrieval of an XML column document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Select from SYSIBM.SYSCOLUMS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Select from SYSIBM.SYSTABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DB2 catalog query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Output for Load job inserting into table with CLOB column . . . . . . . . . . . . . . . . . . . . Output from Unload with LOB column < 32 KB . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

11 11 20 22 28 28 29 30 30 31 34 35 35 35 36 36 40 49 52 53 54 54 55 57 69 69 69 69 70 70 70 70 71 76 77 77 79 79 79 80 80 83 87 92 93 93 95 96 xi

6-6 6-7 6-8 6-9 6-10 6-11 6-12 6-13 6-14 6-15 6-16 6-17 6-18 6-19 6-20 7-1 7-2 7-3 7-4 7-5 7-6 7-7 7-8 A-1 A-2 B-1 B-2 B-3 B-4 B-5

xii

Output from DSNTIAUL where column length > 32 KB . . . . . . . . . . . . . . . . . . . . . . . 96 Using UNLOAD with truncation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Using the UNLOAD utility on a base table with row size > 32 KB . . . . . . . . . . . . . . . 97 Copy of base and LOB table space in one statement . . . . . . . . . . . . . . . . . . . . . . . . 98 Query of SYSCOPY for common RBA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 REPORT TABLESPACESET with CURRENT RULES STD created LOBs . . . . . . . 99 Table SYSIBM.SYSLOBSTATS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Trying to use REORG UNLOAD EXTERNAL on a base table row > 32 KB . . . . . . 104 DSN1COPY error for LOB table space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 JCL for DSN1COPY OBIDXLAT for LOB table space . . . . . . . . . . . . . . . . . . . . . . . 108 Display database after recovery, using last full image copy . . . . . . . . . . . . . . . . . . 113 Display database after Rebuild Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Check Data utility output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Display database after Check Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Display database at the end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 MVS display command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 DB2 PM accounting miscellaneous values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 DB2 PM statistics miscellaneous values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Calculation for LOB space maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Calculation of group buffer pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Inserting a 1 MB LOB with LOG YES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Select into host variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Accounting trace output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Enabled Extender tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Use of file reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 JCL to create the conversion table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Output from the generation job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Contents of CUNUNI01 in SYS1.PARMLIB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Activate the new UNICODE table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 UNI=ALL output in SYSLOG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

Large Objects with DB2 for z/OS and OS/390

Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces.

© Copyright IBM Corp. 2002.

xiii

Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX® CICS® DB2® DB2 Connect™ DB2 Extenders™ DFS™ DFSMS/MVS® DRDA® Enterprise Storage Server™ e (logo)®

IBM® IMS™ MVS™ OS/390® Perform™ QBIC® QMF™ RACF® RAMAC® Redbooks (logo)™

Redbooks™ S/390® Sequent® SP™ System/390® VTAM® WebSphere® z/OS™

The following terms are trademarks of International Business Machines Corporation and Lotus Development Corporation in the United States, other countries, or both: Lotus®

Lotus Notes®

Notes®

The following terms are trademarks of other companies: ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. Other company, product, and service names may be trademarks or service marks of others.

xiv

Large Objects with DB2 for z/OS and OS/390

Summary of changes This section describes the technical changes made in this edition of the book and in previous editions. This edition may also include minor corrections and editorial changes that are not identified. Summary of Changes for SG24-6571-00 for Large Objects with DB2 for z/OS and OS/390 as created or updated on November 9, 2004.

June 2002, First Edition This revision reflects the addition, deletion, or modification of new and changed information described below.

November 2004, First Update Changed information Change bars identify the corrected areas. 򐂰 Corrected information on Reorg and Recover execution against LOBs in the text following 6.2.7, “REORG” on page 100 and removed the previously numbered 6-3 Figure.

© Copyright IBM Corp. 2002. All rights reserved.

xv

xvi

Large Objects with DB2 for z/OS and OS/390

Preface The amount of data to be managed by workstations, middleware systems and mainframes, keeps growing day by day. The times are changing, and so the requirements for a database management system (DBMS) are changing. With the incorporation of object-orientation into DB2 for z/OS and OS/390 (simply referred to as DB2 throughout this redbook), you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 UDB Server for OS/390 Version 6 has two major functions. On one hand it has support for large objects (LOB); and, on the other hand it has support for user defined functions, user defined distinct types, and triggers. These large objects can contain pictures, images, text-documents or even movies, and can be stored directly in the DBMS with sizes up to 2 gigabytes per object and 4,000 terabytes per LOB column. The largest available size in DB2 before LOBs was 32 kilobytes, for data defined as a varchar. Normally, large objects are used and manipulated through graphical user interfaces from a workstation. So with the implementation of LOBs we can exploit the functionalities of the DB2 family and the enterprise server capacity that DB2 for z/OS provides today. The introduction of these new data types implies some changes in the administration processes and programming techniques. In this IBM Redbook we describe the new data types, and provide some useful information on how to design and implement LOBs. We also offer examples of their use, programming considerations, and the new processes which are necessary for administration and maintenance.

The team that wrote this redbook This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Paolo Bruni is a Certified Consultant IT Architect on assignment at the International Technical Support Organization, San Jose Center, where he conducts projects on all areas of Data Management. Before joining the ITSO in 1998, Paolo worked at IBM Italy. During his many years with IBM, in development and in the field, Paolo’s work has been principally related to database systems. Patric Becker is an Application Programmer with Sparkassen Informatik GmbH and Co. KG in Germany. Since joining the company in 1997, Patric has been responsible for several, high availability, Customer Relationship Management, DB2 and IMS applications. For DB2, he has also assumed the role of a database administrator and he was involved in evaluating and applying the new functions of DB2 for OS/390 Version 6 and Version 7. Michael Dewert is a Systems Programmer with more than 12 years experience with DB2 for OS/390. Before joining IBM Germany in 2001, Michael has worked for a consortium of Savings Banks in Northern Germany and his responsibilities included IMS and CICS as well as DB2. More recently he was involved in DB2 Quality Partnership Programs and his area of expertise has been expanded to include performance tools, data sharing, disaster recovery, and distributed platforms.

© Copyright IBM Corp. 2002

xvii

Bob Riehle is a Consultant with 10 years of experience in DB2, having previously worked with IMS for several years. His areas of special interest within DB2 include performance, applications, and subsystem tuning. Bob’s work with his customers ranges from systems high-level architecture and connectivity, to database design and performance, largely related to DB2. Thanks to the following people for their contributions to this project: Emma Jacobs Yvonne Lyon Bart Steegmans International Technical Support Organization, San Jose Center Rich Conway Bob Haimowitz International Technical Support Organization, Poughkeepsie Center Gabrielle Velez International Technical Support Organization, Rochester Center Chuck Bonner Karelle Cornwell Chris Crone Willie Favero Susan Malaika Roger Miller Akira Shibamyia Kalpana Shyam Dave Wang Jane Wang Jay Yothers Gary Wilmot IBM Silicon Valley Laboratory, San Jose Ian Cook Sarah Ellis IBM PIC, Hursley, UK Bernhard Baumgartner IBM Switzerland Toine Michielse Independent Consultant, The Netherlands

Become a published author Join us for a two- to seven-week residency program! Help write an IBM Redbook dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will team with IBM technical professionals, Business Partners and/or customers. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you'll develop a network of contacts in IBM development labs, and increase your productivity and marketability.

xviii

Large Objects with DB2 for z/OS and OS/390

Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html

Comments welcome Your comments are important to us! We want our Redbooks to be as helpful as possible. Send us your comments about this or other Redbooks in one of the following ways: 򐂰 Use the online Contact us review redbook form found at: ibm.com/redbooks

򐂰 Send your comments in an Internet note to: [email protected]

򐂰 Mail your comments to: IBM Corporation, International Technical Support Organization Dept. QXXE Building 80-E2 650 Harry Road San Jose, California 95120-6099

Preface

xix

xx

Large Objects with DB2 for z/OS and OS/390

1

Chapter 1.

Introduction With the incorporation of object-orientation into DB2 for OS/390, you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 for OS/390 Version 6 is, on one hand, support for large objects (LOBs); and, on the other, support for user defined functions (UDFs), user defined distinct types (UDTs), and triggers. For more information on UDFs, UDTs, and triggers, please refer to DB2 for z/OS Application Programming Topics, SG24-6300. This redbook describes the major enhancements that affect application programming when accessing DB2 data on S/390 or z/Series platform, and, while not dealing directly with LOBs, it includes the object-oriented extensions such as triggers, user-defined function and user-defined distinct types, the use of temporary tables, savepoints, and the numerous functions of the SQL language that can help you build powerful, reliable and scalable applications, whether it be in a traditional environment or on an e-business platform. Large objects can contain pictures, images, text-documents or even movies, and can now be stored directly in DB2 with sizes up to 2 GB per object and 4,000 Terabytes per LOB column. The largest available size in DB2 before LOBs was 32 Kilobytes, for data defined as a varchar. Normally, large objects are used and manipulated through graphical user interfaces (GUI) via a Workstation. So with the implementation of LOBs, we can exploit the server capacity that DB2 for OS/390 and z/OS provides today while displaying and feeding them through a user friendly interface. The introduction of these new data types, with their peculiarities, in a traditional environment has some impact on the current, established, administration processes and programming techniques. In this book we describe the various new data types introduced for LOBs support, and provide useful information on how to design and use them. We offer some examples of their use, programming considerations, and the new processes which are necessary for their implementation, administration, and maintenance.

© Copyright IBM Corp. 2002

1

This redbook is structured as follows: 򐂰 Large objects with DB2 In Chapter 2 we provide some preliminary information on object orientation, large objects, and the new data types that DB2 V6 has introduced for their support. 򐂰 Creating LOBs In Chapter 3, we provide information on how to define LOBs, how they are structured in terms of DB2 objects, the impact of the CURRENT RULES special register settings, and some techniques in feeding data into the LOBs. 򐂰 Using LOBs In Chapter 4 we discuss topics related to the differences in using LOBs, when compared with using normal DB2 objects — in particular the SQL semantics, LOB locators, locking, and ROWID. 򐂰 DB2 Extenders and LOBs In Chapter 5 we provide a description of how DB2 Extenders utilize LOBs in the z/OS environment. We look at the setup and administration of the AVI and XML Extenders. 򐂰 Data Administration with LOBs In Chapter 6 we document the differences in administering table spaces pertaining to tables containing LOB columns. We look at the DB2 Catalog, we examine the impact on the various utilities, and document the changes introduced for LOBs support. 򐂰 Performance with LOBs In Chapter 7 we discuss topics directly and indirectly related to LOBs performance. We look at LOB materialization, bufferpool and dataspace use, and then we conclude with more performance specific considerations and recommendations.

1.1 Summary of considerations Before you start implementing LOBs in your DB2 system, be aware that they are new objects, and that you need planning and preparation work to add the set up needed for a successful LOBs implementation to your current environment. There is some cost associated with introducing LOBs support in a production shop. In this section we summarize our findings with recommendations on how to reduce their impact, we point out their differences from the usual data requirements, and warn about possible misuse.

Stay current with maintenance LOB support is extremely pervasive and involves all areas of DB2, with repercussions on the operating system, language support, Extenders, data conversion, client/server connectivity, and back-up and recovery. Given the permutations of functions and data definitions, chances are that you may be exploring new territories. Applying current maintenance and prototyping your application will help you in avoiding pitfalls and gaining valuable operational experience.

Client/server solution Since you are dealing with new large objects, like text or images, it is likely that you need the GUI capabilities of a front end workstation for presenting them. In this type of client/server solution you may also consider the added value of DRDA connectivity for dealing with data conversion, and the file reference variables supported by SQL when feeding data to the host via SQL INSERTs from the distributed DB2. For this type of solution, bandwidth and the speed of your connections become critical, and, with different platforms involved and maybe several geographies, data conversion needs to be verified. 2

Large Objects with DB2 for z/OS and OS/390

Do not misuse LOBs A major recommendation is to use large objects for what they are intended to be: objects with a size bigger than 32 KB. If you use a LOB environment for objects smaller than 32 KB, small LOBs, or SLOBs, you will not benefit from LOBs’ specific design when DB2 accesses them, but you incur some penalties since DB2 assumes that they are real large objects. For instance, you would waste space when storing a column smaller than 32 KB in a LOB column, because a page in a LOB table space can only contain one row worth of data. Also, since LOBs are not physically stored along with other columns of a table, you would create additional overhead when you store frequently accessed columns in a table different from the one containing your other data. Normally, LOB columns are less frequently accessed than other data in your tables.

Logical data design If you plan to use a LOB column for denormalization of your data, because varchar was not big enough to hold the amount of data, and LOBs are, it may be more effective to reconsider the design. Be aware that using LOBs for denormalization does not correspond to their intended use and may result in ineffectiveness.

MVS considerations Verify that you have adequate memory, disk space, and processor capacity before introducing the new LOBs application. When accessing LOBs, DB2 can and will activate data spaces. The activation of data spaces will introduce new messages at the system console and it may influence paging, and you need to talk to your MVS system programmer and agree before hand on the set up of the few new, but important, DB2 and MVS parameters. Before you start implementing LOBs, you should evaluate the DSNZPARMs LOBVALA (maximum data space size per user), and LOBVALS (maximum data space size per system), they will help you in gauging the virtual storage utilized by the data spaces created by DB2 when dealing with LOBs.

CURRENT RULES The value of your CURRENT RULES special register has a major influence when you deal with LOBs. The system default is set to DB2, unless you have specified SQLRULES STD when binding your local plan. You need to evaluate the implication of the value specified for this parameter because it influences the way LOBs are created and dropped. It also has an impact on how applications can deal with LOBs when they use cursors for accessing their values. You may also consider using the CURRENT RULES special register for temporarily overwriting the default value specified for the plan. You need the STD DB2 option active if you want to define objects with your own naming standards.

Application programming challenges Programming for LOBs is a bit more challenging than accessing data from a standard table. Your application programmers have to deal with the new data types as well as virtual space requirements and possible use of LOB locators. Knowledge of locators, their functional description and their impact on address spaces can be very useful to avoid a large consumption of memory and refrain from creating bottlenecks in your system.

Impact of logging You have the option of de-activating logging for LOBs. This makes sense for very large objects which, when inserted or updated, may affect your system with very heavy logging. If you plan to log all changes being made on your LOB values, check back with your database administrator for relieving bottlenecks on DB2’s log data sets by using fast devices and striping. On the other hand, if you do not log, you should be aware of the restrictions imposed

Chapter 1. Introduction

3

on recovery and availability of your LOBs data. Do not define objects as large to simply avoid logging. The disadvantages for your operational data will certainly outweigh the uncertain savings with non-logging. For instance, a full image copy taken with SHRLEVEL CHANGE stays permanently fuzzy without the log.

Backup and recovery scenarios Large objects are stored in a different way compared with other DB2 objects. When you implement a large object structure in your current environment, backing up all table spaces which are involved in LOB processing (there are at least two per LOB column) to ensure consistency is only one side of the scenario. The other side is recovering a set of table spaces associated to a LOB column for which new table space states have been introduced: the auxiliary warning (AUXW) and auxiliary check pending (ACHKP). These states need to be dealt with by the DBA during well rehearsed recovery scenarios.

LOBs and data sharing If you plan to introduce LOBs in a data sharing environment, which is becoming more and more popular, have a close look at parameters which can influence your whole data sharing group. Probable limits and values for additional parameters and buffer pools should be set to values that do not interfere with other components of your system environment.

The LOBs are here! If you have no application using DB2 large objects in your environment, and you think you have no LOBs in your DB2 subsystem, you may be surprised to find out that they do already exist in your DB2 catalog if you have DB2 V7 installed. In this release, if you look closely at the catalog tables, you will find LOB tables already defined within DSNDB06. They are filled in by DB2 when you define Java stored procedures. LOBs are also used by DB2 Extenders, optionally by the Image, Audio, and Video Extenders, while the XML Extender defines LOBs as repositories for documents.

4

Large Objects with DB2 for z/OS and OS/390

2

Chapter 2.

Large objects with DB2 In this chapter we provide some introductory information on object orientation, large objects, and the new data types that DB2 V6 has introduced for their support. This chapter is structured by: 򐂰 Object orientation with DB2 򐂰 Large objects with DB2 򐂰 The new data types

© Copyright IBM Corp. 2002

5

2.1 Object orientation with DB2 The object extensions introduced by DB2 UDB for OS/390 Version 6 offer the benefits of object-oriented technology while increasing the strength of your relational database with an enriched set of data types and functions: 򐂰 Large objects The VARCHAR and VARGRAPHIC data types have a storage limit of 32 KB. Although this may be sufficient for small- to medium-size text data, applications often need to store large text documents. They may also need to store a wide variety of additional data types such as audio, video, drawings, mixed text, graphics, and images. DB2 provides new data types to store these large data objects (LOBs) as strings of up to 2 GB in size. LOBs are well suited to represent large, complex structures in DB2 tables. You can make effective use of multimedia by storing objects such as complex documents, videos, images, and voice. 򐂰 User defined types User defined types (UDTs), like built-in data types, describe the data that is stored in columns of tables where the instances (or objects) of these data types are stored. They ensure that only those functions and operators that are explicitly defined on a distinct type can be applied to its instances. A distinct type is a user-defined data type that shares its internal representation with a built-in data type, but is considered to be a separate and incompatible type for semantic purposes. For example, you may want to define a picture type or an audio type which have different semantics but use the built-in data type BLOB for their internal representation. 򐂰 User defined functions User defined functions (UDFs), like built-in functions or operators, support manipulation of distinct type instances (and built-in data types) in SQL queries. The built-in functions that are supplied with DB2 are a useful set of functions, but they may not satisfy all of your requirements. You can write user-defined functions to meet the specific needs for your installation. For example, a built-in function may perform a calculation you need, but the function does not accept the distinct types you want to pass to it. You can then define a function based on a built-in function, called a sourced user-defined function, that accepts your distinct types. You may need to perform another calculation in your SQL statements for which there is no built-in function. In that situation, you can define and write an external user-defined function. New and extended built-in functions improve the power of the SQL language with about 100 new built-in functions, extensions to existing functions, and sample user-defined functions. 򐂰 Triggers Triggers bring application logic into the database. Triggers automatically execute a set of SQL statements whenever a specified event occurs. These statements can validate and edit database changes, read and modify the database, and invoke functions that perform operations inside and outside the database. 򐂰 Extenders Text Extenders add the power of full-text retrieval to SQL queries by making use of features available in DB2 that let you store unstructured text documents of up to 2 GB in databases.

6

Large Objects with DB2 for z/OS and OS/390

Text Extender offers DB2 users and application programmers a fast, versatile, and intelligent method of searching through such text documents. Text Extender's strength lies in its ability to search through many thousands of large text documents at high speed, finding not only what you directly ask for, but also word variations and synonyms. You are not restricted to searching only in text documents stored in DB2 for z/OS databases, you can also search in text documents stored in files. Text Extender can access any kind of text document, including word-processing documents in their original native form, and offers a rich set of retrieval capabilities including word, phrase, wildcard, and proximity searching using Boolean logic. At the heart of Text Extender is IBM's high-performance linguistic search technology. It allows your applications to access and retrieve text documents in a variety of ways. Your applications can: – Search for documents that contain specific text, synonyms of a word or phrase, or sought-for words in proximity, such as in the same sentence or paragraph. – Do wildcard searches, using front, middle, and end masking, for word and character masking. – Search for documents of various languages in various document formats. – Make a fuzzy search for words having a similar spelling as the search term. This is useful for finding words even when they are misspelled. – Make a free-text search in which the search argument is expressed in natural language. – Search for words that sound like the search term. You can use the DB2 Extenders feature of DB2 to store and manipulate image, audio, video, and text objects. The extenders automatically capture and maintain object information and provide a rich body of APIs. The other extenders in the family let you search for a combination of image, video, and voice data types in one SQL query. They include (AVI) audio, video, and image, as well as XML. These extenders define new data types and functions using DB2's built-in support for user-defined types and user-defined functions. You can couple any combination of these data types, that is, image, audio, and video, with a text search query. The extenders exploit DB2’s support for large objects of up to 2 GB, and for triggers that provide integrity checking across database tables ensuring the referential integrity of the multimedia data.

2.2 Large objects with DB2 Within today's multimedia application environments the dependency relies on many types of large data objects. Those objects may be large text documents, X-ray images, audio messages, pictures and many other images — you name them they are there. Table 2-1 shows some representative sizes of various objects. These are outlined just for comparison purposes. Overall they just give you some idea of the storage requirements of various types of objects that you may have to deal with.

Chapter 2. Large objects with DB2

7

Table 2-1 Typical average size for large objects Object

From

To

Bank checks

30 KB

40 KB

Small image

30 KB

50 KB

Large image

200 KB

3 MB

Color image

20 MB

40 MB

Radiology image

40 MB

60 MB

Video

1 GB / hour

-

Feature length movie

2 GB

-

High resolution video

3 GB / hour

-

High resolution movie

5 GB

6 GB

High definition TV

200 MB / second

-

The data types previously provided by DB2, such as VARCHAR, are not large enough to hold this amount of data, the limit being 32 KB. LOBs support is based on the new set of data types which were introduced with DB2 V6. With large objects support, DB2 stores and manipulates data objects that are much larger. Starting with V6, DB2 provides three data types to store large data objects (LOBs) which support strings of up to 2 GB in size, well beyond the 32 KB supported by a varchar column. New techniques for storing these huge amounts of data have also been created within DB2. The three new data types allow you to store directly in DB2 tables objects in size of up to 2 GB, and 4,000 Terabytes (TB) per LOB column. Their characteristics depend on the way they are stored in your DB2 subsystem: 򐂰 Character Large Objects, called CLOBs 򐂰 Binary Large Objects, called BLOBs 򐂰 Double Byte Character Large Objects, called DBCLOBs DB2 also provides new kinds of data types that are used for better storing and manipulating LOBs: 򐂰 LOB locators 򐂰 ROWIDs

2.3 The new data types In this section we introduce the new data types available in DB2 for LOB support.

CLOBs A character large object (CLOB) is a varying length string measured in bytes that can be up to 2 GB long. A CLOB is used to store large SBCS or mixed character-based data, such as documents written with a single coded character set identifier (CCSID). A CLOB is considered to be a character string.

8

Large Objects with DB2 for z/OS and OS/390

Most text documents stored on other platforms cannot be converted to a CLOB value easily because most known editing applications save data using their own format. This format usually contains an interspersed amount of control information used for font types, font sizes and layout purposes. If you want to store this data as a CLOB value and access it via DB2 functions such as SUBSTR or POSSTR, you may have to convert the data from their format on your PC to a plain text file before inserting into a CLOB column. You may consider using a CLOB column for storing large text documents or even Extended Markup Language (XML) documents. If you plan to store your PC documents in DB2 for z/OS without converting them, because you want to access them from your client applications for further processing, you need to store them as BLOBs to avoid loss of font and layout control information.

BLOBs A binary large object (BLOB) is a varying length string measured in bytes that can be up to 2 GB long. A BLOB is primarily intended to hold non-traditional data such as pictures or images, sounds or voices, or other types of mixed-media. All these objects eventually consist of unreadable binary data strings. When you create BLOBs, the data is stored in a LOB column as it is passed to DB2, without any data conversion. The same rule applies when you retrieve the data. Differently from CLOBs, the values stored in BLOB columns do not go through conversion routines when accessed by DB2. Another use of BLOBs is to hold data for exploitation by distinct types and user-defined functions. Normal character strings defined with the FOR BIT DATA option cannot be assigned a CCSID, and the same rule applies to BLOB strings. A BLOB contains a binary string representing binary data, a sequence of bytes, normally being unreadable. Common character strings defined FOR BIT DATA, which may be used for similar purposes, are not compatible with data type BLOB. However, the CAST specification can be used to cast a character string to a binary string.

DBCLOBs A double-byte character large object (DBCLOB) is a varying length string of double-byte characters that can be up to 2 GB long. Using a DBCLOB column you can store up to 1,073,741,824 double-byte characters in a single DBCLOB value. A DBCLOB is used to store large double byte character set (DBCS) character-based data such as documents written with a double-byte CCSID. A DBCLOB is considered to be a graphic string. You can find double-byte CCSIDs for languages like Japanese (extended Katakana or Katakana-Kanji), Korean, or Chinese (simplified or traditional).

2.3.1 LOB locators In general, when the application is accessing data, DB2 has to deal with the materialization of data across several layers: auxiliary storage, central storage, data spaces, and the movement of data across them. The materialization of LOB values consists of placing LOB data in contiguous storage. Since LOB values can be very large, DB2’s actions are inspired to the objective of avoiding materialization of LOB data until it becomes absolutely necessary. Furthermore, if the application is running on a client machine, distinct from the database server, the transfer of LOB values from the server to the client adds a sizeable time and resource consumption. Application programs typically process LOB values a piece at a time, rather than as a whole. For all those cases where an application does not need (or want) the entire LOB value to be stored in application memory, the application can reference a LOB value via the new large object locator (LOB locator) and avoid materialization.

Chapter 2. Large objects with DB2

9

A LOB locator is a host variable with a value that represents a single LOB instance in the database server. LOB locators have been developed to provide users with a mechanism by which they could easily manipulate very large objects in application programs without requiring them to store the entire LOB value in the application’s memory. You can consider locators roughly as cursors that reference remote data. Locators, and the structures associated to them, are contained in virtual storage blocks allocated by DB2 in the DBM1 address space. The LOB data referenced by the locators is allocated by DB2 in data spaces which are dynamically created for the duration the request.

Purpose of LOB locators A LOB locator is a value generated by DB2 when a LOB string is assigned to a host variable that was previously specified as a LOB locator in your application program. Every LOB locator, also called host-identifier, is a special type of SQL object materialized as a four byte token used to represent the LOB. This representation allows the value of the large object strings to be contained in the host-identifier, rather than the actual string of bytes of the large object. Figure 2-1 provides an example of LOB locator usage.

Application

DB2

1 GB BOOK_TEXT SELECT INTO FROM WHERE

BOOK_TEXT :BOOK_TEXT_LOCATOR BOOK_BASE_TABLE BOOK_NO = :HV_BOOK_NO

:BOOK_TEXT_LOCATOR

Internal pointer

987654321 987654321

Figure 2-1 Assigning a LOB locator for a LOB value

A LOB locator is an association between a host variable and its corresponding LOB value in DB2 at a point in time. The application only accesses the locator while the entire LOB value resides in DB2 and is not propagated to the application program. Using this technique, an application does not need to acquire a buffer large enough to contain the entire LOB value. Instead the application deals only with LOB locators therefore largely reducing the amount of resources to be allocated. The definition and usage are not mandatory; however, considerations on performance will soon lead you to the conclusion that it is a good idea for the applications to use them, since locators dramatically reduce the movement of data between the different address spaces involved in LOB data management as well as greatly reducing connectivity issues.

10

Large Objects with DB2 for z/OS and OS/390

Different types of LOB locators Each LOB format (BLOB, CLOB and DBCLOB) has its own type of locator, so currently there are three different types of LOB locators: 򐂰 Binary Large Object locator, which is associated with large binary strings 򐂰 Character Large Object locator, which is associated with large character strings 򐂰 Double Byte Character Large Object locator, which is associated with large graphic strings The definition of a LOB locator depends on the language you choose for developing programs for LOBs usage. The syntax for defining LOB locators in COBOL is shown in Example 2-1. Example 2-1 Host variable definitions for LOB locators in COBOL 01 BLOB-LOCATOR USAGE 01 CLOB-LOCATOR USAGE 01 DBCLOB-LOCATOR USAGE

IS SQL TYPE IS BLOB-LOCATOR. IS SQL TYPE IS CLOB-LOCATOR. IS SQL TYPE IS DBCLOB-LOCATOR.

The DB2 precompiler converts the locator structure into the COBOL structure as reported in Example 2-2. Example 2-2 What the DB2 precompiler makes of LOB locators 01 BLOB-LOCATOR 01 CLOB-LOCATOR 01 DBCLOB-LOCATOR

PIC S9(9) COMP. PIC S9(9) COMP. PIC S9(9) COMP.

The locator 4-byte value is then stored in a host variable; the program, as already shown in Figure 2-1, can use it to refer to a LOB value. Even if every LOB locator shows up identical for all definitions of host variables, DB2 knows the associated LOB type and will not let you use a locator with a different type of LOB. If you define a CLOB locator and try to use it for a BLOB, SQLCODE -171 is issued. You can only use LOB locators inside an application program, you cannot deal with them interactively. This means that you cannot use them, for instance, with SPUFI or QMF. For more information on LOB locators refer to 4.2, “LOB locators” on page 46.

2.3.2 The ROWID The ROWID data type (and column) definition was introduced with DB2 V6 to uniquely and permanently identify a row in a table. To understand the role of the ROWID, and why DB2 creates a value for a ROWID column whenever a row is inserted in a table containing this new data type, you must first look at and understand the overall picture of the DB2 objects involved in supporting LOBs, as illustrated in Figure 2-2. LOB data is contained in a LOB column which is conceptually part of the base table, but it is physically stored in a separate table. Because it is not part of the base table, it is called LOB table or auxiliary table. The auxiliary table resides in a separate LOB table space. A base table may be associated with many LOB or auxiliary columns of different types and lengths. Each auxiliary column is stored in its own auxiliary LOB table in its own LOB table space. An auxiliary index must be created on every auxiliary table before it can be used. To create a base table that contains a LOB column, you must define a ROWID column. The ROWID acts as, but it is not, a bidirectional pointer to the LOB data associated with the particular row: the LOB column values are associated with the proper base table row in both directions using the base table row’s ROWID value.The auxiliary index, whose key is based on the ROWID, is used to navigate to LOB data associated with the row.

Chapter 2. Large objects with DB2

11

If a base table that contains LOB data is partitioned, you create a separate LOB table space and auxiliary table for each LOB column in each partition. A LOB table space can have a page size of 4, 8, 16 or 32 KB. Since the length of a LOB can exceed 32 KB, it is clear that a LOB can span physical pages, and since there is only one row per page, with LOBs, rows can span pages. LOB pages only contain one LOB value, or row.

Base table space Base table Key

ROWID

Column_2

Key A Key B

LOB 1 value user data A LOB 2 value user data B

Auxiliary index: based on ROWID used to navigate to LOB data

LOB indicator LOB indicator 1 LOB indicator 2

Rows represent LOBs LOBs stored outside base table in auxiliary table Base table space may be partitioned If so, separate LOB table space for each part

LOB table space Auxiliary table ROWID LOB 1 value LOB 2 value

LOB data LOB data for row user data A LOB data for row user data B

Figure 2-2 LOB structure

More information about ROWIDs is reported in 4.6, “Details about ROWID” on page 68.

12

Large Objects with DB2 for z/OS and OS/390

3

Chapter 3.

Creating LOBs In this chapter, we provide information on how to define and load LOBs. We discuss the following topics: 򐂰 򐂰 򐂰 򐂰

Data conversion Storing LOBs CURRENT RULES special register Feeding a LOB column

© Copyright IBM Corp. 2002

13

3.1 Data conversion DB2 for z/OS is often used as the enterprise server of large client server systems. In these environments, character representations may vary on clients and servers across different platforms and across many different geographic locations. In this scenario you may also want to deal with data conversion topics. Large objects can also be subject for conversion topics, depending on the type of data you plan to store in your large objects. See Figure 3-1.

Mixed client/server environment CCSID 273

CCSID 850

CCSID 500

Figure 3-1 Mixed client /server environment with different data types

One area where this sort of environment exists is in data centers of multinational companies. Another example is e-commerce. In both of these examples, a geographically diverse group of users interact with a central server, storing and retrieving data. Today, there are hundreds of different encoding systems. No single encoding could contain enough characters: for example, the European Union alone requires several different encoding schemes to cover all its languages. Even for a single language like English, no single encoding was adequate for all the letters, punctuation, and technical symbols in common use. These encoding systems also conflict with one another. That is, two encoding schemes can use the same number for two different characters, or use different numbers for the same character.

14

Large Objects with DB2 for z/OS and OS/390

DB2 V5 introduced support for storing data in ASCII. This support only solved part of the problem: padding and collating. It did not address the problem of users in many different geographies storing and accessing data in the same central DB2 server. With DB2 V6 and LOBs, chances are that LOBs are being moved to the host from other platforms or other geographies, and a verification of the needed data conversion is recommended. As we mentioned in 2.3, “The new data types” on page 8, a LOB is a string data element just like the more familiar string data elements, such as CHAR or VARCHAR, except that a large object may be as much as 2 GB long. The three new data types that are used to identify large objects are CLOBs, BLOBs, and DBCLOBs. BLOBs are designed to contain binary data. As such, they have no CCSID associated with them. CLOBs and DBCLOBs are designed to contain text data. CLOBs have the normal single byte and mixed CCSIDs associated with them, while DBCLOBs have the graphic CCSID associated with them. For encoding and decoding, DB2 uses the table SYSIBM.SYSSTRINGS for conversion to and from CCSIDs. This has not changed from the past releases of DB2. Within DB2 V7 there is Unicode support. This also requires OS/390 Version 2.8 or above. Example 3-2 gives an idea of the impact using Unicode.

DB2 Unicode support Japan

France

United States

Germany China

Figure 3-2 Unicode considerations

Unicode provides a unique number for almost every character, on any platform, in any language, and by any program. The Unicode character encoding standard is a character-encoding scheme that includes characters from almost all living languages. It is an implementation of the ISO-10646 standard.

Chapter 3. Creating LOBs

15

There are several popular implementations of the Unicode standard such as: 򐂰 UCS-2 - Universal Character Set coded in 2 octets. 򐂰 UCS-4 - Universal Character Set coded in 4 octets. This will become UTF-32. 򐂰 UTF-8 - UNICODE Transformation Format for 8 bit (ASCII safe UNICODE). Characters are encoded in 1 to 6 bytes. 򐂰 UTF-16 - UNICODE Transformation Format for 16 bits. The format is a superset of UCS-2 and contains an encoding form that allows more than 64 KB characters to be represented. DB2 V7 support for UNICODE provides the most popular implementations of UNICODE: UTF-8 and UTF-16. 򐂰 CHAR, VARCHAR, LONG VARCHAR and CLOB data for SBCS data is stored as ASCII (7 bit) CODE CCSID 367 򐂰 CHAR, VARCHAR, LONG VARCHAR and CLOB data for mixed data is stored as UTF-8 (UNICODE CCSID 1208) 򐂰 GRAPHIC, VARGRAPHIC, LONG VARGRAPHIC and DBCLOB data is stored as UTF-16 (UNICODE CCSID 1200) If you are working with character string data in UTF-8, you should be aware that ASCII characters are encoded into one byte lengths. However, non-ASCII characters, for example Japanese characters, are encoded into two or three byte lengths in a multiple-byte character code set (MBCS). Therefore, if you define an 'n' bytes length character column, you can store strings anywhere from 'n/3' to 'n' characters depending on the ratio of ASCII to non-ASCII character code elements. DB2 does not use the table SYSIBM.SYSSTRINGS for conversion to and from UNICODE CCSIDs. Instead DB2 uses OS/390 Conversion Services, a feature shipped with OS/390 Version 2 Release 8, to manage all the conversions to and from UNICODE CCSIDs. Setting up the new UNICODE support using OS/390 Conversion Services needs some planning and investigation before you enable it. Appendix B, “Unicode implementation” on page 147 provides an example of how to set it up.

3.2 Storing LOBs This section gives you an idea of how LOBs are stored in DB2 V7. We provide you with sample DDL needed for the creation of all LOB dependent objects, including a base table, a LOB table space, an auxiliary table, and an auxiliary index. We also discuss the way DB2 stores the data at page level and how buffer pools and data spaces are associated with a LOB table space.

3.2.1 How to store them in DB2 CLOBs, BLOBs, and DBCLOBs are the new data types provided by DB2 for large objects support. DB2 also provides new techniques to accommodate the storing of these possibly huge amounts of data within the DB2 structures. In fact a large object in DB2 can be compared to a more familiar string element, such as a CHAR or a VARCHAR column with the main difference being that these LOB columns can reach the size of 2 GB minus one byte (2,147,483,647 bytes). This requires different handling techniques.

16

Large Objects with DB2 for z/OS and OS/390

Generally, non-LOB columns are stored in what we refer to as a base table. LOBs belong to a base table and are related to it, but they are not stored in the same table with the other non-LOB columns, they are stored in a new type of table space, which is called a LOB table space. See Figure 3-3.

LOB Table Space Picture 1

Base Table Space Base Table Col 1

Col 2

Vala

Valb

Valc

Vald

Auxiliary Table

LOB Col 1

Picture 2

Figure 3-3 Association between base table and auxiliary table

Each LOB column in a base table requires its own LOB table space, so LOB columns are also separated from other LOB columns belonging to the same base table. It is important to understand that a page never contains values of two LOB columns. For a pictorial view of the different LOB columns and its associated LOB table spaces see Figure 3-4.

LO B Table Space 2

B ase Table Space

M ovie 1

B ase Table Col 1

C ol 2

Vala

Valb

Valc

Vald

LO B C ol 1

Auxiliary Table

LO B C ol 2

M ovie 2

LO B Table Space 1 Picture 1

Auxiliary Table

Picture 2

Figure 3-4 Base table with two LOB columns and the two associated LOB table spaces

Chapter 3. Creating LOBs

17

If the base table is partitioned, every LOB column in every partition has its own LOB table space. This means up to 254 LOB table spaces per LOB column if the base table has 254 partitions. Figure 3-5 gives you a better understanding of a multi-LOB column base table and the required LOB table spaces.

Partition 1 - Base Table Space

LOB Table Space 2

Base Table Col 1

Col 1

Vala

Valb

Valc

Vald

LOB Col 1

LOB Col 2

Movie 1

Auxiliary Table

Movie 2

LOB Table Space1 Picture 1

Auxiliary Table

Picture 2

Partition 2 - Base Table Space

LOB Table Space 4

Base Table Col 1

Col 2

Vale

Valf

Valg

Valh

LOB Col 1

LOB Col 2

Movie3

Auxiliary Table

Movie 4

LOB Table Space 3 Picture 3

Auxiliary Table

Picture 4

Figure 3-5 Partitioned base table containing two LOB columns

The table where the rows of a LOB column live, and which is embedded in the LOB table space, is called an auxiliary table (aux table). The LOB column itself is referred to as an auxiliary column, because it is not stored in the base table. The rows in the base table are associated to the LOB columns residing in the auxiliary table in the LOB table space using the ROWID. For further information about ROWIDs, see 4.6, “Details about ROWID” on page 68. In order to quickly locate the proper LOB value in the auxiliary table, an auxiliary index must be created on the auxiliary table. This index includes the ROWID. Additionally a LOB indicator is also stored in the base table. You can find more information about this indicator in “A few more details on the base table” on page 20. See Figure 3-6 for a summary picture of all the objects mentioned here and how they work together.

18

Large Objects with DB2 for z/OS and OS/390

Base Table Space Base Table Col 1

Col 2

LOB Col 1

Vala

Valb

ABCDEFG...

Valc

Vald

1234567...

LOB Col 1 Flags

... ...

LOB Table Space Row ID

LOB Value

ABCDEFG...

1234567...

Picture 1

Picture 2

Row ID Auxiliary Table Auxiliary Index

Figure 3-6 Association between base table, ROWID, auxiliary table, and LOB table space

An important reason for the separation of LOB columns is performance. The decision was made during the design phase of large objects support, based on experiences using middleware systems and LOBs. Assuming table space scans on the base table, LOB values do not have to be processed during these scans. Probably most of the scanning time would be spent in scanning LOB columns if they would reside in the same table as the non-LOB columns. DB2’s Data Manager (DM) only handles column-sizes up to 32 KB. For this purpose a new LOB Manager (LOBM) was introduced in DB2 V6. Based on the nature of LOBs, they can be larger than the biggest available page size (32 KB in DB2 V6 and V7) which is also valid for LOB table spaces. Therefore a single LOB value can span pages. For further information on storing LOBs, see 4.4.1, “Storage of LOBs” on page 60. Because each data set of a LOB table space may grow up to 64 GB, and there may be up to 254 data sets per table space, the maximum size of a non-partitioned LOB table space is 16,256 GB (16 TB) in total. As the number of partitions can grow up to 254 in the base table, there will be 254 single LOB table spaces, each holding up to 16 TB of data as a maximum size. This gives a grand total of 254 x 16 TB = 4,064 TB available for a single column of LOB data.

3.2.2 Creating a table containing LOBs In our example we assume that we need to set up a new table containing two non-LOB columns, a CLOB, and a BLOB column. The base table is stored in the base table space.

Chapter 3. Creating LOBs

19

The first step is creating the base table in a table space already existing in the same database where the LOB table spaces will be stored. There are no special suggestions for a base table space: it is just a normal table space. But we recommend that you have only one base table in a base table space. This will simplify your recovery procedures by dealing independently with each LOB table space. Note: The table space containing the base table has to be in the same database as every associated LOB table space. As an example, we create a base table containing information about a book, storing the book itself as a CLOB, and the image of the book cover as a BLOB. The base table is created with the DDL statement reported in Example 3-1. Example 3-1 DDL for a base table CREATE TABLE BOOK_BASE_TABLE ( BOOK_NO CHAR(10) , DESCRIPTION CHAR(32) , ROW_ID ROWID , BOOK_TEXT CLOB(500M) , BOOK_COVER BLOB(1M) IN LOBDB.BASETS;

NOT NOT NOT NOT NOT

NULL NULL NULL NULL NULL

WITH DEFAULT WITH DEFAULT GENERATED ALWAYS );

The only supported default value for a LOB column is NULL. For CLOBs you can also specify the parameters FOR SCBS, MIXED or BIT DATA. A CCSID EBCDIC or ASCII may also be specified for CLOBs. For BLOBs and DBCLOBs this is not supported, because BLOBs contain binary data and DBCLOBs have a graphic CCSID associated with them. Sizes of LOBs can be assigned in kilobytes (K), megabytes (M) or even gigabytes (G). For syntax on CREATE TABLE statements regarding LOBs, refer to DB2 UDB for OS/390 and z/OS Version 7 SQL Reference, SC26-9944.

A few more details on the base table The additional column containing the new data type ROWID simply contains unique values related to the auxiliary tables in the LOB table spaces. A column of data type ROWID has to be defined within your base table when you use LOBs in your base table. The data type ROWID is stored as a VARCHAR (17) column in the base table. It is implicitly one part of the unique index for each auxiliary table containing the LOB columns in the LOB table spaces, where the LOB columns are stored. Even if a base table contains more than one LOB column, only one ROWID column is needed. So, all LOB columns in one row are associated with the same ROWID value. Regarding this procedure, the ROWID column is a unique and permanent identifier for each row in the base table. A ROWID is a new data type and is not the same as a record identifier (RID), which is internally used by DB2 to reflect the position of a row in a table. But you can find the RID as a part of the externalized ROWID when you select the ROWID column. There are two ways of assigning a ROWID to a base table. 򐂰 The first one is specifying GENERATED ALWAYS in the DDL. Using this syntax, DB2 takes care of the ROWID column and users do not have to take any action regarding ROWIDs. So the ROWID column will not appear in the VALUES clause of an INSERT 20

Large Objects with DB2 for z/OS and OS/390

statement. The recommendation is to use GENERATED ALWAYS unless you want to copy data from other tables into your table. 򐂰 The second option consists on specifying GENERATED BY DEFAULT. Using this syntax, DB2 generates a value only if you do not specify any value in the VALUES clause of an INSERT statement. This is intended to be used only for copying data from other tables. Applications or users should never try to generate ROWID values, because DB2 performs validity checking while inserting a row in the specified table. When you plan to use GENERATED BY DEFAULT be aware that DB2 requires a unique index on the ROWID column on the base table. More information on ROWIDs is reported in 4.6, “Details about ROWID” on page 68. In the table definitions of the new data types CLOB (500 MB) and BLOB (1 MB) in Example 3-1 on page 20, DB2 will implicitly put two LOB indicators into the base table definition. Only indicator columns are stored in the base table in place of the actual LOB columns. A LOB indicator for a LOB column consists of six bytes, and it provides useful information about the stored LOB to DB2 when it accesses the column. The LOB indicators are stored like VARCHAR (4) columns. They are made up of 2-bytes flag and 2-bytes string containing the number of the currently stored version of the LOB. The flag bytes contain a NULL flag which has the information about a NULL, or even a NOT NULL value assigned to a specific LOB value. Another information retrieved from the flag bytes is the zero length flag which indicates that a LOB column does not contain any data. Invalid LOB values are also marked invalid using the flag bytes. By referring to this information, DB2 does not have to read the auxiliary table when any of these conditions is true. Figure 3-7 illustrates the catalog information for the created table.

Catalog description

Book_No

Description

ROW_ID

Book_Text

Book_Cover

CHAR (10)

CHAR (32)

ROWID (17)

CLOB (4)

BLOB (4)

Stored as VARCHAR (17)

Stored as VARCHAR (4)

Stored as VARCHAR (4)

Figure 3-7 Catalog description for table BOOK_BASE_TABLE

The remaining two bytes contain the current version of the LOB value. This is stored to detect mismatch situations between the base table and the auxiliary table. For further information on possible mismatches, see 6.2.8, “CHECK LOB” on page 104. Trying to access the base table now, by using SELECT, UPDATE, INSERT or DELETE, results in DB2 issuing SQLCODE -747. Access to the base table is not allowed, because the base table is marked as incomplete if at least one dependent object is not defined.

Chapter 3. Creating LOBs

21

The information when a table definition is incomplete is stored in column TABLESTATUS in SYSIBM.SYSTABLES introduced in DB2 V6. A value of ‘L’ indicates a missing auxiliary index or auxiliary table for a LOB column. When the content of the value is ‘R’, you have used the GENERATED BY DEFAULT clause for your ROWID column and have not yet created the required single column unique index on that particular column in your base table. Note: You cannot access the base table via SQL until all dependent objects (LOB table space, auxiliary table, and the auxiliary index) are defined. If the CURRENT RULES special register is set to STD, DB2 creates the dependent objects by itself during the processing of the base table DDL. The naming of the automatically created objects are determined by DB2’s default algorithms. We talk about CURRENT RULES in 3.3, “The CURRENT RULES special register” on page 31. You can use the REPORT utility to find out which names DB2 has assigned to your auxiliary objects, but you can also have a look at the catalog as usual. For more information about the REPORT utility, see 6.2.5, “REPORT TABLESPACESET” on page 99.

3.2.3 Creating the LOB table spaces After the base table is created, we define the LOB table space containing the LOB column. In our example we need two LOB table spaces, one for the CLOB column, the other one for the BLOB column. Both LOB table spaces can be created using the sample statement shown in Example 3-2. Example 3-2 DDL for a LOB table space CREATE LOB TABLESPACE IN LOBDB USING STOGROUP PRIQTY SECQTY LOG LOCKSIZE BUFFERPOOL DSSIZE GBPCACHE

BLOBATS1 BLOBSGTS 1000000 1000000 NO LOB BP32K 64G SYSTEM;

The keyword LOB tells DB2 to create the table space with the new format. You cannot store LOB values in any other than LOB table space (for instance the generic LARGE table space). Specifying free space parameter (FREESPACE and PCTFREE) has no influence with LOBs. The second LOB table space holding the CLOB column is created in a similar way. Every LOB column needs its own LOB table space. Partitioning of LOB table spaces is not allowed, but they are divided in pagesets in accordance with the partitioned base table definition. Compression is not supported for LOB table spaces. For a more detailed description of how the data is stored in a LOB table space, see 4.4.1, “Storage of LOBs” on page 60.

22

Large Objects with DB2 for z/OS and OS/390

Buffer pools and LOB table spaces If you want to store huge amounts of data in your LOBs, you may consider using separate storage groups and buffer pools from your other data in order to avoid interference resulting in a possible impact on performance. This is because a single LOB value is able to use up to 2 GB of your buffer pool if assigned to the same buffer pool as the non-LOB table spaces. If all LOBs are going to be read, the content of the buffer pool may mainly consist of LOBs.

Using LOG YES or LOG NO Logging LOB data can create a bottleneck in your DB2 subsystem, because each logged LOB can grow up to 1 GB in size and the amount of data to be logged grows according to the actual size of the manipulated LOBs. DB2 does not allow you to turn on logging for LOBs bigger than 1 GB. Specifying LOG YES for a LOB table space tells DB2 to log almost all data manipulations on LOB columns stored in the associated LOB table space, except delete operations. When you delete a LOB value, no LOB data is written to the log, only LOB system pages are written to the log data sets, because a LOB delete internally is translated into only de-allocation of the pages where the LOB is stored. The de-allocation of a LOB is flagged in the space map pages for all pages containing the LOB information, including the LOB map pages. For more information about the structure of a LOB table space, refer to 4.4.1, “Storage of LOBs” on page 60. When you insert a LOB, DB2 writes the entire LOB value to the log. The entire LOB is also written to the log even when you update a LOB value, because updates consist of one singleton delete (where DB2 does not write data in the log) and one insert into the auxiliary table. Depending on the size of your LOBs and the frequency you regularly insert or update them, the data to be logged may grow rapidly. When you plan to use LOG YES, make sure that writing the redo logs will not become a critical factor! To prevent your system from the probable bottleneck caused by logging large LOB data, DB2 allows you to turn off logging for large objects. LOG NO simply tells your DB2 subsystem to suppress writing redo logs for every LOB column in your LOB table space. The force at commit protocol ensures that LOB values persist after a unit of work is committed, because LOG NO LOB values are written at COMMIT. LOB data associated with a LOB table space defined with LOG YES is written to disk as usual for other data when buffer pool thresholds are reached. If you specify LOG YES for a LOB table space where the LOB size exceeds 1 GB, DB2 issues SQLCODE -355 when creating the auxiliary table. This indicates that a LOB column is too large to be logged. Keep in mind that LOG YES is the default value when you create your LOB table space without specifying the LOG clause. Note: LOBs larger than 1 GB cannot be logged. Changes to the base table are not affected by specifying the LOG clause on the LOB table space. You can find a detailed description of LOG impact between LOG YES and LOG NO in 7.3.1, “Logging with LOBs” on page 128. For information about the logging impact when running the Load utility, see 6.2.1, “LOAD” on page 94.

Chapter 3. Creating LOBs

23

DB2 rollback without undo log records As new pages are allocated while inserting a LOB, they are simply de-allocated if DB2 does a rollback. If a LOB is deleted, pages are also simply re-allocated as available in the LOB table space. Since updating a LOB means deleting a LOB (de-allocating pages) and inserting a new one (allocating new pages), at rollback time already allocated pages are de-allocated and previously de-allocated pages are re-allocated again. This mechanism is known as Shadow Copy Recovery, see “Shadow Copy Recovery” on page 67. Because of allocation and de-allocation of data pages, no UNDO logs are written for LOBs, regardless what LOG parameter you use. Header pages, space map pages, and the new LOB map pages (see “LOB table space organization” on page 61) are logged even if LOG NO is specified. They are backed out using the same procedure used for other types of table spaces. The same rule applies if an application terminates abnormally after changing the database. Even in this case, those changes made in the current unit of work are backed out along with all other changes, too. If LOG NO is specified, DB2 does not log any changes applied to any LOBs in the associated LOB table space. Therefore recovery of LOB table spaces can be more difficult than recovering any other table space. Let us assume the scenario depicted in Figure 3-8. LOB Table Space

Base Table Space Base Table Col 1

Col 2

Vala

Valb

Valc

Vald

Picture 1

Auxiliary Table

LOB Col 1

Picture 2

Figure 3-8 Applying changes to a LOB table space created with LOG NO

A full Image Copy is taken of a base table space and the associated LOB table spaces. After completion of the full Image Copies, one LOB is inserted, another one is deleted and one more is updated. Assume now that you have to recover the base table space and the LOB table space after all updates are done because of a system failure. For the base table DB2 can apply all necessary changes from the log. Recovering the LOB table space for DB2 is more difficult with a specified LOG NO option. After using the full image copy, DB2 notices that a new LOB was inserted without writing any logs about its value. So the new LOB is marked invalid in the auxiliary table. The next important log point is deleting LOB number two. DB2 knows from the logged system pages what has happened and marks the pages formerly used by LOB number two as available. The next point on the log brings us to a LOB update. Since UPDATE consists of DELETE and INSERT, the pages are deallocated in the LOB table space and for not having a log to apply for the new LOB value, it is also marked as invalid in the LOB table space. In our case the LOB table space is set in a new AUX WARNING (AUXW) state. Trying to access an invalid LOB value results in SQLCODE -904. For more information on recovery scenarios, see 6.2.13, “RECOVERY” on page 109.

24

Large Objects with DB2 for z/OS and OS/390

Attention: Choose Image Copies with SHRLEVEL REFERENCE when using LOG NO, and define carefully when to take them based on your application needs. Image Copies using SHRLEVEL CHANGE allow updates to occur, and with a LOG NO table space, you have unusable copies. With DB2 V6 and V7, the LOG NO clause can be provided for LOB table spaces, but no other types of table spaces support this option.

Locking for LOBs There are at least two different types of lock sizes you can choose between. Valid parameters are LOCKSIZE LOB and LOCKSIZE TABLESPACE. Choosing LOCKSIZE ANY implies the use of LOCKSIZE LOB for DB2. Two new types of locks were introduced for LOBs in DB2 Version 6: the S-LOB and the X-LOB locks. They are very similar to the common S- and X-Locks. There are no U-LOB locks because of a different update mechanism taking place for LOBs. See “Using LOG YES or LOG NO” on page 23 for a description of updating techniques for LOBs. There is no support for Uncommitted Read (UR) on LOBs. Assuming LOCKSIZE LOB, selecting a LOB acquires an S-LOB lock on the accessed LOB, even if you use ISOLATION (UR) in a package or WITH UR in your SQL statement. This is because of the new table space format where physical correctness has to be guaranteed while retrieving the LOB value, which may be spanned over many pages. Acquiring S-LOB locks prevents the application from retrieving only partial LOB data. Deleting a LOB also requests an S-LOB lock on a specific LOB. But how does this work with other transactions selecting the same LOB? A transaction can delete a LOB that another transaction is reading at the same time, but the space is not reused until all readers of the LOB have committed their work. So S-LOB locks do not prevent a LOB from being deleted. Inserting a LOB acquires an X-LOB lock on the new LOB, the lock is released at COMMIT as usual. If a single LOB is locked by an X-LOB lock, no readers can access them before it is committed. As every LOB lock is like a row lock, the number of acquired locks can increase dramatically, especially when mass-updates occur on LOBs, or many LOBs are inserted via subselect. You can find more details about locking for LOBs in 4.5, “Locking” on page 62.

Buffer pool and page size considerations You can assign 4 KB, 8 KB, 16 KB or 32 KB buffer pools to your LOB table space. But which one to use depends on several considerations. Choosing a page size is the common trade-off between minimizing the number of getpages, which simply maximizes performance, and not wasting space. Because one data page in a LOB table space never stores data of more than one LOB value, the space not used by the LOB value in the last page remains unused. To estimate a good average size of a LOB value, use the formula: Size of a LOB = (Average length of all LOBs) * 1.1

Use the rule-of-thumb shown in Table 3-1 to choose a page size that minimizes the number of getpages.

Chapter 3. Creating LOBs

25

Table 3-1 Choosing a LOB page size that minimizes getpages Average LOB size (ALS)

Recommended page size for LOB table space

ALS ≤ 4 KB

4 KB

4 KB < ALS ≤ 8 KB

8 KB

8 KB < ALS ≤ 16 KB

16 KB

16 KB < ALS

32 KB

Using this technique, a LOB value of 22 KB means 10 KB of unused space, but only one getpage request for DB2. Therefore you have to analyze your data to determine what is best for you and your LOBs. Normally, you have 32 KB data pages assigned to your LOB table space, because you should use LOBs only for real large objects. If your LOB data is all the same size, it may be easier to choose a page size that makes more efficient use of data pages without impacting performance. For LOBs that have all the same size, consider Table 3-2, which maximizes space savings without sacrificing performance. Table 3-2 Choosing a LOB page size for LOBs all of the same size LOB size (LS)

Recommended page size for LOB table space

LS ≤ 4 KB

4 KB

4 KB < LS ≤ 8 KB

8 KB

8 KB < LS ≤ 12 KB

4 KB

12 KB < LS ≤ 16 KB

16 KB

16 KB < LS ≤ 24 KB

8 KB

24 KB < LS ≤ 32 KB

32 KB

32 KB < LS ≤ 48 KB

16 KB

48 KB < LS

32 KB

Also see buffer pool considerations and buffer pool thresholds (DWQT and VDWQT) in 7.2, “Buffer pools and group buffer pools” on page 125 for more detailed information on this topic. If you have decided the page size you want to use, DB2 allocates at least n KB of space even if you tell DB2 to allocate less than the minimum amount of space for that page size. Table 3-3 provides information of the minimum space requirements for primary (PRIQTY) and secondary (SECQTY) quantity and how they may change from your specification. Table 3-3 Primary and secondary quantity with LOBs PRIQTY and SECQTY

26

Specification in KB

Page size

Resulting allocation in KB

< 200

4 KB

200

< 400

8 KB

400

< 800

16 KB

800

< 1,600

32 KB

1600

> 4,194,304

Any

4,194,304

Large Objects with DB2 for z/OS and OS/390

Need to specify DSSIZE for LOB table spaces DSSIZE represents the data set size value passed to DB2 to indicate the maximum allowed size. Specifying DSSIZE 64 GB allows one data set of size up to 64 GB. Table 3-4 summarizes the partition and partitioned table space sizes for the current DB2 versions. Table 3-4 Summary of data set, partition, and partitioned table space sizes DB2 Version

Number of partitions

Maximum size each

Total maximum size

V5*

254

4 GB

1,016 GB

V6 and V7**

254

64 GB

16,256 GB

Note: * Requires LARGE parameter in CREATE TABLESPACE Note: ** Requires DSSIZE and DFSMS/MVS 1.5 and SMS-managed table space

Up to DB2 V4 the total maximum size of a partitioned table space was 64 GB. Starting with DB2 V5, with the introduction of the LARGE parameter at creation time, partitioned table spaces may have a total size of 1,016 GB, corresponding to up to 254 partitions each with a data set size of 4 GB. DB2 V6 introduced the DSSIZE parameter to replace LARGE, and in combination with the extended addressability function of SMS managed VSAM data sets, can extend the data set size to 64 GB. Since each LOB table space may consist of 254 data sets, you can reach the maximum amount of 16,256 GB (approximately 16 TB) for a single LOB column in one LOB table space. Considering a partitioned base table with 254 partitions, these 16,256 GB can occur up to 254 times (one LOB table space for each partition of the base table), so you can have 4,129,024 GB or 4,129 TB for one single LOB column. The default value passed to DB2, if DSSIZE is not specified, is 4 GB per data set. This means a maximum value of 1,016 GB (approximately 1 TB) for a non-partitioned base table and 258,064 GB (approximately 258 TB) for a partitioned base table consisting of 254 partitions. The maximum amount of space for PRIQTY and SECQTY you can specify is at least 4,194,304 KB. So make sure that your specified PRIQTY and n times SECQTY, if allocated, is about 64 GB. The value of n depends on the version of DFSMS running in your system. Starting with DFSMS Version 1 Release 4, 254 extends are supported. Extended addressability for VSAM data sets, leading to a maximum file size of 64 GB, was introduced in DFSMS Version 1 Release 5. To benefit from the maximum size provided for LOBs, DFSMS Version 1 Release 5 is required. Note: If DSSIZE is greater than 4 GB, make sure that this data set belongs to a DFSMS class that is defined with the extended addressability attribute, and also that the automatic class selection routine associates this data set with this data class. Otherwise DB2 is not able to allocate the requested space, and it will issue SQLCODE -904.

GBPCACHE parameter A new GBPCACHE SYSTEM parameter was added for LOBs to prevent LOB data from flooding your group buffer pool. Specifying GBPCACHE SYSTEM caches only the LOB control (system) information in the group buffer pool. In a data sharing environment GBPCACHE SYSTEM is recommended for large objects. See 7.2, “Buffer pools and group buffer pools” on page 125 for more information.

Chapter 3. Creating LOBs

27

3.2.4 Creating the auxiliary tables The third step is creating an auxiliary table, holding the data of a LOB column. There can only be one auxiliary table per LOB table space. The DDL for creating an auxiliary table in a LOB table space is shown in Example 3-3. Example 3-3 DDL for an auxiliary table CREATE AUXILIARY IN STORES COLUMN

TABLE BLOB_AUX_TABLE_1 LOBDB.BLOBATS1 BOOK_BASE_TABLE BOOK_COVER;

This statement creates the auxiliary table BLOB_AUX_TABLE_1 in the LOB table space created in 3.2.3, “Creating the LOB table spaces” on page 22. The other auxiliary table storing the CLOB column is created in a similar way. There is no need to specify column names or column types for auxiliary tables. Using the STORES clause, new with Version 6, DB2 is told what column of which base table you want to store in the created auxiliary table. The associated table consists of only one column, the LOB column. Note: A LOB table space and its associated base table space must be stored in the same database. Otherwise SQLCODE -764 will be issued. If the referenced base table is partitioned, there must be a LOB table space and an auxiliary table for each LOB column in each partition of the base table. In this case, Example 3-4 shows sample DDL for creating the auxiliary table. Example 3-4 DDL for an auxiliary table containing data of one base table partition CREATE AUXILIARY IN STORES COLUMN PART

TABLE BLOB_AUX_TABLE_1 LOBDB.BLOBATS1 BOOK_BASE_TABLE BOOK_COVER n;

The PART clause indicates which partition’s BOOK_COVER column you want to store in this auxiliary table, where n is the number of the partition. FIELDPROCs, EDITPROCs, VALIDPROCs, and check constraints cannot be defined on LOB columns.

How DB2 locates the assigned LOB values We have only defined one column for the auxiliary table, but DB2 requires two more columns for the table: a column containing the ROWIDs of the base table and one column storing the current version number of the LOB. We report the information stored in the catalog for the three columns of the auxiliary table in Figure 3-9. Using this redundant data, DB2 is able to quickly locate rows in the auxiliary table. In order to do so, we need to define another and last new object: the auxiliary index.

28

Large Objects with DB2 for z/OS and OS/390

C a ta lo g d e s c rip tio n A U X ID

A U X VA L U E

AUXVER

VA R C H A R (1 7 )

B L O B (4 )

S M A L L IN T (2 )

L E N G T H 2 = 1 ,04 8,57 6

Figure 3-9 SYSIBM.SYSCOLUMNS contents for TBNAME= BOOK_AUX_TABLE

3.2.5 The auxiliary index The last step when creating LOBs is to define an auxiliary index for the LOB table space, as shown in Example 3-5. An auxiliary table must have exactly one index, and there cannot be any additional column in the index. Example 3-5 DDL for an auxiliary Index CREATE UNIQUE INDEX LOBDB.BLOBAIX1 ON BLOB_AUX_TABLE_1 USING STOGROUP BLOBSGIX PRIQTY 100 SECQTY 10;

As you can see, no index columns are defined within this index, because DB2 automatically creates the key definition. The index definition consists of a two key value: The 17-byte system generated ROWID stored as VARCHAR, and a 2-byte version of the LOB stored as SMALLINT, for 21 bytes in total (including the 2-byte length field for VARCHAR columns). No index keys can be defined. If you even try to specify a key column, DB2 will issue SQLCODE -767, missing or invalid column specification for an index. Figure 3-10 shows the key columns of an auxiliary index.

AUXID VARCHAR (17) AUXVER SMALLINT

Auxiliary Index Figure 3-10 Index keys for an auxiliary index

Chapter 3. Creating LOBs

29

Note: You cannot define an index on a LOB column. DB2 V7 uses the index on the auxiliary table to locate a LOB value for a particular row containing a LOB within the base table. An index defined on an auxiliary table will automatically be defined as a unique index, fed by a ROWID from the base table. The buffer pool you may want to assign to this index does not need any special considerations, because the index only contains two relatively small columns.

Displaying LOB objects For our base table containing one BLOB column and one CLOB column, the display database will look like Example 3-6. Example 3-6 Displaying a database for LOBs DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = LOBDB STATUS = RW DBD LENGTH = 4028 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----BASETS TS RW BLOBATS1 LS RW CLOBATS1 LS RW BLOBAIX1 IX RW CLOBAIX1 IX RW ******* DISPLAY OF DATABASE LOBDB ENDED **********************

3.2.6 Adding a LOB column to an existing table Another possible situation is the need for adding a particular LOB column to an already existing table; this table becomes the base table for your LOB column(s). Before a LOB column can be added, we have to be sure that a column of data type ROWID is already in the table. If it is not, we have to add it using the statement reported in Example 3-7. Example 3-7 Adding a ROWID column ALTER TABLE NEW_LOB_TABLE ADD ROW_ID ROWID

NOT NULL GENERATED ALWAYS;

A table can only have one ROWID column and you cannot add a ROWID column to a created temporary table. We recommend that ROWIDs are always generated by using DB2’s mechanism. Specifying GENERATED BY DEFAULT generates a ROWID only if no value for the ROWID column is provided while inserting into the table. If a value for a ROWID column is provided, DB2 takes it, and inserts the value into the base table. By providing a value for a ROWID column, for instance when moving data across two DB2 subsystems, it is unlikely, but it may happen, that

30

Large Objects with DB2 for z/OS and OS/390

DB2 generates a ROWID which is already stored in the table. Because the GENERATED BY DEFAULT clause always requires a unique index on that column, an insert with a DB2 generated ROWID value can result in SQLCODE -803. In this case it is your responsibility to resolve the duplication. Once a column of data type ROWID is added, you can proceed with creating the LOB column(s) using the alter table statement shown in Example 3-8. Example 3-8 Adding a LOB column ALTER TABLE NEW_LOB_TABLE ADD PICTURE BLOB(1M)

NOT NULL WITH DEFAULT;

Adding a LOB column is not allowed for created temporary tables. The same is true when adding ROWID columns. Instead of specifying BLOB (1M), like in this example, you can use the ALTER to define any other possible LOB column type to DB2. After you have added to the base table the designated LOB columns, you should continue with the same actions shown earlier starting with 3.2.4, “Creating the auxiliary tables” on page 28. Remember that only one ROWID column is needed, even if you want to add more than one LOB column to the base table. Note: When you add one or more LOB columns to the base table, the table is marked incomplete until all dependent objects are created.

3.3 The CURRENT RULES special register The content of the CURRENT RULES special register has impact on your DB2 subsystem. The default value for this parameter is set via the SQLRULES parameter in your BIND PLAN statements. When you do not specify SQLRULES in your BIND PLAN statement, the default is DB2. Possible values for the register are ‘STD’ (standard) and ‘DB2’. We describe the effects of this parameter on your environment so that you can choose the desired value for your needs. The CURRENT RULES register influences the following activities within your system: 򐂰 CREATE and DROP the auxiliary objects as LOB table space, auxiliary table, and auxiliary index 򐂰 Application development for LOBs using CURSORs

3.3.1 Impact on CREATE and DROP LOBs In this section we examine the impact at CREATE and DROP time depending on the CURRENT RULES setting. You have to decide if you want DB2 to take care of your naming conventions, or if you prefer naming the objects and specifying the parameters by yourself. We recommend using CURRENT RULES DB2, since this enables you to set up the LOB environment more appropriate to your needs.

CREATE necessary LOB objects If you specify CURRENT RULES DB2, you have to create all required objects by yourself. You can then specify all the parameters mentioned in 3.2.3, “Creating the LOB table spaces” on page 22, and in the following sections.

Chapter 3. Creating LOBs

31

If you specify STD, DB2 creates all the auxiliary objects you need for the base table at the time it processes the CREATE statement for your base table. So DB2 creates the LOB table space, the auxiliary table and the auxiliary index for you. If the auxiliary objects defined in A.1, “DDL for a LOB environment” on page 138 had been created using CURRENT RULES STD, they would have looked as in Table 3-5 for the LOB table space, and in Table 3-6 for the auxiliary index. Table 3-5 LOB table space created using CURRENT RULES STD TSNAME

BUFFERPOOL

LOCKSIZE

LOG

CLOSE

L1LX$XO8

BP32K

ANY

YES

YES

DSSIZE

PQTY/SQTY

FREEPAGE

PCTFREE

GBPCACHE

4G

12,800 / 12,800

0

5

NONE

The name of the LOB table space is an 8-character string every time DB2 creates it for you. The first character is an L, followed by seven random characters. The auxiliary table is created as PAOLO.CLOB_DOCUM1LXA0LID. The table name of the auxiliary table is an 18-character string. The first five characters of the name are the first five characters of the name of the base table. The second five characters are the first five characters of the name of the LOB column. The last eight characters are randomly generated. If a base table name or a LOB column name is less than five characters, DB2 adds underscore characters to the name to pad it to a length of five characters. Table 3-6 Auxiliary index created using CURRENT RULES STD IXNAME

IXSPACE

BUFFERPOOL

CLOSE

ICLOB_DOCUM1LXA RNX

ICLOBRDO

BP2

YES

PIECESIZE

PQTY/SQTY

FREEPAGE

PCTFREE

4,194,304

12 / 12

0

10

The name of the auxiliary index is also 18 characters long. The first character of the name is an ‘I’. The next ten characters are the first ten characters of the name of the auxiliary table. The last seven characters are generated randomly. The index has the COPY NO attribute, therefore full image copies and recover utilities are not allowed. You can also modify an existing table for holding a LOB column using the ADD column option in the ALTER TABLE statement. First you ADD the ROWID column to your designated base table, then, at the time you ADD the appropriate LOB column with the ADD LOB column statement, DB2 also creates all the needed auxiliary objects, if CURRENT RULES STD is chosen. If CURRENT RULES DB2 is chosen, you have to create the objects on your own.

DROP base object Whenever you drop a base table or a base table space, the associated objects are dropped depending on the way they were created. If you have created the auxiliary objects using CURRENT RULES STD, the LOB table space is implicitly dropped when you drop the base table or the base table space. The auxiliary table and the auxiliary index are dropped as well. If you have created the auxiliary objects by yourself, only the auxiliary table and the auxiliary index are dropped when you either drop the base table space or the base table. The LOB table space remains in your system.

32

Large Objects with DB2 for z/OS and OS/390

3.3.2 Impact on cursors fetching LOB values In general, an application program declares a cursor and fetches the cursor in a specific section of an application program. The type of variables you fetch the cursor into cannot vary, even if you fetch the same cursor in two different sections of your application program for whatever reasons. Fetching a LOB column now gives you the flexibility to fetch into a large object host variable or into a large object locator variable. DB2 allows you to make a decision of the kind of variable you want to fetch the LOB value into, depending on the current value of special register CURRENT RULES at the time the cursor is opened.

Using CURRENT RULES DB2 򐂰 After the cursor is opened, if the first FETCH executed uses a LOB locator to access a LOB column, no subsequent FETCH for that cursor can fetch that LOB column into a LOB host variable. 򐂰 After the cursor is opened, if the first FETCH executed uses a LOB host variable to access a LOB column, no subsequent FETCH for that cursor can fetch that LOB column into a LOB locator.

Using CURRENT RULES STD 򐂰 After the cursor is opened, and after the first FETCH executed, regardless of where the LOB value is fetched into (LOB locator or LOB host variable), a subsequent FETCH for that cursor can FETCH the LOB column into either a LOB locator or a LOB host variable. Although a CURRENT RULES special register set to STD gives you more flexibility to let your application program switch between both methods, you can get better performance if you use a value of DB2 in distributed environments. When you use the STD option, the server has to send and receive network messages for each FETCH to indicate whether the data being transferred is a LOB locator or a LOB value. With the DB2 option, the server knows the size of the LOB data after the first FETCH, so an extra message about the LOB data size is unnecessary. The server can send multiple blocks of data to the requester at one time, which reduces the total time for data transfer. Using the STD option, DB2 cannot make any assumptions about what the requesting application may want on the next fetch.

3.4 Feeding a LOB column Creating the environment for storing LOB values is one thing, providing the values is another. There are at least four different possible locations where the large objects are originating before they are inserted into a LOB column. The choice on how to bring them up to your host environment depends on their location and their size. Even if you have lots of large files already stored in your middleware, you can also consider uploading them via FTP to your host, and then access them with applications running on the mainframe. Applications running in a distributed environment can use the same technique as OS/390 applications for inserting LOBs, using distributed relational database architecture (DRDA) connections (for instance via DB2 Connect), open database connection (ODBC) calls, or Java using SQLJ or JDBC. Table 3-7 summarizes the alternatives on where LOBs can come from and how to feed them into your LOB columns. Table 3-7 Where large objects come from and how LOB size

Where is the data?

Method to feed LOBs

≤ 32 KB

Client

Cross-Loader, Application

Chapter 3. Creating LOBs

33

LOB size

Where is the data?

Method to feed LOBs

≤ 32 KB

Host

LOAD, Application

> 32 KB

Client

Application, DB2 Extenders

> 32 KB

Host

Application

3.4.1 LOAD with LOB columns smaller than 32 KB This is probably the easiest way to bring your LOBs into DB2. Objects smaller than 32 KB are not really large, they may also be called small large objects, or SLOBs. For this kind of objects, consider also the alternative of storing them in a VARCHAR column, because LOB processing is more effective for large columns than for small columns. See Figure 7-7 on page 130 and Figure 7-8 on page 132 for a performance comparison between a VARCHAR column and an appropriate LOB column. We also provide trace information on accessing a 32 KB column defined as VARCHAR and as a LOB column in 7.3.4, “Comparing SQL accounting profiles” on page 131. If you are going to use the LOAD utility to load a base table, consider that the maximum row size of a row being loaded cannot exceed 32 KB. Every LOB data type (BLOB, CLOB and DBCLOB) in the input data set has to be preceded by its length in a 4-byte binary field. The length field does not include itself, only the length of the LOB data to be loaded. The length field must start at the column given as start-position in the LOAD statement. For CLOB columns you can also specify MIXED when it contains mixed SBCS and DBCS data. If you specify MIXED, then any required CCSID conversions use the mixed CCSID for the input data, otherwise any conversions will use the SBCS CCSID for the input data. For further information regarding data conversions, see 3.1, “Data conversion” on page 14. The LOAD statement may look like the one reported in Example 3-9 for loading a base table containing a simple CLOB column. Example 3-9 LOAD a base table containing a LOB columns LOAD DATA INDDN (SYSREC) RESUME YES LOG NO INTO TABLE PAOLO.CLOB_BASE_TABLE ( BOOK_NO POSITION (01:10) CHAR (10) ,DESCRIPTION POSITION (11:32) CHAR (32) ,BOOK_TEXT POSITION (43) CLOB )

If your base table is partitioned, you need to add the PART n clause to the LOAD statement. For more information on the LOG clause, see 6.2.1, “LOAD” on page 94 and specifically Table 6-1 on page 95. Since the maximum record length of a data set is 32,760, this is not a very likely way to feed your LOBs in the database, especially if there is more than one LOB column in your base table.

3.4.2 Inserting LOBs via host application The LOAD utility cannot feed our LOB columns with values greater than 32 KB. To handle this situation, and assuming that the LOB data is already available in your OS/390 environment, you have to write an application program which performs the inserts of your LOBs.

34

Large Objects with DB2 for z/OS and OS/390

You have to distinguish between two situations: 򐂰 Inserting LOBs using a host variable large enough to hold the entire LOB value 򐂰 Inserting LOBs using a host variable not large enough to hold the entire LOB value, and introducing the use of locators

Inserting LOBs using a host variable Inserting a LOB already available at the host is probably the second way to feed your LOBs. There are two main techniques on how to move a LOB value to your host variable. The first one requires a data set containing the entire LOB value in one row. The only thing the application has to do is read a file, move the value to the assigned host variable, provide the correct value to the length field, and execute the SQL INSERT statement. The second technique consists of having a data set where a certain number of rows have to be tied together to build the entire LOB. This may be a more common method because of the maximum MVS allowed record length of 32,760 bytes, which is really a small LOB. The rows can be tied together in one variable as shown in the pseudo-code in Example 3-10. Example 3-10 Tieing rows together in one variable MOVE 0 PERFORM UNTIL EOF-INPUT READ FILE INTO INPUTRECORD IF NOT EOF-INPUT MOVE LENGTH OF INPUTRECORD MOVE INPUTRECORD ADD LENGTH END-IF END-PERFORM

TO HV-LOB-LENGTH

TO LENGTH TO HV-LOB-DATA (HV-LOB-LENGTH + 1:LENGTH) TO HV-LOB-LENGTH

The definition of a host variable, large enough to contain an entire LOB value, is reported in Example 3-11. You need to include it in your WORKING-STORAGE SECTION of your application program if you try to insert a 1 MB CLOB, as in our example. Example 3-11 host variable declaration for a LOB column 01 HV-LOB

USAGE IS SQL TYPE IS CLOB (1M).

Using this syntax, DB2 generates the definition reported in Example 3-12 for your application program. Example 3-12 What the DB2 precompiler makes of your host variable declaration 01 HV-LOB. 02 HV-LOB-LENGTH PIC S9(9) COMP. 02 HV-LOB-DATA. 49 FILLER PIC X(32767). [repeated 31 times] 49 FILLER PIC X(32).

The last filler is used to match exactly the requested host variable size of 1 MB as declared for the CLOB. Be aware that your application only defines host variables in a size you are allowed to acquire, regarding your JCL and your system. You must acquire buffers large enough to store your LOBs. This may be difficult for very large LOBs. For an example, IBM COBOL for OS/390 and VM 2.2.1 supports up to 16,777,215 bytes (16 MB minus one byte) for one item in the WORKING-STORAGE SECTION or in the LINKAGE-SECTION. Because

Chapter 3. Creating LOBs

35

of this limitation, for LOBs bigger than 16 MB minus 1 byte, you have to use another method for feeding your LOBs. See “Inserting LOBs using locators” on page 36” for details. The application has to ensure the delivery of a correct value in the length field of your LOB host variable. Once the host variable contains the entire LOB, you can simply issue an SQL INSERT on the base table to pass your LOB to DB2. You do not need to worry about storing LOBs in the auxiliary table — this is DB2’s job. From the application programmer’s point of view there is only one table containing all the columns being used for LOB processing. The statement shown in Example 3-13 is a possible way for inserting your LOB data. Example 3-13 Inserting a single LOB value using one host variable EXEC SQL INSERT INTO BASE_TABLE (COL1, COL2, LOB) VALUES (:HV-COL1 ,:HV-COL2 ,:HV-LOB) END-EXEC

It is recommended that an application commits after completing the unit of work while inserting a LOB, because COMMIT releases X-LOB locks taken during insert and frees allocated storage in the data space used by the LOB. A sample program showing how to insert a LOB using one host variable is included in the SAMPLE1.TXT file described in Appendix D, “Additional material” on page 155. You can find more information on inserting LOB values and DB2 data spaces in “INSERT” on page 122. This method is one of the fastest to feed your LOBs, because SQL does not have to be invoked many times. The only restriction is that inserting LOBs bigger than the maximum supported variable size of your programming language is not possible using this method. If you want to insert a LOB larger than the maximum supported variable size of your application language, please refer to the section “Inserting LOBs using locators” on page 36.

Inserting LOBs using locators If you are not able to acquire a buffer large enough to hold your LOB data, you have to move it in pieces into a DB2 LOB column. This technique is supported by using LOB locators. We show two examples of pseudo code using locators. In the first one we append values in single chain passing through an intermediate locator till we are ready to insert the full LOB, in the second one we use instead multiple chains of locators.

Single locator chain For this case see the pseudo-coding reported in Example 3-14 for a suggestion on inserting large LOBs. Example 3-14 Pseudo-code inserting LOBs > 32 KB with one locator chain Definitions: HV-LOB LOB-LOCATOR-1 LOB-LOCATOR-2

USAGE IS SQL TYPE IS CLOB (1M) USAGE IS SQL TYPE IS CLOB-LOCATOR USAGE IS SQL TYPE IS CLOB-LOCATOR

Pseudo-Code: MOVE 0 TO HV-LOB-LENGTH EXEC SQL SET :LOB-LOCATOR-1 = ‘’ END-EXEC

36

Large Objects with DB2 for z/OS and OS/390

PERFORM UNTIL EOF-INPUT READ FILE INTO INPUTRECORD IF NOT EOF-INPUT PERFORM BUILD-HOST-VARIABLE END-IF END-DO PERFORM FINAL-INSERT :BUILD-HOST-VARIABLE MOVE LENGTH OF INPUTRECORD MOVE INPUTRECORD ADD LENGTH

TO LENGTH TO HV-LOB-DATA (HV-LOB-LENGTH + 1:LENGTH) TO HV-LOB-LENGTH

IF HV-LOB-LENGTH > 1000000 THEN PERFORM APPEND-LOCATOR MOVE 0 MOVE SPACE END-IF

TO HV-LOB-LENGTH TO HV-LOB-DATA

:APPEND-LOCATOR EXEC SQL SET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-1, :HV-LOB) END-EXEC EXEC SQL FREE LOCATOR :LOB-LOCATOR-1 END-EXEC MOVE LOB-LOCATOR-2

TO LOB-LOCATOR-1

:FINAL-INSERT IF HV-LOB-LENGTH > 0 THEN PERFORM APPEND-LOCATOR END-IF EXEC SQL INSERT INTO BASE_TABLE (KEYCOL1, COL2, LOB) VALUES (:HV-KEYCOL1, :HV-COL2, :LOB-LOCATOR-1) END-EXEC EXEC SQL FREE LOCATOR :LOB-LOCATOR-1 END-EXEC

At the top of Example 3-14 you can find the declaration of a CLOB host variable. In this case the size is 1 MB because we assume 1 MB as the maximum usable host variable size in our environment. The associated locators are also defined at the beginning of the pseudo-code. The first initialization of LOB-LOCATOR-1 is done by its first reference in the CONCAT statement. Then the application reads the input file in a loop and it builds a host variable using the rows from the input file until an end-of-file condition occurs. After reading one input record, it is appended to our 1 MB CLOB host variable. After the host variable is nearly filled up (CURRENT-SIZE > 1000000, the correct value depends on the possible length of your

Chapter 3. Creating LOBs

37

input records), the host variable is appended to LOB-LOCATOR-1 using the CONCAT function of DB2 to the already existing LOB value, where LOB-LOCATOR-1 is referring to, and set to LOB-LOCATOR-2. The locator is not assigned to a specific table yet, only the reference to a value inside of DB2 is created. Using this technique, LOB materialization takes place outside of the application in the data space created dynamically by DB2 for building the entire LOB. If the host variable is filled with some data even after an end-of-file condition was detected, the content of the host variable is applied one last time to the locator. After all records are appended to your LOB locator (ensure that the correct record length is passed to DB2), the application finally inserts the LOB, using the locator for providing the needed host variable for the LOB value. Attention should be paid to the FREE LOCATOR statement. If you do not FREE the used locators, DB2 will tend to keep them around in buffers allocated out of the DBM1 address space where the locators reside, and this may cause problems in already virtual storage constrained environments. The intent of the FREE LOCATOR statement is to release the allocated virtual storage space used by the locator itself as well as the virtual storage buffers in the data space containing the data referenced by the locator. Unless the LOCATOR has been defined with HOLD, an SQLCOMMIT will also free the locator as well as the related and allocated entry in LOB data space. If you do not free the locators after using them, the storage will remain allocated until COMMIT. If you are trying to insert really large LOBs, your application could fail for having reached the limit of virtual storage set for the allocated data space, or the DBM1 address space could reach its limit because of heavy usage of storage caused by the storage structures associated with active locators. Make sure to issue the FREE LOCATOR statement for each append within the concatenation loop as well, this decreases an internal counter used by DB2 to control the structure built to reference the first locator. This minimizes the risk of causing virtual storage problems when inserting many LOBs without issuing a COMMIT between your LOB insert statements. We recommend for you to COMMIT after each single LOB is inserted by your application. The MOVE simply moves the value of our second locator to the first locator, which makes the first locator available again but with the value of the second locator which we use temporarily for the concatenation. We have used a final insert statement at the end of the sample program, because we want to avoid long lock durations on the base table. Another technique would be to insert the first input record into the table, assign a locator to the LOB value and start building the locator for a final update on the LOB column using the locator. When you plan to use the updating method, be aware that you hold a lock on the base table and on the auxiliary table. The exclusive lock on the base table probably prevents access by other users to the base table, depending on the lock size you use. Error-handling routines and possible data movement to non-file-variables are not mentioned in the example above. A sample program showing how to insert a LOB using a 10 MB host variable and a single locator chain is included in the SAMPLE2.TXT file described in Appendix D, “Additional material” on page 155.

Multiple locator chains The method described in Single locator chain is fast, but there is also one disadvantage while using it: Every pass through the loop acquires a new LOB locator also pointing to a previous one. Using this method, DB2 builds a chain of locators. Each locator is associated with a control structure that describes how to construct the new LOB value by concatenating a value copied from HV-LOB-DATA into a LOB data space with the contents of the identified LOB

38

Large Objects with DB2 for z/OS and OS/390

locator. This particular use of LOB locators does not hurt in terms of additional storage acquired in the LOB data space, the agent’s LOB storage limit and the system’s storage limit. But it is not too hard to imagine situations where you could run out of space very quickly without proper locator management. Each time a concatenation is added to the chain, the entire chain is interrogated via recursion. Let us consider the chains we build using the concatenation technique as secondary chains. See Figure 3-11 for a better understanding of what happens in a data space when you use the appending mechanism as mentioned above.

LOCATOR-1 Version 3

host variable #3

LOCATOR-1 Version 2

host variable #2

LOCATOR-1 Version 1

host variable #1

Figure 3-11 Secondary chain of locators

After assigning the first version of a locator to a host variable, the locator only contains the first host variable (Version 1). After issuing the first concatenation of :LOCATOR-1 = CONCAT (:LOCATOR-1, :HV-LOB), the second version of the same locator is created. Version 2 of the locator contains the new host variable and points to Version 1 of the same locator. So DB2 builds a chain of locators. Appending data to a locator becomes more expensive when the chain grows, because of the recursive interrogation technique we mentioned above. You can especially run into a long chain when you build a large object of comparatively small input records only using locators. This is not the recommended way to feed LOBs, but you may probably run into this situation. To improve performance, we can build a primary chain containing the previously built secondary chain to reduce the number of recursions when appending a new host variable to LOCATOR-1. So DB2 only has to go through the long chain containing all host variables when we append a secondary chain to a primary chain. See Figure 3-12 for a visualization of primary and secondary chains.

Chapter 3. Creating LOBs

39

LOCATOR-2 Version 1

LOCATOR-1 Version 3

host variable #3

LOCATOR-2 Version 2

LOCATOR-1 Version 6

host variable #6

LOCATOR-1 Version 2

host variable #2

LOCATOR-1 Version 1

host variable #1

LOCATOR-1 Version 5

host variable #5

LOCATOR-1 Version 4

LOCATOR-2 Version 1

host variable #4

Figure 3-12 Primary chain of locators containing secondary chains

As soon as a secondary chain becomes very large, we can source it out to a primary chain to reduce the level of recursion we have to go through when we append another host variable to our secondary chain. You can use this method, when appending big host variables to a single locator does not satisfy your performance requirements after a large number of concatenations for a secondary chain. The pseudo-code reported in Example 3-15 shows you how to perform the same insert as above including a technique dealing with primary and secondary chains. Example 3-15 Pseudo-code inserting LOBs > 32 KB with multiple locator chains Definitions: HV-LOB LOB-LOCATOR-1 LOB-LOCATOR-2

USAGE IS SQL TYPE IS CLOB (100K) USAGE IS SQL TYPE IS CLOB-LOCATOR USAGE IS SQL TYPE IS CLOB-LOCATOR

Additional instruction before reading the 1st INPUTRECORD: EXEC SQL SET :LOCATOR-2 = ‘’ END-EXEC Pseudo-Code: :APPEND-LOCATOR ADD 1

TO APPEND-LOCATOR-COUNTER

EXEC SQL SET :LOB-LOCATOR-1 = CONCAT (:LOB-LOCATOR-1, :HV-LOB) END-EXEC IF APPEND-LOCATOR-COUNTER = 1000 THEN EXEC SQL SET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-2, :LOB-LOCATOR-1) END-EXEC

40

Large Objects with DB2 for z/OS and OS/390

EXEC SQL SET :LOB-LOCATOR-1 = ‘’ END-EXEC MOVE 0 END-IF

TO APPEND-LOCATOR-COUNTER

:FINAL-INSERT IF HV-LOB-LENGTH > 0 THEN PERFORM APPEND-LOCATOR END-IF IF APPEND-LOCATOR-COUNTER > 0 THEN EXEC SQL SET :LOB-LOCATOR-2 = CONCAT (:LOB-LOCATOR-2, :LOB-LOCATOR-1) END-EXEC END-IF EXEC SQL INSERT INTO BASE_TABLE (KEYCOL1, COL2, LOB) VALUES (:HV-KEYCOL1, :HV-COL2, :HV-LOB-LOCATOR-1) END-EXEC EXEC SQL FREE LOCATOR :LOB-LOCATOR-1,:LOB-LOCATOR-2 END-EXEC

The difference between these two examples is the locator management in the APPEND-LOCATOR subroutine. After 1,000 times of appending a host variable (which means nearly 1 GB of data in our example), we create a primary chain by issuing SET :LOCATOR-2 = CONCAT (:LOCATOR-2, :LOCATOR-1). After LOCATOR-1 was outsourced successfully, we can reset it to point to an empty string to start building a new secondary chain. It is good programming practice to free both locators after the final insert statement in order to release the virtual storage allocated out of the data space, and make it available for further usage in the application. Otherwise the storage remains allocated until the implicit or explicit COMMIT releases the entire data space. Before you start coding your application, consider that every invocation of SQL takes a certain amount of time. So try to keep SQL calls at a minimum number and exploit as much storage as you can get for your host variable. This results in a reduction of SQL calls and leads you to a better performing application. But also be aware of storage allocations issues if you do not free your locators. A sample program showing how to insert a LOB using a 10 MB host variable and two locator chains is included in the SAMPLE3.TXT file described in Appendix D, “Additional material” on page 155.

Chapter 3. Creating LOBs

41

42

Large Objects with DB2 for z/OS and OS/390

4

Chapter 4.

Using LOBs In this chapter we discuss topics related to the differences in using LOBs when compared with the use of the normal DB2 objects. The chapter is structured as follows: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

SQL semantics with LOBs LOB locators Handling BLOBs, CLOBs, DBCLOBs LOBs are different DB2 objects Locking Details about ROWID

© Copyright IBM Corp. 2002

43

4.1 SQL semantics with LOBs There are some differences between SQL functions when accessing LOBs and when accessing other normal columns stored in a table. Logically, a row in the base table also contains the LOB value, and this is true from the application’s point of view. Physically, DB2 stores them in two different table spaces. You cannot access the auxiliary table via SQL where DB2 stores the LOB values. DB2 protects the auxiliary table by issuing SQLCODE -766 when you try to access it directly by using SQL. In general, LOBs can be referenced in all the string functions with the exception of those that relate to date and time. LOBs have the same set of restrictions as other long strings. Some functions cannot be used on LOB columns for obvious reasons. You cannot use the following functions on LOB columns: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

GROUP BY clause HAVING clause ORDER BY clause SELECT DISTINCT clause Column function Datetime function DECIMAL or NULLIF function WHEN clause of a CASE expression Subselect of a UNION without the ALL keyword

Most of the restrictions on LOB values are due to the fact that LOB values cannot be compared, except with the LIKE predicate.

LOB functions One way of manipulating large objects without retrieving the entire object is to use functions. Many of the string functions also work with LOBs. The built-in functions allow you to concatenate strings, get a substring, find the LOB length, find the position in the LOB of a search string, and cast the LOB to another type. UDFs can be used as well. The list of available LOB functions includes: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

CONCAT SUBSTR LENGTH POSSTR IFNULL VALUE / COALESCE Casts UDFs LIKE within predicates

In the following sections we discuss the most important of the functions listed above.

How CONCAT operates The CONCAT function allows you to put two strings together so that they end up as one string. You can do the same with your host language, but depending on what language you use there are some limits within which you cannot move easily. For example, you may not be able to concatenate two host variables containing each 10 MB of data, because the resulting variable would be bigger than the maximum allowed size of a variable, as allowed by COBOL for OS/390 and VM 2.2.1. But be aware not to use the CONCAT function every time you want to string two short variables together (like two variables of each 80 bytes), because invocation

44

Large Objects with DB2 for z/OS and OS/390

of SQL is more expensive than a single MOVE statement in COBOL, for example. Try to avoid unnecessary use of this function and use simple statements in your host-language as long as you can afford it. You can spend much CPU and elapsed time on thoughtless use of CONCAT operations. An SQL statement without any data access like this one uses about 7,500 machine instructions more than the same MOVE statement without any SQL invocation. The amount of data being moved is nearly the same, so you can assume the mentioned overhead for concatenating variables using SQL compared to a MOVE statement.

A few details about SUBSTR Your application programs can use the SUBSTR function to retrieve part of a LOB value. The string delivered to your application can consist of up to 2,147,483,647 bytes, so there are no limits in accessing parts of your LOBs. When your declared LOB column is smaller than 2 GB, the maximum value returned by the SUBSTR function is the declared size of your large object. Depending on what SUBSTR operation you perform, DB2 does not materialize the LOB, because the information where to look for the requested substring is stored in the LOB map pages.

The built-in function POSSTR You can use the POSSTR function, new with DB2 V6, to locate the starting position of one string within another string. POSSTR returns the first occurrence of one string. The string you want to locate (the search string) in the source string can consist of up to 4,000 bytes, which can also be represented by a host variable. The POSSTR function returns the position of your search string for BLOB and CLOB values. For DBCLOBs, a returned position is a DBCS character. For finding the second, third or nth occurrence of a string using POSSTR, please refer to 4.3.3, “Finding the nth occurrence of a string” on page 57.

IFNULL, VALUE, and COALESCE When you use the IFNULL function in your select statement, you can tell DB2 which value you want it to return to your application when an accessed LOB value is null. For example, you can return text saying ‘value unknown’ to your application if a particular LOB value is null. You can also return the value of an already assigned LOB locator, probably pointing to a picture saying “Image not found”. IFNULL (:LOB-LOCATOR, ‘unknown value’) IFNULL (:LOB-LCATOR, :LOCATOR-IMAGE-NOT-FOUND)

IFNULL is identical to the COALESCE and VALUE functions, except that IFNULL is limited to two arguments instead of multiple arguments. Because a NULL indicator is one of the two stored flags in the base table, DB2 does not need to access the auxiliary table at all, if a LOB column is stored as a null value.

Using CAST for LOBs You can also use CAST functions on your LOB values or LOB locators, even to convert your current LOB value into another value. They can be used to get around some of the restrictions on LOB values. To give you an idea of CASTing between data types, assume the following statement: SELECT LOB FROM BASE_TABLE WHERE SUBSTR (LOB,1,11) = ‘IBM Redbook’

You could expect that the statement returns you all the LOB values which start with ‘IBM Redbook’. Because large objects are subject to long string column restrictions, DB2 issues SQLCODE -134, complaining about an improper use of a long string column. To solve this problem, you can replace the statement using the following syntax: Chapter 4. Using LOBs

45

SELECT LOB FROM BASE_TABLE WHERE CHAR (SUBSTR (LOB,1,11)) = ‘IBM Redbook’

This statement delivers the result you could have expected when you issued the first statement. Table 4-1 shows the list of the allowed LOB conversions. Table 4-1 Casting large objects BLOB CHAR VARCHAR CLOB GRAPHIC VARGRAPHIC DBCLOB BLOB

CLOB CHAR VARCHAR CLOB GRAPHIC (*) VARGRAPHIC (*) DBCLOB (*)

DBCLOB CHAR (**) VARCHAR (**) CLOB (**) GRAPHIC VARGRAPHIC DBCLOB

(*) CAST is only supported if the data is Unicode. (**) CAST is only supported if the data is Unicode. The result length for these casts is 3 * LENGTH (graphic string)

The reason for the length growing up to three times with DBCLOB conversions into a CHAR, VARCHAR or CLOB column, is to allow for expansion when the data is converted from UTF-16 to UTF-8. A character that takes two bytes to represent in UTF-16, can take three bytes in UTF-8.

How does LIKE operate on LOBs? Depending on the value specified for LIKE search arguments, LOB Manager takes different actions. These actions correspond to common LIKE operations used for non LOB values. When you issue LIKE ‘Chapter 8%’, only the first nine bytes of a LOB value according to qualifying base table rows are scanned to verify the result. A LIKE ‘%Chapter 8%’ clause can be worse, because now DB2 has to scan the entire LOB value until it finds the first occurrence of the string to qualify the LOB value for your SQL statement. This kind of statement can be very time consuming, depending on your average LOB size, and the number of touched rows in the base table. In general, after issuing an SQL statement against the base table, all base table columns appearing in the WHERE clause of your statement are checked before a condition on the auxiliary table is checked, even if the columns in the base table are not indexed. This avoids unnecessary scans of your LOB values and only those LOB values are scanned which already have qualified from the base table point of view.

4.2 LOB locators You can use locators everywhere in your application program where you can access a LOB value by itself. The idea behind locators is that an application program only deals with a reference to a particular LOB and DB2 performs the real operations on the LOB value. Using this method, the LOB value is not stored in the application’s memory. To deal with a locator instead of the entire value, an application program simply selects the LOB column into a LOB locator host variable, not into the generated LOB host variable. By selecting into a locator variable DB2 assigns a 4-byte value to the locator which is delivered to the application. The entire LOB value is not retrieved nor delivered to the application. By selecting the whole LOB into a locator variable, you avoid materialization of the LOB, because DB2 does not have to access every page of the LOB, it just keeps the reference.

46

Large Objects with DB2 for z/OS and OS/390

For example, when selecting a LOB value, an application program could select the entire LOB value and place it into a host variable of the same size as the LOB (which is acceptable if the application program is going to process the entire LOB value at once), or it could instead select the LOB value into a LOB locator. Then, using the LOB locator, the application program can issue further database operations on the LOB value (such as applying scalar functions SUBSTR, CONCAT, LENGTH, doing an assignment, searching the LOB with LIKE or POSSTR, or applying UDFs against the LOB) by supplying the locator as input. The resulting output of the locator operation, for example the amount of data assigned to the application’s host variable, would then typically be only a subset of the input LOB value. Once a locator is set to a particular LOB value, there is no action you can take within the data base to change that value from the application’s point of view until the locator is freed. You can free a locator by issuing a FREE LOCATOR statement or by completing the current unit of work. But how does this work? Locators do not force extra copies of the data in order to provide this function. When you assign a locator to a specific LOB value, an S-LOB lock (see 4.5, “Locking” on page 62 for a description of LOB locks) is acquired on the LOB value, and it is only released at COMMIT or when you explicitly free the locator. When you issue the HOLD LOCATOR statement, an assigned locator survives the current unit of work and is valid until the thread terminates or a FREE LOCATOR statement is passed to DB2. Either one of these events will release the S-LOB lock taken on the LOB value in the auxiliary table.

4.2.1 Getting to know LOB locators You assign a locator to a LOB value when selecting a LOB column into a LOB locator. Using this method, DB2 detects that the entire LOB is going to be selected into a locator column and therefore it does not materialize the associated LOB value and also does not provide the LOB value to your application program. Instead, DB2 provides a value for your locator into your locator host-variable where your application is selecting the LOB value into. You can use a LOB locator anywhere a LOB value can be used, for instance in expressions where you would normally use an entire LOB value.

Can the LOB value change while a locator is assigned? Simply speaking: Yes, it can. But it will not change for your application program once you have associated a locator to a particular LOB value. So what is DB2 doing when you have a locator assigned to a LOB value (remember: a locator represents a LOB value at a point in time) and somebody else is updating or deleting the LOB? DB2 ensures that the locator still represents the original value that existed at the time the locator was set. See figure Figure 4-1.

Chapter 4. Using LOBs

47

TRANSACTION 1

01 movie_loc ... IS SQL TYPE IS BLOB-LOCATOR SELECT MOVLENGTH, LOBMOVIE INTO :length, :movie-loc FROM LITERATURE WHERE TITLE = 'The Maltese Falcon';

TRANSACTION 2 UPDATE LITERATURE SET LOBMOVIE=:edited-version WHERE TITLE = 'The Maltese Falcon';

INSERT INTO VIDEOLIB (LENGTH, FILM, DESC) VALUES(:length, :movie-loc, 'Uncut/Original');

Figure 4-1 Concurrent LOB access using LOB locators

In this example, transaction 1 selects a non-LOB column into a host-variable and also a LOB column into a locator variable. After the SELECT is completed in transaction 1, transaction 2 updates the LOB column which is already referenced by a locator in transaction 1. After the LOB is updated by transaction 2, a new value is inserted into the VIDEOLIB table in transaction 1, using the previously assigned locator for the LOB value by DB2. But the value referenced to by the locator is still the same as before the updating transaction has updated the LOB value. How is this possible? Operations on the original LOB value have definitely no effect on the value referenced by a locator. Let us first have a look at LOB delete: a DELETE statement only de-allocates pages in the Auxiliary Table where the LOB data is stored. But the data still remains in the pages, it is not deleted. Now let us have a close look on how a LOB is updated: Updating a LOB consists of DELETE and INSERT, so the pages are de-allocated again and the data still remains in the auxiliary table. The INSERT statement cannot use the de-allocated pages because of the S-LOB lock on the LOB value acquired by the former issued SELECT statement into the LOB locator. The conclusion is that these data pages where a locator is pointing to cannot be allocated again, even if they are going to be de-allocated by a DELETE statement, as long as an S-LOB lock persists on the referenced LOB value.

48

Large Objects with DB2 for z/OS and OS/390

Using locators across multiple units of work A LOB locator is only a mechanism used to refer to a LOB value during a unit of work. Ordinarily, a locator is freed, implying release of acquired space by DB2 in a LOB data space if the locator is not referenced further, when the application COMMITs its data or the associated thread terminates. Acquired locks are also released at COMMIT time as usual. To increase locator durations beyond COMMIT points, a HOLD locator statement can be issued. This statement increases the duration of the specified locator, and the locks held on the locator, until a FREE locator statement is issued for the same locator. SQL ROLLBACK also frees every locator having the hold property. If no FREE locator statement is issued, the locator is valid until the thread terminates. In Example 4-1 you can find the syntax of FREE locator and HOLD LOCATOR statements. Example 4-1 Syntax for FREE locator and HOLD locator Hold beyond COMMIT: EXEC SQL HOLD LOCATOR :HV-LOCATOR-1, :HV-LOCATOR-2 END-EXEC Free the locator if it is not needed any more: EXEC SQL FREE LOCATOR :HV-LOCATOR-1, :HV-LOCATOR-2 END-EXEC

A locator is freed when one of the following conditions occur: 򐂰 An SQL FREE LOCATOR statement occurs 򐂰 An SQL ROLLBACK statement occurs 򐂰 The associated thread terminates If a locator has the hold property, it survives the SQL COMMIT statement. Without the hold property, it does not survive the SQL COMMIT statement. A locator obtains the hold property by the SQL HOLD LOCATOR statement. If you receive SQLCODE -423 (INVALID LOCATOR VALUE) after you have specified more than one host-variable for a FREE LOCATOR statement, only those locators up to the invalid locator will be freed. According to this result, when you receive SQLCODE -423 after issuing a HOLD LOCATOR statement, all locators listed in the statement after the first invalid locator will not be held.

Can I free a referenced locator? Assume two locators as shown in Figure 3-12 on page 40, where LOCATOR-2 references LOCATOR-1. When you now free LOCATOR-1 using the FREE LOCATOR statement, it does not release any DB2 resources in this case, since LOCATOR-2 still refers to LOCATOR-1. The locator is only freed externally. This means that the application cannot refer to it; when it does, it probably gets SQLCODE -423 indicating an invalid locator. Internally, DB2 keeps LOCATOR-1 around as long as it is referenced.

When to use locators If LOBs are not too large, they can be managed as other data. LOBs may be retrieved, inserted and updated, just like any other type of data. Everything works well as long as you have enough main storage space to store the LOB. If you think of a CLOB containing a book’s text and of an application which has to extract only a single chapter of the book, without the use of locators you have to retrieve the whole object (probably 2 GB of data) in your Chapter 4. Using LOBs

49

application and then to extract the data you need. Because of their size, LOBs may be unmanageable for a single application. Huge amounts of storage may be needed to buffer their values, and it may not be possible to acquire contiguous buffers of sufficient size, since this storage must reside within the region size of the address space in which the application program is running. To give you an easy method to deal with LOB columns, DB2 provides locators to make LOB access manageable. Using a locator may be a good choice in the following situations: 򐂰 򐂰 򐂰 򐂰

Only a part of a LOB is needed by the application Only a part of a LOB is moved to a client The entire LOB does not fit into the application’s memory A temporary LOB value is needed but it has not to be stored in DB2

You cannot use any of the locators mentioned above in mathematical operations. The reason is due to the need of preventing the locators from being corrupted by the application.

Materialization when using locators If you do not use the locator technique to access your LOBs, then DB2 has to materialize the value represented by a large object when you access it. In this case, DB2 materializes the LOB by moving it through the buffer pool into the user’s address space. Basically, materialization is avoided when you select LOBs or parts of their values into locators. When an application retrieves a LOB value, whether it is assigned to a locator or not, it goes through the buffer pool into the user address space. As soon as a LOB value has to be materialized and a locator is involved, materialization of a LOB takes place in a data space created by DB2. This mainly occurs when you update a LOB value by using locators. In this case, the DBM1 address space only contains the LOB’s control structure. DB2 tries to avoid materialization wherever it is possible, but if it is no longer possible, it will materialize a certain value in a data space or in the user address space, depending on the way a LOB is accessed. For more information about materialization, see 7.1, “LOB materialization” on page 120.

Locators and expressions LOB locators may also represent more than just base values. They can also represent the value associated with a LOB expression. For example, a LOB locator can represent the value associated with: EXEC SQL SET LOCATOR = SUBSTR (LOB1, :START1, :LENGTH1) CONCAT SUBSTR (LOB2, :START2, :LENGTH2) END-EXEC

The same statement can also be issued even if LOB1 and LOB2 are referenced by locators. You are also able to use other LOB functions for further reference in an expression. Regarding this implementation, DB2 for OS/390 and z/OS provides you with the opportunity to build new locators using other locators and expressions of LOBs or locators. This gives you the flexibility to deal with large objects in various scenarios without allocating huge amounts of storage when you deal with LOBs.

Examples of using locators A LOB locator is also allowed to represent LOB expressions, this means substrings of an entire LOB value. A LOB expression is defined as any expression that refers to a LOB column or results in a LOB data type. LOB functions may also be part of a LOB expression. Also LOB expressions may reference other LOB locators, which does not simplify this topic at all.

50

Large Objects with DB2 for z/OS and OS/390

A LOB expression is any string expression that contains a LOB value. It is possible to associate the result of a LOB expression to a LOB locator. Since a LOB locator can be used anywhere a LOB value can be used, the LOB expression associated with a LOB locator could make reference to other LOB locators.

Concatenating two strings using locators When you plan to use any string expressions on a LOB value, you can also use a locator instead of the entire LOB value. For example: when you try to concatenate two LOBs of 10 MB each, you do not have to read them both into two large host variables and string them together, you can instead let DB2 do the work for you. Assume that LOCATOR-1 is pointing to a CLOB value containing a 10 MB string, and LOCATOR-2 is pointing to another CLOB value of another 10 MB. After assigning both locators, you can simply issue this to string them together: EXEC SQL SET :LOCATOR-3 = CONCAT (:LOCATOR-1, :LOCATOR-2) END-EXEC

The new value of both strings, now a total 20 MB, is assigned to the new locator LOCATOR-3 where you can now refer to. To store the new value in a table, you can insert a LOB using the locator as a reference in an insert statement or perform other operations like SUBSTR.

Referring to a block of data inside of a CLOB If you want to refer to a single chapter in a document, you can do this also by simply using locators. The first thing an application has to do is to assign a locator to a particular LOB value. The starting position of the chapter you want to refer to is easily determined by the integer value resulting from the POSSTR function as follows: EXEC SQL SET :START-POSITION = POSSTR (:LOB-LOCATOR, ‘Chapter 8’) END-EXEC

You can determine the end of the chapter you want to deal with in the same way using the POSSTR function. In our example the position of the beginning of Chapter 9: EXEC SQL SET :END-POSITION = POSSTR (:LOB-LOCATOR, ‘Chapter 9’) END-EXEC

After issuing this statement, the locator END-LOCATOR points to the beginning of Chapter 9. When your application wants to deal with Chapter 8 directly, you can set the content of the chapter to a host variable being large enough to contain the entire text for Chapter 8. Otherwise simply set the string containing Chapter 8 to a new LOB locator as shown below: EXEC SQL SET :CHAPTER-8-LOCATOR = SUBSTR (:LOB-LOCATOR ,:START-POSITION ,:END-POSITION - START-POSITION) END-EXEC

The SUBSTR function uses the LOB locator (instead of the whole LOB value) as a string-expression, the START-POSITION represents a starting-position and the expression END-POSITION - START-POSITION is provided as a length value. Note: Try to avoid the use of string expressions (such as SUBSTR) of BLOB values, because binary documents may not be usable in parts, such as pictures, executable files or even movies. The use of parts for further processing depends on the type of data you store in your LOB columns.

Chapter 4. Using LOBs

51

4.3 Handling BLOBs, CLOBs, DBCLOBs Dealing with all kinds of large objects is challenging from the application’s point of view. You face new challenges in dealing with objects bigger than most common data objects. In this section we discuss application programming techniques to manipulate LOBs without retrieving them, and also unloading LOBs to store them in a data set. If you need your data in a flat file, you cannot use normal unloading techniques, because the data has to be split over more than one row (the maximum LRECL you can use is 32,760). You can find examples for inserting LOB data via an application in 3.4, “Feeding a LOB column” on page 33.

4.3.1 Manipulating a LOB without retrieving it In most common scenarios you would only need to manipulate CLOBs. Manipulating BLOBs can result in unusable binary data if important parts are removed or updated. The updated binary value may become unreadable for the application using this data. As an example, just think of updating parts of a JPEG picture or an MPEG movie. Cutting out or updating some data in the movie may destroy it for further use.

Case 1: Deleting a specific part of a LOB To delete a special part of a LOB value, the first step is to locate the start and the end positions for your delete. In our example we are going to delete ‘Chapter 8’ of our book CLOB. The first step our application should do is to assign a locator on the LOB we want to update. After the locator is set, we use the POSSTR statement to figure out the position of the beginning of ‘Chapter 8’. The position of the end of ‘Chapter 8’ in our book is marked by the string ‘Chapter 9’. When our application knows both positions, it can assign a new locator using the SUBSTR function to point to the beginning of the book up to the beginning of ‘Chapter 8’ minus one byte, concatenating the rest of the book beginning at the end position of ‘Chapter 8’, which is the beginning of ‘Chapter 9’. After the new locator is established, the LOB is finally updated. See Example 4-2 for a pseudo-code performing the actions as mentioned above. Example 4-2 Delete Chapter 8 of book CLOB using locators Definitions: LOB-LOCATOR-1 LOB-LOCATOR-2

USAGE IS SQL TYPE IS CLOB-LOCATOR USAGE IS SQL TYPE IS CLOB-LOCATOR

START-POSITION END-POSITION

PIC S9(9) USAGE IS BINARY PIC S9(9) USAGE IS BINARY

Pseudo-Code: EXEC SQL SELECT INTO FROM WHERE END-EXEC

LOB :LOB-LOCATOR-1 BASE_TABLE KEYCOL1 = :HV-KEYCOL1

EXEC SQL SET :START-POSITION = POSSTR (:LOB-LOCATOR-1, ‘Chapter 8’) END-EXEC EXEC SQL SET :END-POSITION = POSSTR (:LOB-LOCATOR-1, ‘Chapter 9’)

52

Large Objects with DB2 for z/OS and OS/390

END-EXEC EXEC SQL SET :LOB-LOCATOR-2 = SUBSTR (:LOB-LOCATOR-1, 1, :START-POSITION - 1) CONCAT SUBSTR (:LOB-LOCATOR-1, :END-POSITION) END-EXEC EXEC SQL UPDATE BASE_TABLE SET LOB = :LOB-LOCATOR-2 WHERE KEYCOL1 = :HV-KEYCOL1 END-EXEC EXEC SQL FREE LOCATOR :LOB-LOCATOR-1, :LOB-LOCATOR-2 END-EXEC

A sample program showing how to delete a specific part of a LOB using locators is included in the SAMPLE6.TXT file described in Appendix D, “Additional material” on page 155.

Case 2: Updating a specific part of a LOB When you consider updating your LOB values in your application, you can build the new content of the LOB in the same way we have shown for insert cases in 3.4.2, “Inserting LOBs via host application” on page 34. The other way to apply necessary updates consists of locator usage. Using locators, you can simply replace parts of your LOB value or even delete them. Depending on the size of your LOBs and the number of updates you want to perform on a single LOB value, you have to decide when you want to use locators and when to replace a whole LOB value by building it in the application’s memory. You can easily compare updating a part of a LOB with deleting a part of a LOB as shown in case one, except that the new locator is set up in a different way. After determining the start and end position of your update, the only difference to Example 4-2 is the changed assignment of LOCATOR-2. Example 4-3 Updating a part of a CLOB EXEC SQL SET :LOB-LOCATOR-2 = SUBSTR (:LOB-LOCATOR-1, 1, :START-POSITION - 1) CONCAT :NEW-TEXT CONCAT SUBSTR (:LOB-LOCATOR-1, :END-POSITION) END-EXEC

So LOCATOR-2 is made of the former text referenced by LOCATOR-1 up to the start position minus one byte, the NEW-TEXT variable which can consist either of a host variable or a LOB locator, and the remaining text referenced by LOCATOR-1 from your end position up to the end of the LOB value Using the technique mentioned above, you are also able to insert a certain text at a particular position in your CLOB. The only thing you have to figure out is the position where you want to insert the text in your CLOB. A sample program showing how to update a specific part of a LOB using locators is included in the SAMPLE7.TXT file described in Appendix D, “Additional material” on page 155.

Chapter 4. Using LOBs

53

Let us now assume that you want to insert more text at the end of ‘Chapter 8’. For this reason you only have to find a position in the LOB where you want to place the new text value. After you have determined the correct position by using POSSTR, you can assign a new locator to the complete new value of the LOB. Example 4-4 shows you how you can perform an insert of new text to a known position in a LOB. Example 4-4 Inserting a text at a particular position EXEC SQL SET :LOB-LOCATOR-2 = SUBSTR (:LOB-LOCATOR-1, 1, :START-POSITION - 1) CONCAT :NEW-TEXT CONCAT SUBSTR (:LOB-LOCATOR-1, :START-POSITION) END-EXEC

In this example the NEW-TEXT variable can also be a host variable or another LOB locator.

4.3.2 Unloading a LOB via application Unloading a LOB value into a certain data set is easy if compared to the manipulating techniques described in 4.3.1, “Manipulating a LOB without retrieving it” on page 52. In general, the method you use to unload LOB values depends on the maximum available size of a host variable. If you are able to allocate a host variable being large enough to contain the maximum size of a LOB value you want to unload, unloading is easier than using locators.

Case 1: Unloading a LOB using one host variable Assume you want to unload a 10 MB CLOB value of a text document. The first thing you have to make sure is the format in which the data is needed, because you cannot write it to a single row in a data set. So the application has to split up the data into the requested file format. In our example we assume an output file of LRECL 1024 using record format FB. So the application selects the entire LOB value into a host variable and writes the value of the host variable in 1,024 byte pieces to an output data set. Example 4-5 provides a pseudo-code to perform a LOB unload via application using one host variable. Example 4-5 Unloading LOB data using one host variable Definitions: HV-LOB

USAGE IS SQL TYPE IS CLOB (10M)

Pseudo-Code: EXEC SQL SELECT INTO FROM WHERE END-EXEC

LOB :HV-LOB BASE_TABLE KEYCOL1 = :HV-KEYCOL1

PERFORM WRITE-DATA :WRITE-DATA MOVE 1 PERFORM UNTIL BYTE-COUNTER > HV-LOB-LENGTH MOVE HV-LOB-DATA (BYTE-COUNTER:OUTPUT-FILE-LENGTH) WRITE OUTPUT-RECORD ADD OUTPUT-FILE-LENGTH

54

Large Objects with DB2 for z/OS and OS/390

TO BYTE-COUNTER

TO OUTPUT-RECORD TO BYTE-COUNTER

END-PERFORM

This solution works fine as long as the maximum LOB value is not bigger than the largest size of a host variable you are allowed to acquire. If you are not able to acquire a host variable of the size of your maximum LOB value, you can use locators for unloading your data as ontinued in Case 2. A sample program showing how to unload a LOB using a host variable is included in the SAMPLE4.TXT file described in Appendix D, “Additional material” on page 155.

Case 2: Unloading an entire LOB using locators When you are not able to define a host variable of a size of the LOB data you want to unload, you can use a different technique. A method to unload huge amounts of data is using locators to reference a whole LOB value. The application only retrieves a part of a LOB which can be easily written to an output file. After processing the retrieved part, the next part is retrieved and also written to an output file. We assume the same file attributes as in Example 4-5. Example 4-6 gives you an idea how to unload a LOB value using locators when you are only allowed to use a 1 MB host variable and your LOBs are larger than 1 MB. Example 4-6 Unloading a LOB using locators Definitions: HV-LOB LOB-LOCATOR-1

USAGE IS SQL TYPE IS CLOB (1M) USAGE IS SQL TYPE IS CLOB-LOCATOR

Pseudo-Code: EXEC SQL SELECT INTO FROM WHERE END-EXEC

LENGTH(LOB), LOB :CURRENT-LENGTH, :LOB-LOCATOR-1 BASE_TABLE KEYCOL1 = :HV-KEYCOL1

COMPUTE AMOUNT-FULL-SUBSTR = CURRENT-LENGTH / MAX-VAR-SIZE COMPUTE AMOUNT-REMAIN-SUBSTR = CURRENT-LENGTH - (AMOUNT-FULL-SUBSTR * MAX-VAR-SIZE) PERFORM VARYING SUBSTR-COUNTER FROM 1 BY 1 UNTIL SUBSTR-COUNTER > AMOUNT-FULL-SUBSTR EXEC SQL SET :HV-LOB = SUBSTR(:LOB-LOCATOR-1, 1, MAX-VAR-SIZE) END-EXEC PERFORM WRITE-DATA END-PERFORM IF AMOUNT-REMAIN-SUBST > 0 THEN EXEC SQL SET :HV-LOB = SUBSTR(:LOB-LOCATOR-1,1,:AMOUNT-REMAIN-SUBSTR) END-EXEC PERFORM WRITE-DATA END-IF EXEC SQL FREE LOCATOR :LOB-LOCATOR-1 END-EXEC

Chapter 4. Using LOBs

55

Before we start retrieving parts of the LOB, the application sets a locator to a LOB value in order not to allow changes to the LOB value during the unit of work. Otherwise the content of the LOB can change while the application processes the LOB data. In this case we acquire a locator to refer to a frozen LOB value for our unit of work. The idea behind retrieving values via a locator is the same idea we use when we insert LOB value: try using large host variables to reduce the number of your SQL calls. A sample program showing how to unload a LOB using a host variable of 1 MB and locators is included in the SAMPLE5.TXT file described in Appendix D, “Additional material” on page 155. The variable AMOUNT-FULL-SUBSTR tells your program how often it has to perform the SUBSTR to retrieve a part of the LOB value of the size of your largest host variable. For the remaining bytes of the LOB we calculate AMOUNT-REMAIN-SUBSTR to retrieve those bytes which are not covered by the previously issued statements. After retrieving the results of each SUBSTR function, the data is written to a file using the WRITE-DATA subroutine as mentioned in case one. This example for retrieving your LOB data avoids materialization of the LOB, because DB2 knows where the parts you want to retrieve via the SUBSTR function are stored. Therefore, DB2 uses the LOB pageset structure to locate the data pages you want to retrieve. For a detailed description of the LOB pageset structure, see 4.4.1, “Storage of LOBs” on page 60. Figure 4-2 shows you how a delete can affect your unit of work when you do not use a locator technique to freeze a LOB value from your application’s view, or isolation levels that can protect you from this situation. In our specific example we assume cursor stability (CS) as the isolation level with CURRENTDATA(NO) for lock avoidance and RELEASE(COMMIT).

T1 SELECT SUBSTR (LOB, 1, 1024) WHERE KEYCOL = Vala

S-LOB Lock

SELECT SUBSTR (LOB, 1025, 1024) WHERE KEYCOL = Vala

S-LOB Lock

T2

S-LOB Lock

SELECT SUBSTR (LOB, 2044, 1024) WHERE KEYCOL = Vala

SQLCODE 100

DELETE LOB WHERE KEYCOL = Vala COMMIT

Figure 4-2 Accessing a LOB without a locator reference using ISOLATION (CS)

As you can see, every SELECT SUBSTR acquires an S-LOB lock for the time it takes to retrieve the requested value. As soon as DB2 has delivered the value (depending on your current settings for DB2 locking), it releases the S-LOB lock. The LOB lock is taken again by DB2 as soon as the next SELECT SUBSTR statement is issued. When you use this method, a second transaction is able to delete the LOB which is currently processed by another transaction.

56

Large Objects with DB2 for z/OS and OS/390

To avoid this kind of situation, use a locator to freeze the object you currently access. When your application assigns a locator to a particular LOB value, the lock is held by DB2 until you explicitly free the locator or issue DB2 COMMIT as you can see in Figure 4-3.

SELECT LOB INTO :LOB-LOCATOR WHERE KEYCOL = Vala

T1

T2

S-LOB Lock

SELECT SUBSTR (:LOB-LOCATOR, 1, 1024) S-LOB Lock SELECT SUBSTR (:LOB-LOCATOR, 1025, 1024)

DELETE LOB WHERE KEYCOL = Vala COMMIT end of UOW / thread

SELECT SUBSTR (:LOB-LOCATOR, 2049, 1024) . . . end of UOW / thread

Figure 4-3 Processing a LOB using a locator reference

Transaction two acquires an S-LOB lock which is compatible with the S-LOB lock already held by transaction one. After transaction two issues a COMMIT and its thread terminates, the LOB is only visible for transaction one. The pages are finally deallocated when transaction one releases its S-LOB lock for the accessed value.

Case 3: Unload parts of a LOB using locators In some cases your application may want to retrieve only a known portion of a LOB value. In this case you do not have to retrieve the entire LOB value, you can retrieve just parts of it. You can retrieve parts with and without locators. If you want to have a LOB’s part in your host variable for further processing, you can use the SELECT in case one including the LOB function to retrieve only a part of the value. But if even the part you want to retrieve is too big for one single host variable, you can use the pseudo-code mentioned in Case 2 to retrieve only the data you need.

4.3.3 Finding the nth occurrence of a string Since the POSSTR function provides you with the ability to find the first occurrence of a string, you probably want to find the second or third position of your search string. DB2 allows you to use POSSTR function to succeed anyway, but you have to combine it with SUBSTR, since both functions you are able to quickly find the position you need. Only from the application point of view it is a bit more difficult than finding the first position. Example 4-7 provides a possible solution for searching a LOB value for the position of a search string you really need. Example 4-7 Finding a specific occurrence of a string EXEC SQL SET :POS = POSSTR (:LOB-LOCATOR, :SEARCH-STRING) END-EXEC [determine if correct position is returned]

Chapter 4. Using LOBs

57

IF WRONG-POSITION THEN MOVE 0

TO FINAL-POS START-POS

PERFORM UNTIL CORRECT-POSITION ADD POS TO FINAL-POS COMPUTE POS-START = FINAL-POS + STRING-LENGTH EXEC SQL SET :POS = POSSTR (SUBSTR (:LOB-LOCATOR, :POS-START), :SEARCH-STRING) END-EXEC [determine if correct position is returned] END-PERFORM END-IF

The first POSSTR statement returns as usual the first search string position in the LOB value ‘hiding’ behind a LOB locator.

Cursors and LOB values When you plan to vary between host variables, where you fetch your LOB values into, using a cursor (such as FETCH into host variable or FETCH into LOB locator variable), make sure that the CURRENT RULES special register has the correct value. CURRENT RULES STD lets you switch between host variables you fetch the cursor into. For more information about CURRENT RULES, see 3.3.2, “Impact on cursors fetching LOB values” on page 33.

List prefetch for LOBs With respect to the size of LOBs, the only prefetch for LOBs DB2 performs is for a single LOB value. If a LOB occupies more than one page, LOB Manager will prefetch up to a chunk of pages at a time for the LOB value. You have to look at prefetching for LOBs as a different dimension from prefetching normal rows to reduce I/O overhead as much as possible.

4.4 LOBs are different DB2 objects Dealing with LOBs means dealing with new objects in your DB2 subsystem, and that includes new ways to do so. In the following sections we talk about differences between standard DB2 tables and the new LOB tables and columns.

Loading a LOB column Because the maximum LRECL for a file in z/OS or OS/390 is 32,760 bytes, large objects bigger than 32 KB cannot be loaded using the LOAD utility. An input file is not designed to contain a input variable spanned over more than one single row. For LOB values less than or equal to 32,756 bytes (you have to add a four byte length column preceding the data in the file) the LOAD utility can be used. For inserting LOB values bigger than 32,756 bytes you have to use an application program. But DB2 allows you to use the LOAD utility if you are going to load LOB columns up to 32,756 bytes even if the LOB column itself is declared as 2 GB. For additional information regarding LOAD, see 3.4.1, “LOAD with LOB columns smaller than 32 KB” on page 34 and 6.2.1, “LOAD” on page 94. Note: You always LOAD the data into the base table and DB2 will automatically store the LOB data in the associated auxiliary table.

58

Large Objects with DB2 for z/OS and OS/390

New lock sizes With LOBs, DB2 V6 also introduced a new way to store huge amounts of data that can span over many pages. Common locking techniques acquire locks on table spaces, partitions, tables, pages or rows. Considering a LOB table space, there is only one table space, containing one table which possibly holds several LOB values. DB2 does not acquire a lock on many LOB data pages, nor on the entire table space. Therefore, two new lock sizes are introduced to efficiently handle locking for large objects: 򐂰 Shared LOB lock (S-LOB lock) 򐂰 Exclusive LOB lock (X-LOB lock) These new types of locks are also acquired via internal resource lock manager (IRLM). LOB locks are not related to any pages at all. DB2 takes both lock types explicitly by a combination of the LOB table space, the associated ROWID, and the LOB version number. For a more detailed description on locking for large objects and lock sequences, see 4.5, “Locking” on page 62.

Update mechanisms for LOBs When you update a large object, DB2 uses an updating technique different from the standard objects in DB2. Updating a LOB for DB2 means de-allocation of used data pages, and allocating and inserting new data pages which contain the new LOB value. For lock-related information on this topic, see 4.5.5, “Locks with UPDATE” on page 67, and for rollback-related information, see “Shadow Copy Recovery” on page 67.

Large objects and indexes For obvious reasons a LOB column in the auxiliary table is not indexable. The maximum length of an index column is 255 bytes and LOBs are even bigger. Therefore you cannot create an index on a LOB column.

Compression DB2 does not allow you to specify COMPRESS YES for LOB table spaces. Most objects stored in a BLOB column are already compressed anyway, like JPEG pictures or ZIP folders. Regardless of compression for a LOB table space, you can specify COMPRESS YES for the table space containing the base table.

EDITPROCs, FIELDPROCs, and VALIDPROCs LOB values cannot be compared, except with the LIKE predicate, and since they are not stored along with other columns, they are not available to any database procedures such as EDITPROCs, FIELDPROCs or VALIDPROCs. Note that although they are not available to these types of procedures, they are available to any triggers that are defined on the base table.

Check constraints A check constraint cannot reference a LOB column.Those values are not designed to be eligible to work with constraints, because they are too large to check when changing a row’s content in a table.

Chapter 4. Using LOBs

59

Data capture and data propagation (DpropR) Even if LOB values have been defined with LOG NO, DpropR is able to replicate those values. The capture program reads the LOB descriptor to determine if any data in the LOB column has changed or not, and places an indicator in the capture data table. When the apply program reads the indicator, it then copies the entire LOB value, not just the changed parts of the LOB value. The apply program always copies the most current version of a LOB column directly from the source table, in our case the auxiliary table. So it replicates only full LOBs, parts of a LOB are not replicated. Replication is not supported for DB2 Extenders for Text, Audio, Video and Image or other extenders where additional control files associated with the extender’s LOB column data are maintained outside of the database. For further details on DpropR replicating LOBs, refer to DB2 UDB Replication Guide and Reference Version 7, SC26-9920.

4.4.1 Storage of LOBs LOB table spaces have an entirely new format. Because a LOB entry in a LOB column can span pages, pages have to be chunked together. A chunk is referred to as 16 pages of contiguous space acquired in a LOB table space. Depending on your size of data, a certain number of chunks is allocated. A single LOB value can be stored using many chunks also as using page allocations of less than 16 contiguous pages. Figure 4-4 shows you a possible storage of a single LOB value. The LOB in our example is stored in a chunk containing pages 1 to 16, pages 21 to 28, another chunk beginning at page 49 and at last four single pages starting at page 83.

4

. .

.

16 24

3 2

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

22 21

. .

52

. . .

10 01101011 01 011001 01 1001 . . .

.

11 01 10 11 00

.

. .

64 86 85

50 49

1001101011 01011001 011001 . . .

11 01 10 11 00

11 01 10 11 00

51

11 01 10 11 00

11 01 10 11 00

84

11 01 10 11 00

83

. . .

1001101011 01011001 011001 . . .

Figure 4-4 LOB value spanned over pages using chunks and non-chunks

60

28

23

1

1001101011 01011001 011001 . . .

. . .

Large Objects with DB2 for z/OS and OS/390

11 01 10 11 00

11 01 10 11 00

11 01 10 11 00

LOB table space organization Storing LOBs causes the table spaces containing their values to be organized in a slightly different way in order to support pageset structures for columns spanning the maximum available page size. As in every other table space, a LOB table space has only one header page. The header page contains DB2 internal control information which is needed when you access your LOB data. The new type of pages are LOB space map pages. They are structured like a multi level index (an index containing several levels of leaf pages and are basically pointers to chunks and pages. You can find at least one LOB map page for every version of every single LOB value in your LOB table space. When we talk about de-allocation of LOB pages, the space map pages have set the invalid flag for all pages containing the LOB information, which includes both LOB map pages and LOB data pages. LOB map pages contain descriptive information about LOB values, the chunk information itself is stored in LOB map pages, which point to the associated data pages for a LOB value. After reading the information stored in the LOB map pages, DB2 knows where to find the information it has to retrieve to access a particular LOB value. See Figure 4-5 for a possible LOB map page information about a single value.

Space Map Page

LOB MAP Page #P

Chunk

104

16

Chunk

410

16

Page

503

9

Page

602

8

Page

664

3

LOB MAP Pages

16

106 105 104

LOB Data Page

9

. . .

}

Page

}

Allocation Chunk/Page

505 504 503

. . .

LOB Data Page

Figure 4-5 LOB data pages chunked together via space map and LOB map pages

You can think of two different allocation units that a LOB map page points to. First, it can point to a chunk of data pages, which is nothing more than 16 pages of contiguous space (indicated by #P = 16 in Figure 4-4). As soon as DB2 uses partial chunks to store parts of the LOB value, it contains the page number where the allocation starts and the number of contiguous pages used.

Chapter 4. Using LOBs

61

For a detailed description of LOB system pages, refer to the licensed documentation DB2 UDB for OS/390 and z/OS Version 7 Diagnosis Guide and Reference, LY37-3740. Note: When using DSN1PRNT, you may also see pages referring to LOB values that have been already deleted and whose space was not reused up to now.

4.5 Locking Locking considerations for LOBs are different from normal locking techniques for other DB2 objects. Because a single LOB can grow up to 2 GB and therefore it’s data is spanned over many pages, a new mechanism was introduced to ensure row consistency and data integrity while users are accessing LOB values. This is done using a combination of base table locks and the new LOB-locks, which were introduced in DB2 V6. For the following examples and explanations we assume LOCKSIZE ANY. If you specify other lock sizes in your environment, the locked objects may differ.

The LOB lock A LOB lock is a new type of lock which is used to provide concurrent access on LOBs. These new types of locks support access to large objects for reading and updating applications without interfering with each other. You can find a shared LOB lock (S-LOB lock) and an exclusive LOB lock (X-LOB lock) as new lock types for large objects in your system. There is no need for an update LOB lock, because updating a LOB value always means deleting and inserting a value, so that there will not be any direct updates on LOB data pages. A LOB lock always locks an individual LOB and it comes with a corresponding lock on the base table to ensure row consistency. Locking both tables at access time belongs to the storage-concepts how data of one logical row is stored in two different tables. A LOB lock only occurs as a column lock on a single value. There is no locking for untouched LOBs, because a data page cannot contain values of more than one single LOB value. But if you look at lock escalation mechanisms or if you explicitly specify LOCKSIZE TABLESPACE, you can find more locks than the number accessed by your application. Lock on LOB values are only escalated to the table space level, because there is only one table in each LOB table space which can be ignored for locking purposes. See Figure 4-6.

62

Large Objects with DB2 for z/OS and OS/390

LOB table space

LOB table space lock

escalation

Auxiliary

LOB

Table

lock

Figure 4-6 Lock escalation on LOBS

4.5.1 Locks with simple reads In this case LOBs differ from the objects in DB2. There is also a difference on how a LOB value is accessed, because the data describing the LOB is stored in a base table, while the LOB value itself is stored in an auxiliary table. When you select a LOB value into your host variable, you have to code your SQL statement on the base table. Accessing the auxiliary table via SQL is not supported. The first thing DB2 does is access the base table to find out which row in the auxiliary table it has to access. For this access to the base table, an intent-share (IS) lock is needed on the table space containing the base table, on the base table itself and a share (S) lock on the page accessed while retrieving or validating the rows of the base table. After the particular LOB is located, a IS-lock on the LOB table space and a single S-LOB lock is acquired on the LOB value to ensure row consistency. The lock is held by DB2 until the whole value is delivered to your host variable, so that no updating transactions can interfere with your application and the LOB will not disappear while your application is reading it. Figure 4-7 gives you an idea of how DB2 serializes locks when accessing a single LOB value. The numbers indicated in the figures in this section reflect the sequence in which the locks are taken. Using lock avoidance or uncommitted read does not lock the accessed row in the base table while you are retrieving the LOB value. In this case, the part of a ‘logical row’ (containing the entry in the base table and the associated LOB value) which resides in the base table can be updated by concurrent transactions while you retrieve the associated LOB value.

Chapter 4. Using LOBs

63

BASE Table Space: IS (1)

LOB Table Space: IS (4)

BASE Table: IS (2)

Page: S (3)

LO B: S (5)

Figure 4-7 Scan lock sequence

Note: This differs from the general DB2 lock mechanism, where no parts of a row can be updated while it is delivered to your application. Using lock avoidance or uncommitted read delivers the row as it was at the point in time when your SQL statement was issued.

4.5.2 Using ISOLATION (UR) for LOBs You can use ISOLATION (UR) for your application, but DB2 has to request an S-LOB lock to ensure data integrity while the application retrieves the data. But what is DB2 doing when I access my LOB data even with ISOLATION (UR)? First, when using uncommitted read, an S-lock is acquired on the base table to prevent you from being hit by mass delete statements. This type of lock is also referenced as a mass delete lock. The next lock to be requested is an intent-share (IS) lock on the LOB table space and an S-LOB lock on the LOB itself. An S-LOB lock has to be requested to protect the LOB value from updates or deletes while the application retrieves the value. As mentioned before, a LOB value can span pages and it takes time to retrieve it; during this time no other transaction is allowed to change the value. Using S-LOB locks, even while using uncommitted read, prevents LOB data from being corrupted by someone else during the delivery process to your application by the LOB Manager. Figure 4-8 shows LOB lock serialization when you use uncommitted read.

BASE Table Space:

LOB Table Space: IS (2)

BASE Table: S Mass-Delete (1)

LOB: S (3)

Figure 4-8 Scan lock sequence using uncommitted read

64

Large Objects with DB2 for z/OS and OS/390

4.5.3 Locks with INSERT Inserting a LOB from a locking point of view requires more locks than inserting data in a normal table, because there are two tables involved every time during an insert process. At first, an intent exclusive (IX) lock is taken on the base table space and the base table. The same type of lock is acquired for the LOB table space after establishing the first two locks. After the three IX locks are established, an exclusive (X) lock is taken on the data pages which are used for inserting the LOB value. The last lock is an X-lock on the page in the base table where the new row is stored. The reason why insert applications get an X-LOB lock is to prevent access by scanners before the LOB is completely inserted. The actions performed by DB2 when you insert a LOB value from the locking point of view are shown in Figure 4-9.

BASE Table Space: IX (1)

LOB Table Space: IX (3)

BASE Table: IX (2)

Page: X (5)

LOB: X (4)

Figure 4-9 Insert lock sequence

4.5.4 Locks with DELETE In this section we differentiate between the locking operations occurring with a singleton delete and a mass delete.

Deleting a single LOB value A LOB delete requires more locks than just those on the LOB table space, as you can see in Figure 4-10.

Chapter 4. Using LOBs

65

BASE Table Space: IX (1)

LOB Table Space: IS (4)

BASE Table: IX (2)

Page: X (3)

LOB: S (5)

Figure 4-10 Delete lock sequence

After establishing the usual IX locks on the base table space and the base table, only a intent-share (IS) lock is requested for the LOB table space. The page containing the associated data in the base table is also provided with an X-lock. There is no IX lock request for the LOB table space, because deleting a LOB value internally means de-allocating the allocated pages used by the LOB, and not directly updating the data pages. For this reason, the requested lock on the LOB is only a shared LOB (S-LOB) lock. If an S-LOB lock exists for a LOB value, the de-allocation is done after all other locks acquired by readers (see 4.5.1, “Locks with simple reads” on page 63 for locks requested by LOB readers) are released. So deleting applications acquire S-LOB-locks to reserve space in case of a rollback. Even if the deleting application commits the unit of work or its thread ends, the COMMIT is done and the LOB is only accessible for the thread currently reading the LOB value.

Comparison with a mass delete A mass delete also places the common IX-lock as seen in the examples above. For the base table, DB2 requires an X-lock and also a mass-delete X-lock on the base table. An X-lock for the LOB table space is acquired, too. See Figure 4-11 for a brief overview of locks taken at mass-delete time.

BASE Table Space: IX (1) BASE Table: IX (2) X (3) X mass delete (4)

Figure 4-11 Locks acquired by mass delete

66

Large Objects with DB2 for z/OS and OS/390

LOB Table Space: X (5)

4.5.5 Locks with UPDATE Discussing what happens when a LOB value is updated by an application program is quite easy after dealing with locks established at delete and insert times. A LOB update consists of a single LOB delete and another LOB insert, you can find both techniques only serialized when updating a LOB column. After DB2 takes both IX-locks on the base table space and the base table, it also requests an X-lock for a particular row in the base table. After all locks are done by the internal resource lock manager (IRLM), DB2 acquires an IS-lock on the LOB table space and an S-LOB lock to delete the old LOB. When the LOB is deleted, the new row is inserted after changing the IS-lock on the LOB table space to an IX-lock and an X-LOB lock is set for the new LOB value. You can find the sequence visualized in Figure 4-12.

BASE Table Space: IX (1)

LOB Table Space: IS (4), IX (6)

BASE Table: IX (2)

Page: X (3)

Old

New

LOB: S (5)

X (7)

LOB:

Figure 4-12 Lock sequence when updating a LOB column

4.5.6 Some general information about locking In this section we add some considerations on LOB locking.

Conflicts between SELECTers and DELETErs After you have read the previous sections, you can ask about concurrency of selecters and updaters. While readers and deleters both take S-LOB locks, they can coexist without interfering with each other. The pages where the LOB data resides are only de-allocated if there are no more S-LOB locks on the LOB, which simply means that nobody accesses the LOB at this particular point in time.

Shadow Copy Recovery When we talk about a shadow copy recovery, we talk about de-allocation and re-allocation of data pages in a LOB table space. When you delete a LOB value, the deletion de-allocates pages and logs only the de-allocation. The LOB data is left in the pages within the LOB table space as a shadow copy of the deleted data. A rollback simply results in re-allocation of the previously deleted pages. The S-LOB lock taken at delete prevents a later insertion from re-using the space de-allocated by the deletion. The S-LOB lock does not prevent deletions of that LOB but ensures that the space allocated to the LOB is not reused until the LOB lock is released. If the LOB is deleted, the shadow copy of that version of the LOB will persist until all LOB locks on that LOB version are released.

Chapter 4. Using LOBs

67

When no LOB locks are taken There are some situations where DB2 knows that it does not make sense to ask IRLM for a lock. There are at least four reasons in DB2 V7 when no LOB locks are acquired. 򐂰 Selecting a LOB value that is NULL or zero length 򐂰 Deleting a LOB value that is NULL or zero length 򐂰 Inserting a LOB value that is NULL or zero length 򐂰 Updating a LOB value that is NULL or zero length to zero length or NULL Changes on base table rows are logged as usual, only indicator column changes are logged with the base table log record.

4.6 Details about ROWID There are two different ways of defining a column to be a ROWID data type in a CREATE TABLE statement: 򐂰 COLNAME ROWID GENERATED ALWAYS 򐂰 COLNAME ROWID GENERATED BY DEFAULT Using the GENERATED ALWAYS keyword, DB2 always generates a ROWID when inserting a row. Applications and users are not allowed to insert a ROWID. If you use GENERATED BY DEFAULT, users and applications can supply a value for a ROWID column as long as the value was previously generated by DB2 and a unique, single column index that exists on the ROWID column. DB2 checks that the value you are going to insert is a valid ROWID. It is not sufficient to provide unique numbers yourself. You should only use this parameter when inserting data from another table for purposes of moving data. The recommended usage is GENERATED ALWAYS. As mentioned above, you have to create a unique index on the ROWID column when you specify GENERATED BY DEFAULT. Make sure that there is no way to use the GENERATED ALWAYS clause before implementing GENERATED BY DEFAULT, because the additional index on a table may increase your response time for inserting and deleting transactions on the base table. The index is not affected by an UPDATE statement since the ROWID is not updateable. If you try to update a ROWID column, DB2 issues SQLCODE -151, because the catalog description indicates that this column cannot be updated. Attention: When you specify GENERATED BY DEFAULT for a ROWID column, make sure that a single column unique index exists on your ROWID column. ROWID values can never contain null values, so the ROWID column has to be defined as NOT NULL. Be aware that a ROWID column implies some restrictions, preventing the values in the column from being manipulated: 򐂰 Users are not allowed to update a ROWID column. 򐂰 Null values can not be assigned to ROWID columns. 򐂰 EDITPROCs, FIELDPROCs and CHECK CONSTRAINTs are not provided for ROWIDs. 򐂰 It is not allowed to load a single partition or a range of partitions if a column of data type ROWID is part of the partitioning key.

68

Large Objects with DB2 for z/OS and OS/390

The ROWID column is stored like a VARCHAR (17) column. In DB2 V7 two different types of ROWIDs can be defined. To give you a better understanding of the different occurrences of a ROWID consider the following scenarios:

Case 1: A new table is created including a ROWID column First, we have a look at a table where the CREATE TABLE statement already contains a ROWID column (for all examples, it does not matter if the ROWID column is defined using GENERATED ALWAYS or GENERATED BY DEFAULT) as shown in Example 4-8. Example 4-8 DDL for a table containing a ROWID column CREATE TABLE CUSTOMER ( CUSTNO CHAR(10) , CUSTNAME CHAR(32) , ROW_ID ROWID

NOT NULL WITH DEFAULT NOT NULL WITH DEFAULT NOT NULL GENERATED ALWAYS );

Now you insert a row into the CUSTOMER table, as shown in Example 4-9, without providing the ROWID column, because it will be generated by DB2 at insert time. Example 4-9 Inserting a row in CUSTOMER table INSERT INTO CUSTOMER (CUSTNO, CUSTNAME) VALUES (‘0000000018’, ‘ANNIKA THOMAS’);

After a new table is created, all CUSTNOs being inserted are associated with a unique ROWID value. After you insert a row (at this point in time a ROWID is associated with the inserted row), a SELECT on ROWID returns the following result set, as shown in Example 4-10. Example 4-10 ROWID value of a table created with ROWID column CUSTNO CUSTNAME ROW_ID ---------+---------+---------+---------+---------+---------+---------+---------+ 0000000018 ANNIKA THOMAS 63C6AB6415CED248260401D370140100000000000201 ---------+---------+---------+---------+---------+---------+---------+---------+

As you can see, the generated ROWID is externalized as a 44-byte value, but stored as VARCHAR (17). When you use the DSN1PRNT utility, you can see the following HEX values for the above mentioned ROWID shown in Example 4-11: Example 4-11 DSN1PRNT of ROWID in hex value 000E 63C6AB64 15CED248 260401D3 7014

The value ‘000E’ declares the length of the ROWID column, which is currently 000E in hex and 14 in decimal. Comparing to the information stored in the DB2 catalog that ROWIDs are stored as VARCHAR (17) columns, there are three bytes left for future extensions of ROWIDs.

Case 2: Adding a ROWID column via ALTER TABLE Things look different if a ROWID column is added when a table already contains many rows. Consider the table mentioned in Example 4-8 without a ROWID column and the table already contains lots of rows. Let us issue an ALTER TABLE statement as shown in Example 4-12.

Chapter 4. Using LOBs

69

Example 4-12 ALTER TABLE adding a ROWID column ALTER TABLE EXAMPLE ADD ROW_ID ROWID NOT NULL GENERATED ALWAYS;

Note: A ROWID column must be defined using NOT NULL. When you add a ROWID column, the NOT NULL attribute contradicts the normal usage of ALTER. In fact when ALTERing non-ROWID columns, you must specify NOT NULL WITH DEFAULT. The ALTER TABLE statement does not affect any columns stored in the table, so no ROWID is stored up to now. If you do a SELECT on ROWID, you will receive a result set like the one in Example 4-13. Example 4-13 ROWID value of a table where a ROWID column was added CUSTNO CUSTNAME ROW_ID ---------+---------+---------+---------+---------+---------+---------+ 0000000018 ANNIKA THOMAS 40000000800001300022020100000000000201 ---------+---------+---------+---------+---------+---------+---------+

As you can see, the ROWID now only consists of 38 bytes, compared to the previously mentioned ROWID of 44 bytes. If you use DSN1PRNT again to look into your table space, you will not find any value for the ROWID column inside your table space. When you select the ROWID value, it is only generated at SELECT time, and stored inside your host variable. You can select the ROWID for a specific row several times, and the value in the ROWID column never changes. The first time you update the row, the ROWID is physically stored in the table, with the same value delivered to you by DB2 when you selected the row before. When you use DSN1PRNT again after you performed an UPDATE on the row, you will see results similar to Example 4-14. Example 4-14 DSN1PRNT of ROWID in hex value after adding and updating a row 000B 40000000 80000130 002202

Now the value ‘000B’ declares the length of the ROWID column, which means 11 in a decimal value. Adding the two bytes length field, as usual, we now have a ROWID column made of 13 bytes. So if an already existing row is updated and no ROWID value is stored for the updated row up to now, the new ROWID is at least 11 bytes plus the two bytes length field. Let us now have a close look at the new rows which are inserted after the ROWID column was added to the table. All rows inserted after the ROWID column was added to the table have a ROWID of length X’000E’. This means 14 bytes plus two additional bytes for the length field. The first row shown in Example 4-15 represents the ROWID value for an ‘old’ row being updated after ROWID was added to the table. It is important to know that this row already had been in the table before the ROWID column was added. The second row shows the ROWID value for a new row inserted after the ROWID column was added. Example 4-15 ROWID values of updated and inserted columns CUSTNO CUSTNAME ROW_ID ---------+---------+---------+---------+---------+---------+---------+---------+

70

Large Objects with DB2 for z/OS and OS/390

0000000018 MRS. THOMAS 40000000800001300022020100000000000201 0000000026 MR. KLINGEN 3F0DA87C95CED612260401D370180100000000000202 ---------+---------+---------+---------+---------+---------+---------+---------+

In Example 4-16 you can find the DSN1PRNT output for both ROWID values: Example 4-16 DSN1PRNT of updated and inserted ROWIDs in hex value 000B 40000000 80000130 002202 000E 3F0DA87C 95CED612 260401D3 7018

As indicated above, the pre-existing row has a ROWID length of ‘000B’ which is 11 bytes, the newly inserted row uses 14 bytes (‘000E’ as length field) to store the ROWID. In conclusion, it is possible that no ROWID value is stored in the table space even if a ROWID column exists in the table, and you can find two different occurrences of ROWIDs even if they are stored in the table, depending on when the ROWID column was added to the table and whether or not the row existed at that time. ROWID is a data type that can be used outside LOBs. ROWID gives you guaranteed unique values. This can be useful for applications and table designs where artificial keys have to be generated by the application to ensure uniqueness of a particular row. Using a ROWID column, DB2 is able to handle this special requirement for you. The ROWID behavior, that the first bytes in general appear to be pseudo-random, makes a ROWID column a good choice for a partitioning key if you want to spread your data randomly across partitions. Note: Using a ROWID as a partitioning key is not a good idea when you have to process your data in the order of another key value!

Chapter 4. Using LOBs

71

72

Large Objects with DB2 for z/OS and OS/390

5

Chapter 5.

DB2 Extenders and LOBs In this chapter we provide a brief description of DB2 Extenders available for the z/OS environment. We then look at the set up and administration of these extenders with regard to LOB utilization. The chapter is structured as follows: 򐂰 򐂰 򐂰 򐂰

Generalities AVI Extenders XML Extenders XML and LOBs

© Copyright IBM Corp. 2002

73

5.1 Generalities DB2 for OS/390 has introduced extenders to enhance the usability and functionality of the RDMS. They can be divided into text and AVI extenders for text, audio, video, image, and XML extenders for XML. These extenders define new data types and functions using DB2 UDB for OS/390’s built-in support for user defined functions (UDF) and user distinct types (UDT). They handle new non traditional data types in advanced applications, improving application development and reducing complexity. Figure 5-1 shows the general DB2 Extender architecture.

A rchitectural view of D B 2 E xten ders Application s App lications

SQ S QL

Stream ed Data D ata

DB 2 Client Fu nctio n

D B2 C lient/C AE E xten ded xtend ed SQ S QL AP I C AP I

D B 2 U DB D B Server

Static D ata

Stored Proc.

UDT

U DF

B usin ess Bu siness D ata

S ervers Externally S tored M M D ata

Static D ata

B LO B

Bu siness Business Data D ata

MM Attributes D ata

S earch Support Data

Internally Stored M M Data

Figure 5-1 Extender architecture

5.2 AVI Extenders The DB2 Audio, Video, and Image Extenders enable you to store image, audio, and video files in database tables alongside other traditional character and numeric data that are relevant to your data model. These extenders provide additional data types specifically for large image audio, and video file types and specialized query functions for altering, analyzing, and retrieving them.

74

Large Objects with DB2 for z/OS and OS/390

Features of the Image Extender 򐂰 Import and export images and their attributes into and out of a database. When you import an image, the DB2 Image Extender stores and maintains image attributes such as size in bytes, format, height, width, and number of colors. 򐂰 Controls access to images with the same level of protection as traditional business data. 򐂰 Convert the format of images. You have the flexibility of importing or exporting an image in its source format or converting the format of the image when importing or exporting. You can also scale an image, rotate it, do black-white image inversion, or change representational characteristics, such as bits per sample and compression type. 򐂰 Secure and recover images. Images and their attributes that you store in a DB2 UDB database have the same security and recovery protection as traditional business data. 򐂰 Query images based on related business data or by image attributes. You can search for images based on data that you maintain, such as a name, number, or description; or by data that the DB2 Image Extender maintains, such as the format of the image or its distribution of colors. Now you can create a QBIC query easier than ever before. 򐂰 Generate and display image thumbnails and full images. A thumbnail is a miniature version of an image. When you import an image into a database, the DB2 Image Extender creates and stores a thumbnail of the image. You can use the DB2 Image Extender to retrieve a thumbnail or a full-size image. You can then use the DB2 Image Extender to invoke your favorite browser to display the thumbnail or full-size image. 򐂰 The DB2 Image Extender supports a wide variety of image formats, such as GIF, JPEG, BMP, and TIFF.

Features of the Audio Extender 򐂰 Import and export audio clips and their attributes into and out of a database. When you import an audio clip, the DB2 Audio Extender stores and maintains audio attributes such as number of audio channels, transfer time, and sampling rate. 򐂰 Secure and recover audio data. Audio clips and their attributes that you store in a DB2 database are afforded the same security and recovery protection as traditional business data. 򐂰 Query audio clips based on related business data or by audio attributes. You can search for audio clips based on data that you maintain, such as a name, number, or description; or by data that the DB2 Audio Extender maintains, such as the format of the audio or the date and time that it was last updated. 򐂰 Play audio clips. You can use the DB2 Audio Extender to retrieve an audio clip. You can then use the DB2 Audio Extender to invoke your favorite audio browser to play the audio clip. 򐂰 The DB2 Audio Extender supports a variety of audio file formats, such as WAVE and MIDI, and can work with different file-based audio servers.

Features of the Video Extender 򐂰 Import and export video clips and their attributes into and out of a database. When you import a video clip, the DB2 Video Extender stores and maintains video attributes such as frame rate, compression format, and number of video tracks. 򐂰 Secure and recover video data. Video clips and their attributes that you store in a DB2 database are afforded the same security and recovery protection as traditional business data. 򐂰 Query video clips based on related business data or by audio attributes. You can search for video clips based on data that you maintain, such as a name, number, or description;

Chapter 5. DB2 Extenders and LOBs

75

or by data that the DB2 Video Extender maintains, such as the format of the video or the date and time that it was last updated. 򐂰 Play video clips. You can use the DB2 Video Extender to retrieve a video clip. You can then use the DB2 Video Extender to invoke your favorite video browser to play the video clip. 򐂰 The DB2 Video Extender supports a variety of video file formats, and can work with different file-based video servers. The installation of the audio, video, image extenders involves some effort in configuring the environment, both on the desktop, and the OS/390 platform.

5.2.1 Desktop The desktop configuration includes the use of DB2 connect either Personal or Enterprise edition, and the installation of the AVI extenders. After the installation, the environmental variables must be set up through Win NT Settings-> Control Panel-> Environment Tab to define the path which contains the multimedia files.This path is used by the extender to retrieve and store multimedia files.

5.2.2 Database On the mainframe, the configuration includes building a database MMDBSYS for holding the administration tables for use by the extenders. Then binding the plans and packages used by the extenders. After installing the extenders, you must bind the packages through DB2 connect issuing the following command from the CLP: bind path/dmbmvsb3.lst grant public isolation cs

The table spaces used for storing the LOB data should be defined in the MMDBSYS database. Important: PTF UQ63855 for APAR PQ54511 resolves a problem on ENABLE COLUMN encountered going through the DB2CLP Extender accessing from the workstation.

5.2.3 WLM environment You must add the steplib for the extender object modules to the WLM started tasks, along with creating a DMBENVAR DD card pointing to the USS path for the DB2 extenders created during the DB2 install. Example 5-1 shows the sample DMBWLM1 procedure. Example 5-1 Sample DMBWLM1 procedure //V71AWLM1 PROC DB2SSN=V71A,NUMTCB=18,APPLENV=DMBWLM1 //TCBNUM1 EXEC PGM=DSNX9WLM,TIME=1440, // PARM='&DB2SSN,&NUMTCB,&APPLENV',REGION=0M //STEPLIB DD DSN=DSN710.SDSNLOAD,DISP=SHR <== DB2 Load Library // DD DSN=DMB.V71.LOADLIB,DISP=SHR <== DB2EXT Load library //CEEDUMP DD SYSOUT=H //SYSPRINT DD SYSOUT=H //SYSDUMP DD SYSOUT=H //DMBENVAR DD DISP=SHR,DSN=&HQL.DMB.ENVAR <= DB2EXT ENV Variable File //*DSNAOTRC DD DISP=SHR,DSN=&HQL.DMB.DSNAOTRC <= DB2 CLI Trace

76

Large Objects with DB2 for z/OS and OS/390

The following example is a sample of the contents of global environment variables defined in file DMB.ENVAR: Example 5-2 DMBENVAR DD card input DB2MMPATH=/usr/lpp/db2ext_07_01_00/samples:/tmp DB2MMTEMP=/tmp DB2MMSTORE=/tmp DB2MMEXPORT=/tmp

5.2.4 Administration tasks to enable a table to use extenders The following list is an ordered summary of the administration tasks you perform when you use the extenders the first time. You use DB2 commands or statements to perform some tasks. You perform other tasks with the DB2 extenders. This sequence assumes that your DB2 system is running: 1. Connect to the database server using the Extender CLP 2. Enable the database server using the Extender CLP 3. Create a table and column using DB2 4. Enable a table in the database using the Extender CLP 5. Enable a column in the table using the Extender CLP

ENABLE SERVER command By issuing the ENABLE SERVER command, through the DB2EXT.CLP, necessary UDTs are created for use by the extender functions, such as DB2AUDIO, DB2VIDEO, and DB2IMAGE. An example of the ENABLE SERVER command is listed in Example 5-3. Example 5-3 Example of ENABLE SERVER command ENABLE SERVER FOR (DB2IMAGE,DB2AUDIO,DB2VIDEO) Using tablespace-name WLM ENVIRONMENT XXXXXXXX external security (USER,DB2)

Table definitions include these UDT's as the column characteristics. Table 5-1 shows the UDT’s used with the AVI extenders. The DB2 Image and Audio Extenders define user-defined types (UDTs), or distinct types, and user-defined functions (UDFs) for storing and manipulating image and audio files. By using the DB2 support for large object types (LOBs), the extenders define the distinct data types DB2IMAGE, DB2AUDIO,and DB2VIDEO. For each of these types, the actual contents of the image, audio, or video file that they encapsulate can be stored as a binary large object (BLOB) in the database table, or outside of the database in a file system. In both cases, the contents of these distinct types are stored in an administrative support table that is maintained by the extenders. Only a character string called a handle that represents the type is actually stored in the user table. If the file is stored as a BLOB, it is stored in a column of an administrative support table. If the file is stored outside of the database, a file identifier (or pointer) to that file on the file system is stored in an administrative support table.

Chapter 5. DB2 Extenders and LOBs

77

Table 5-1 UDTs used with AVI Extenders User Defined Types created by DB2 AVI extenders UDT

Source Type

Description

DB2IMAGE

Varchar(250)

Image handle — A variable length string that contains information needed to access an image object. Image handles are stored in a user table column enabled for the Image Extender.

DB2AUDIO

Varchar(250)

Audio handle — A variable length string that contains information needed to access an audio object. Audio handles are stored in a user table column enabled for the Audio Extender.

DB2VIDEO

Varchar(250)

Video handle — A variable length string that contains information needed to access a video object. Video handles are stored in a user table column enabled for the Video Extender.

In response to the above ENABLE SERVER command, the Image Extender: 򐂰 򐂰 򐂰 򐂰 򐂰

Creates a user-defined type that is named DB2IMAGE for image objects Creates a user-defined type that is named DB2AUDIO for audio files Creates a user-defined type that is name DB2VIDEO for video clips Creates administrative support tables for image,audio, and video objects Creates user-defined functions for multi media objects

The UDFs will run in the MVS Workload Manager (WLM) environment that is named DMBWLM1. The tablespace name is the name of a table space used to store support tables and their indexes. It must be defined in the MMDBSYS database. The external security parameter is either: 򐂰 USER: This allows each UDF to execute as if the primary authid is the owner of the function. Or, 򐂰 DB2: This uses the internal DB2 security and it is the default. To enable tables to use the extender functions, the UDFs, UDTs, and TRIGGERS need to be created to administer the use of LOB columns in tables. The extenders provide UDFs for storing these media data types. Through parameters to the supplied functions, you specify the content of the media file, its file format, and whether to store the file as a BLOB in the database or outside the database in the file system. The content of a file can be a buffer or file located on the client machine from where the function is called or a file on the file system where the database server resides.

78

Large Objects with DB2 for z/OS and OS/390

Creating a table using the DB2IMAGE UDT Example 5-4 shows how to define a table using the DB2IMAGE UDT. Example 5-4 Defining the table for an image CREATE TABLE employee (id CHAR(6) name VARCHAR(40) picture DB2IMAGE)

/*name of the table*/ /*employee identification*/ /*employee name*/ /*employee picture*/

ENABLE TABLE command After the server has been enabled, and a table created using the appropriate UDT, the enable table command must be issued from the client DB2EXT.CLP to allow the table to store the media in a specific table space. The enable table statement is shown in Example 5-5. Example 5-5 Example of ENABLE TABLE command enable table table_name for extender_name using tablespace_name,,tablespace_name

In response to the ENABLE TABLE command, the Image Extender: 򐂰 Identifies the table for use. 򐂰 Creates administrative support tables that hold attribute information for image objects in enabled columns. The administrative support tables are stored in a table space that is named in the enable table statement and defined in the MMDBSYS Database. A 32 KB bufferpool is necessary for this table space. 򐂰 Creates an auxiliary table to hold LOB data for enabled columns. The auxiliary table is stored in a LOB table space that is named as the second table space in the enable table statement and is defined in the MMDBSYS Database. 򐂰 Creates indexes for the administrative support tables and the auxiliary LOB table.

ENABLE COLUMN command After the server and table have been enabled to allow for the storage of the correct type of media, the table column is enabled to allow for media storage. The enable column statement is shown in Example 5-6. Example 5-6 Example Enable Column Command enable column table_name col_name for extender_name

In response to the ENABLE COLUMN command, the Extender: 򐂰 Identifies the column for use. 򐂰 Creates triggers. These triggers update various administrative support tables in response to insert, update, and delete operations on the table.

Inserting LOB data A program calling the appropriate UDF is used to insert data into either the LOB columns defined in the database, or to point to a file on the file system.

Chapter 5. DB2 Extenders and LOBs

79

Example 5-7 shows the SQL statements in a C application program, which request an Image Extender UDF named DB2Image to store an image in a database table; the content of the source image is in a server file. It is accessed within the UDF by using a file reference variable. See A.3, “Using file reference variables” on page 144. Example 5-7 Storing an image EXEC SQL BEGIN DECLARE SECTION; long hvStorageType; EXEC SQL END DECLARE SECTION; hvStorageType=MMDB_STORAGE_TYPE_INTERNAL EXEC SQL INSERT INTO EMPLOYEE VALUES( '128557', 'Anita Jones', DB2IMAGE( CURRENT SERVER, '/employee/images/ajones.bmp', 'ASIS', :hvStorageType, 'Anita''s picture') );

/*id*/ /*name*/ /*Image Extender UDF*/ /*database server*/ /*image content*/ file reference /*keep the image format*/ /*store image in DB as BLOB* /*comment*/

When insert, update, and delete operations are performed on the tables defined within the extender, triggers update various administrative support tables in response. These support tables contain additional information about the extenders. Some of the tables identify user tables and columns that are enabled for the Extender. Other support tables contain attribute information about objects in enabled columns. You use extender application programming interfaces to display images and play audio or video objects. You code these APIs using client function calls in C. The functions are run in the client. Example 5-8 shows C statements which include an API that is named DBiBrowse. The API retrieves the data for an image handle and starts a browser to display the image. Example 5-8 Browsing an image EXEC SQL BEGIN DECLARE SECTION; char hvImg_hdl[251]; EXEC SQL END DECLARE SECTION EXEC SQL SELECT PICTURE INTO :hvImg_hdl WHERE NAME= 'Roger Rabbit'; rc=DBiBrowse( "ib %s", MMDB_PLAY_HANDLE, hvImg_hdl, MMDB_PLAY_NO_WAIT);

/*image browser*/ /*use image handle*/ /*image handle*/ /*run browser independently*/

If the actual files are stored as BLOBs, they are provided the same security, concurrency, and integrity constraints as any other type of data inside the control of DB2. Users must have the required privilege to select, insert, or update the contents or the files, as well as the additional metadata that the extenders store about the image or audio data. These files and their metadata also can be backed up and recovered in the same way as any other data in DB2. For more information on the AVI extenders go to: http://www.ibm.com/software/data/db2/extenders/imgfmt.htm

80

Large Objects with DB2 for z/OS and OS/390

5.3 XML Extenders The XML Extender is a new feature of DB2 and a successful implementation requires careful planning. The XML Extender provides the following user-defined types (UDT) for use with XML columns: 򐂰 XMLVarchar 򐂰 XMLCLOB 򐂰 XMLFILE Table 5-2 shows the default UDTs and the length assignment after enabling the XML SERVER. If the CLOB length must be expanded, you need to drop each stored procedure, and recreate them with an increased CLOB limit. Table 5-2 DEFAULT UDTs after enabling the XML SERVER Schema

Data TYPE NAME

SCHEME

SOURCE DATA TYPE

LENGTH

DB2XML

XMLCLOB

SYSIBM

CLOB

2147483647

DB2XML

XMLVARCHAR

SYSIBM

VARCHAR

3000

Db2XML

XMLFILE

SYSIBM

VARCHAR

512

All the XML Extender facilities supplied for application programs run in the OS/390 MVS environment as stored procedures or user-defined functions (UDFs). User-defined types (UDT) are data types created by a DB2 application or tool. These data types are used to identify the storage type of XML documents in the application table. You can also store XML documents as files on a local file system specifying a file name. All the XML Extender's user-defined types have the qualifier DB2XML, which is the schema name of the DB2 XML Extender user-defined types. For example:

db2xml.XMLVarchar The XML Extender provides storage functions to be used with the XML Column storage type. Table 5-3 shows the storage functions and the base file type associated with each. Table 5-3 XML Extender storage functions The XML Extender storage functions Basetype

Stored in DB2 as: XMLVCHAR

XMLCLOB

XMLFILE

Varchar

XMLVCAHR

N/A

XMLFileFromVarchar()

CLOB

N/A

XMLCLOB

XMLFileFromCLOB()

File

XMLVarcharFromFile()

XMLCLOBFromFile()

XMLFILE

The DB2 XML Extender provides powerful UDFs to store and retrieve XML documents in XML columns, as well as to extract XML element or attribute values. A UDF is a function that is defined to the database management system and can be referenced thereafter in SQL statements. The XML Extender provides the following types of UDFs: 򐂰 Storage It stores intact XML documents in XML-enabled columns at XML data types. Chapter 5. DB2 Extenders and LOBs

81

򐂰 Extract It extracts XML documents, or the values specified for elements and attributes as base data types. 򐂰 Update It updates entire XML documents or specified element and attribute values. Some of the UDFs that refer to the XMLFILE data type, require access to an HFS system. The DB2 XML trace file, is also written to an HFS file. Table 5-4 identifies the XML Extender storage UDFs, and their functions. Table 5-4 Storage UDFs used with XML Extenders The XML Extender storage UDFs Storage user defined functions

Return type

Function

XMLVarcharFromFile()

XMLVARCHAR

Reads an XML document from a file on the server and returns the value of the XMLVARCHAR type.

XMLCLOBFromFile()

XMLCLOB

Reads an XML document from a file on the server and returns the value of the XMLCLOB type.

XMLFileFromVarchar()

XMLFILE

Reads an XML document from memory as VARCHAR, writes it to an external file, and returns the value of the XMLFILE type, which is the file name.

XMLFileFromCLOB()

XMLFILE

Reads an XML document from memory as CLOB or a CLOB locator, writes it to an external file, and returns the value of the XMLFILE type, which is the file name.

Table 5-5 identifies the XML Extender extract UDFs, their functions, and returned values. Table 5-5 Extract UDFs used with XML Extenders XML Extender extract UDFs

82

TABLE FUNCTION

RETURNED COLUMN

RETURNED TYPE

extractinteger

returnedInteger

INTEGER

extractSmallint

returnedSmallint

SMALLINT

extractDouble

returnedDouble

DOUBLE

extractReal

returnedReal

REAL

extractChar

returnedChar

CHAR

extractVarchar

returnedVarchar

VARCHAR

extractClob

returnedClob

CLOB

Large Objects with DB2 for z/OS and OS/390

XML Extender extract UDFs extractDate

returnedDate

DATE

extractTime

returnedTime

TIME

extractTimestamp

returnedTimestamp

TIMESTAMP

5.3.1 XML Toolkit The XML Extender requires IBM XML Toolkit for OS/390 (FMID HXML120) 5655-D44, a no-charge product. Some functions also require UNIX Systems Services. XML Toolkit for OS/390 is available on tape, or via a download from the Internet, to your PC, then uploaded to the host. The following files constitute the package, although SMP/E knowledge is required especially with regards the SMP/E GIMZIP function: 򐂰 XML Toolkit 1.2.0 for z/OS and OS/390 README file in text format: XMLSMPE.README.txt (21.4KB) 򐂰 XML Toolkit 1.2.0 for z/OS and OS/390 package in pax format: XMLSMPE.pax.Z (41.4MB) 򐂰 XML Parser 1.2.0 for z/OS and OS/390, C++ Edition: XMC331B.tar (16.5MB) 򐂰 XML Parser 1.2.0 for z/OS and OS/390, Java Edition: XMJ311B.tar (3.8MB)

5.3.2 WLM application environment You need to set up a WLM Application Environment, running WLM in goal mode and build the procedure reported in Example 5-9. The key point is to include the XML Toolkit library in the STEPLIB. Example 5-9 WLM procedure //ISC3XML PROC RGN=0K,APPLENV=ISC3XML,DB2SSN=ISC3,NUMTCB=8 //IEFPROC EXEC PGM=DSNX9WLM,REGION=&RGN,TIME=NOLIMIT, // PARM='&DB2SSN,&NUMTCB,&APPLENV' //STEPLIB DD DISP=SHR,DSN=ISC710P1.RUNLIB.LOAD // DD DISP=SHR,DSN=SYS2.DB2.V710.SDSNLOAD // DD DISP=SHR,DSN=SYS2.DB2.V710.SDXXLOAD // DD DISP=SHR,DSN=SYS1.SIXMMOD1 <===================XML Toolkit Library //SYSIN DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSPRINT DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND) //SYSTSPRT DD UNIT=SYSDA,SPACE=(4000,(20,20),,,ROUND)

5.3.3 General host requirements DB2 V7 (HDB7710) and the XML Extender (JDB771X, Compid 5740XYR06) must both be installed in the same CSI. RRS is required for XML Extender stored procedures and UDFs that run in WLM managed address spaces.

Chapter 5. DB2 Extenders and LOBs

83

The Odb2 command line tool is required to allow SQL from the UNIX Shell. This can be downloaded and installed from the following Web site, and then select Tools and Toys. http://www.s390.ibm.com/products/oe/

The PTF UQ52836 is highlighted as a requirement. It upgrades XML/390 beta code to GA. This is a must requirement.

5.3.4 Workstation requirements Install the XML Extender administration Wizard from: http://www.ibm.com/software/data/db2/extenders/xmlext/downloads.html

The Administration Wizard also requires the following software on the client desktop: 򐂰 DB2 UDB Connect Personal or Enterprise Edition 򐂰 JAVA Development Kit 1.1.7 or higher The initial environment for XML Extenders is now complete for OS/390 or z/OS.

5.4 XML and LOBs DB2's XML Extender provides the ability to store and access XML documents, to generate XML documents from existing relational data, and to insert rows into relational tables from XML documents. XML Extender provides new data types, functions, and stored procedures to manage your XML data in DB2. With the content of your structured XML documents in a DB2 database, you can combine structured XML information with traditional relational data. Based on the application, you can choose whether to store entire XML documents in DB2 as in user-defined types provided for XML data (XML data types), or you can map the XML content as base data types in relational tables. For XML data types, the XML Extender adds the power to search rich data types of XML element or attribute values, in addition to the structural text search that the DB2 Text Extender for OS/390 provides.

What XML Extender can do for your applications Store entire XML documents as column data or externally as a file, while extracting desired XML element or attribute values and storing it in side tables, which are indexed subtables used for high-speed searching. By understanding the elements and attributes most frequently searched, you can set up your side table structures. When planning for your sidetables, you must consider how many tables to create, how to organize them, and whether or not to create a default view for the tables. These decisions are partly based on whether or not you have multiple occurring elements or attributes. These sidetables are a subset of the existing tables, containing the same columns as those elements or attributes defined in the DAD. Side tables are DB2 tables containing only a subset of the original tables, used to extract the content of an XML document that will be searched frequently. The location path of the element or attribute is mapped to a table and column, indexed, and used for searches. When the XML document is updated in the application table, the values in the side tables are automatically updated. Figure 5-2 depicts the relationship between the XML document stored in the XML COLUMN and the side tables created to improve search performance.

84

Large Objects with DB2 for z/OS and OS/390

Figure 5-2 XML columns with side tables

For elements or attributes in an XML document that have multiple occurrences, you must create a separate side table for each XML element or attribute with multiple occurrences, due to the complex structure of XML documents. This means that elements or attributes that have multiple occurring location paths must be mapped to a table with only one column, the column for that location path. You cannot have any other columns in the table, whether or not they have multiple occurrences

Default views and query performance When you enable an XML column, you can specify a default, read-only view that joins the application table with the side tables using a unique ID, called the ROOT ID. With the default view, you can search XML documents by querying the side tables. The advantage of querying the default view is that it provides a virtual single view of the application table and side tables. However, the more side tables are created, the more expensive the query. Therefore, creating the default view is only recommended when the total number of side-table columns is small. Applications can create their own views, joining the important side table columns. By storing the documents as column data, you can: 򐂰 Perform fast search on XML elements or attributes that have been extracted and stored in side tables as SQL basic data types, and then create indexes on them. 򐂰 Update the content of an XML element or the value of an XML attribute 򐂰 Extract XML elements or attributes dynamically using SQL queries 򐂰 Validate XML documents during insertion and update

Chapter 5. DB2 Extenders and LOBs

85

򐂰 Perform structural-text search with the Text Extender

How XML and DB2 work together XML Extender provides the following features to help you manage and exploit XML data with DB2: 򐂰 Administration tools to help you manage the integration of XML data in relational tables 򐂰 Storage and access methods for XML data within the database The XML Extender provides three UDTs: XMLVARCHAR, XMLCLOB, and XMLFILE. XMLVARCHAR and XMLCLOB stores the XML document as a VARCHAR and CLOB respectively, while XMLFILE stores the XML document as a file on a local file system. The XML Extender allows you to store DTDs, the set of declarations for XML elements and attributes. When a database server is enabled for XML, a DTD repository table (DTD_REF) is created. Each row of this table represents a DTD with additional metadata information. Users can access this table to insert their own DTDs. The DTDs are used for validating the structure of XML documents. You specify how structured XML documents are to be processed by the XML EXTENDER using a document access definition (DAD) file. The DAD file is an XML-formatted document that maps the XML document structure to a DB2 table. The DAD file specifies whether you are storing documents using the XML COLUMN method, or defining an XML COLLECTION for composition or decomposition. These methods have very different uses, but can be used in the same application. Storing whole XML documents as column data is known as the XML COLUMN method of storage and access. Composing or decomposing the contents of XML documents with one or more relational tables, uses the XML COLLECTION method of storage and access

XML column This method helps you store intact XML documents in DB2. The XML column method works well for archiving documents. The documents are inserted into columns that are enabled for XML and can be updated, retrieved, and searched. Element and attribute data can be mapped to DB2 tables (side tables), which can be indexed for fast search. Figure 5-3 shows how DB2 can store a complete, tagged document in a CLOB column of a table using the XMLCLOB data type.

86

Large Objects with DB2 for z/OS and OS/390

Figure 5-3 XML column

In Example 5-10 below, a small C program segment with embedded SQL (SQL statements coded within an application program) illustrates how an XML document is retrieved from a file to memory. Example 5-10 Retrieval of an XML column document This example assumes that the column ORDER is of the XMLFILE type. EXEC SQL BEGIN DECLARE SECTION; SQL TYPE IS CLOB_LOCATOR xml_buff; EXEC SQL END DECLARE SECTION; EXEC SQL CONNECT TO SALES_DB EXEC SQL DECLARE c1 CURSOR FOR SELECT Content(order) from sales_tab EXEC SQL OPEN c1; do { EXEC SQL FETCH c1 INTO :xml_buff; if (SQLCODE != 0) { break; } else { /* do whatever you need to do with the XML doc in buffer */ } } EXEC SQL CLOSE c1; EXEC SQL CONNECT RESET;

Chapter 5. DB2 Extenders and LOBs

87

XML collection This method allows for the storing of untagged XML data in one or more database tables. XML data is either decomposed from incoming XML documents or used to compose outgoing XML documents. If your data is to be shared with other applications, you may want to be able to compose and decompose incoming and outgoing XML documents and manage the data as necessary to take advantage of the relational capabilities of DB2. This type of XML document storage is called XML collection. Figure 5-4 shows how DB2 stores data in the XML Collection which stores separate elements and attributes in different tables used in composing or decomposing documents. .

Figure 5-4 XML collection

DTD The Document Type Definition (DTD) lets us define the different pieces of data we plan to model, along with the relationships between them. The ability to include this semantic information is the source of XML's power, and its main advantage over HTML. For an example of how to create a DTD, see: http://www.ibm.com/developerworks/library/buildappl/writedtd.html#h2

DAD One of the inputs into both storage and retrieval is the user specified mapping file that creates the relationship between relational data and the XML document structure. This mapping file is called a document access definition (DAD) and provides a way to produce an XML document with the XML attributes and elements that you name as you please, and in any shape you want.

88

Large Objects with DB2 for z/OS and OS/390

For an example of how to construct DADs and an explanation of their uses, see: http://www.ibm.com/software/data/pubs/papers/db2webservices/db2webservices.pdf

UDTs The XML Extender provides XML user defined types in which you define a column to hold XML documents. Table 5-6 defines these UDTs. Table 5-6 User defined types Use-defined type column

Source data type

Usage description

XMLVARCHAR

VARCHAR(varchar_len)

Stores an entire XML document as VARCHAR inside DB2. Used for small documents stored in DB2.

XMLCLOB

CLOB(clob_len)

Stores an entire XML document as CLOB inside DB2. Used for large documents stored in DB2.

XMLFILE

VARCHAR(1024)

Stores the file name of an XML document in DB2, and stores the XML document in a file local to the DB2 server. Used for documents stored outside DB2.

UDFs Here are a list of the UDFs in use by the XML extender for the manipulation of XML data within the RDMS. 򐂰 Storage It stores intact XML documents in XML-enabled columns at XML data types. 򐂰 Extract It extracts XML documents, or the values specified for elements and attributes as base data types. 򐂰 Retrieval It retrieves data to either a CLOB stored in a database or to an external server file. 򐂰 Update It updates the entire XML document or specified element and attribute values.

Security XML Documents stored as columns in DB2 tables are afforded the same security as any other user table column. When an administrator enables a database server for use with the XML Extender, he can specify which type of security to use. Either SECURITY USER, which assigns the primary ID of the process which calls the function to the XML UDF execution environment, or SECURITY DB2 which uses the ID of the WLM to access the XML UDF execution environment.

Chapter 5. DB2 Extenders and LOBs

89

When a UDF attempts to access a file, USS calls an external security package to get the USERID associated with the UDF. For SECURITY USER, the file system checks against the USERID of the process that calls the UDF. Because you can assign users to groups and assign different authorities to different groups, you can control access to files on a group-by-group level. SECURITY DB2 requires less administration in that the you need to assign an authorization ID, USERID, and Group ID to the WLM address spaces which run the UDFs. When you use the SECURITY USER you must assign a USERID and Group ID for every legitimate user of the files. In both cases you must coordinate the file system permissions with the Group and USERIDs. Some XML Extender administrative functions require special authority.

Administrative functions The ENABLE or DISABLE SERVER command should be executed by the DB2XML userid which has SYSADM or DBADM. When issuing the ENABLE or DISABLE COLUMN commands the administrator must have table owner privileges on the table which contains the column to be enabled. The administrator also has to have privileges for table spaces and buffer pools. When using the ENABLE COLLECTION administrative command, the administrator must have table owner privileges on all tables in the collection and use of buffer pools and table spaces.

Composition and decomposition of documents An application developer needs access to the following tables: 򐂰 Composition – – – –

DTD_REF (to access the DTD) XML_USAGE (to access the DAD files) SELECT on all tables referenced INSERT on the result table

򐂰 Decomposition – – – – – –

DTD_REF (to access the DTD) XML_USAGE (to access the DAD files) INSERT on all tables to be modified UPDATE on all tables to be modified SELECT on any table referenced in the DAD file Access to the DB2 catalog

In conclusion, DB2 extenders provide the functionality to make LOBs useful within your application. By using the Extender APIs, it makes application development using these multimedia files easier and more structured.

90

Large Objects with DB2 for z/OS and OS/390

6

Chapter 6.

Data administration with LOBs In this chapter we document the differences in administrating table spaces pertaining to tables containing LOB columns. We go through the various DB2 utilities and document their changes because of the introduction of LOB columns. The chapter is structured as follows: 򐂰 򐂰 򐂰 򐂰

The DB2 Catalog and LOBs Utilities Recovery scenario Operational scenario

© Copyright IBM Corp. 2002

91

6.1 The DB2 Catalog and LOBs If you are using LOBs, the table space holding LOBs data is managed using statistics in SYSIBM.SYSTABLEPART, the same as the regular table spaces. In this section you will find a description of the additional columns that have been provided by DB2 V6 to the system catalog in order to support LOBs. This may help you navigating through the DB2 system catalog searching for LOB related information. We also briefly describe the LOB tables that are created within the DB2 system catalog by DB2 V7.

6.1.1 Some catalog definitions for LOBs In this section we list the catalog tables and show the major DB2 catalog entries related to LOBs.

SYSIBM.SYSAUXRELS The table SYSIBM.SYSAUXRELS shows you all relationships between your base tables and their related auxiliary tables created in your system to store LOB columns. When you have a partitioned base table space, the partition number is also indicated. The columns you can retrieve from the table are: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

TBOWNER, authorization ID of the owner of the base table TBNAME, name of the base table COLNAME, name of the LOB column in the base table PARTITION, partition number when the base table space is partitioned, otherwise it is 0 AUXTBOWNER, authorization ID of the owner of the auxiliary table AUXTBNAME, name of the auxiliary table AUXRELOBID, internal identifier of the relationship between base table and auxiliary table IBMREQD, internal DB2 release dependency indicator, for internal use only

SYSIBM.SYSCOLUMNS If there are LOB columns defined within a base table, you will find them in the catalog table SYSIBM.SYSCOLUMNS as listed in Example 6-1. Example 6-1 Select from SYSIBM.SYSCOLUMS NAME -------AUXVALUE AUXVALUE

TBNAME -------SYSJARCLASS_SOURCE SYSJARDATA

TBCREATOR --------SYSIBM SYSIBM

COLTYPE LENGTH -------- -----CLOB 4 BLOB 4

LENGTH2 ----------10485760 104857600

򐂰 The TBNAME specified within the SYSIBM.SYSCOLUMNS row will indicate the name of the base table, because the LOB column is logically part of the base table. 򐂰 The field COLTYPE in the table will be set to indicate the data types of BLOB, CLOB, and DBCLOB. 򐂰 The LENGTH catalog column also will be set to a value of four for all LOB columns, because four bytes of internally defined information will be stored within each row of the base table for every defined LOB. 򐂰 The LENGTH2 column is set to the specified maximum length of the LOB column.

92

Large Objects with DB2 for z/OS and OS/390

SYSIBM.SYSLOBSTATS Three fields from SYSIBM.SYSLOBSTATS can be used to manage space for LOB table spaces: 򐂰 FREESPACE, kilobytes of free space in extents with respect to high used RBA (HURBA) 򐂰 AVGSIZE, average number of bytes per LOB 򐂰 ORGRATIO, indication of optimal physical location of active LOBs

SYSIBM.SYSTABLEPART There are some more fields from SYSIBM.SYSTABLEPART, already used for regular table spaces, that can also be used to manage your LOB table spaces and its partitions: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

CARDF, the number of LOBs SPACEF, KB of space allocated PQTY, 4 KB primary extent allocation SQTY, 4 KB secondary extent allocation DSNUM, number of linear data sets for table space EXTENTS, total primary and secondary extents

SYSIBM.SYSTABLES The rows contained in SYSIBM.SYSTABLES which describe the auxiliary table and the base table are listed in Example 6-2. Example 6-2 Select from SYSIBM.SYSTABLES NAME --------------BLOB_AUX_TABLE BLOB_BASE_TABLE

CREATOR -----PAOLO PAOLO

TYPE ---X T

TSNAME -------BLOBATS BLOBBTS

RECLEN STATUS ------ -----0 75

TABLESTATUS -----------

򐂰 RECLENGTH indicates the length of the stored record in the base table pages. It includes the actual four bytes of internally defined information that are stored within the base table for each defined LOB column. For the row describing an auxiliary table, the RECLENGTH field is set to 0. 򐂰 The value blank for the STATUS column indicates that the table has no parent index, and the definition is complete. The 'X’ instead indicates that the table has a parent index and that the table definition is complete. The value 'I’ indicates that the table definition is incomplete. The reason the table is incomplete is defined in the TABLESTATUS column. 򐂰 The TABLESTATUS contains 'L’ if the table definition is incomplete because an auxiliary table or auxiliary index has not been defined for a LOB column. It contains 'P’ if the table definition is incomplete because it lacks a parent index.

6.1.2 LOBs defined in DB2 catalog Even if you have no application using DB2 large objects in your environment, starting with V7, DB2 already defines LOBs in DBSNDB06. If you run a SELECT on SYSIBM.SYSAUXRELS using WHERE clause AUXTBOWNER = ‘SYSIBM’, you will see the two tables listed in Example 6-3. If you check their contents in the catalog, you will find out that tables SYSIBM.SYSJARCLASS_SOURCE and SYSIBM.SYSJARDATA contain LOB definitions. Example 6-3 DB2 catalog query SELECT COLNAME, PARTITION, AUXTBOWNER, AUXTBNAME FROM SYSIBM.SYSAUXRELS WHERE AUXTBOWNER = ‘SYSIBM’

Chapter 6. Data administration with LOBs

93

COLNAME -----------------CLASS_SOURCE JAR_DATA

PARTITION --------0 0

AUXTBOWNER ---------SYSIBM SYSIBM

AUXTBNAME -----------------SYSJARCLASS_SOURCE SYSJARDATA

Both tables are related to the JAVA stored procedures implementation within DB2. SYSIBM.SYSJARDATA has a 100 MB BLOB column for the contents of JAR files and it is the auxiliary table for SYSIBM.SYSJAROBJECTS. The SYSIBM.SYSJARCLASS_SOURCE has a 10 MB CLOB for the contents of the class in JAR files and it is the auxiliary table for SYSIBM.SYSJARCONTENTS.

6.2 Utilities In this section we discuss considerations related to LOBs when using the following utilities: 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰 򐂰

LOAD DSNTIAUL (this is a sample program, not a utility, but often used for unloading) UNLOAD COPY REPORT TABLESPACESET RUNSTATS REORG CHECK LOB CHECK DATA CHECK INDEX DSN1COPY RECOVERY

6.2.1 LOAD Because there is a limit of 32 KB for the size of the input record of the LOAD utility, you can use it only to LOAD LOB columns of 32 KB or less. When the input record is greater than 32 KB, you have to load the LOB data separately. See the sample job DSN8DLPL in the SDSNSAMP DB2 distribution libraries for an example. If you restart a LOAD job for a table with LOB columns that specified the RESUME YES option, you must use RESTART CURRENT. If you use RESTART PHASE to restart a LOAD job which specified RESUME NO, the LOB table spaces and indexes on auxiliary tables will be reset. For a table with LOB columns, you cannot restart a LOAD job that specifies the INCURSOR option. Indexes on the auxiliary tables are not built during the BUILD phase. Instead, LOB values are inserted (not loaded) into auxiliary tables during the RELOAD phase as each row is loaded into the base table, and each index on the auxiliary table is updated as part of the INSERT operation. Because the LOAD utility inserts keys into an auxiliary index, free space within the index may be consumed and index page splits may occur. Consider reorganizing an index on the auxiliary table after LOAD completes to introduce free space into the index for future INSERTs and LOADs.

94

Large Objects with DB2 for z/OS and OS/390

Note: Apply PTF UQ65857 for APAR PQ59820. It solves a page addressing problem and improves LOAD performance. LOAD allows you to specify LOG YES or NO during their execution. A LOB table space can also be defined with LOG YES or LOG NO and this will affect logging while updating or loading a LOB column. Table 6-1 shows the effect on logging output and LOB table space with the possible combinations of the two parameters. Table 6-1 Impact of logging on LOB tables Load and Reorg log impact on a LOB table LOAD LOG keyword

LOB table space LOG attribute

What is logged

LOB table space status after utility completes

LOG YES

LOG YES

Control information and LOB data

No pending status

LOG YES

LOG NO

Control information

No pending status

LOG NO

LOG YES

Nothing

COPY Pending

LOG NO

LOG NO

Nothing

COPY Pending

Loading LOB columns with input records that are < 32 KB works just like loading other non-LOB columns. Example 6-4 shows you the output from a LOAD job, loading data into a table with a CLOB column defined. Notice that even though NOCOPYPEND was specified in the input cards, the load job places both the base and aux table in copy pending. Notice also that the base table had been defined in a compressed table space. This is allowed for the table space containing the base table, but not for the LOB table space. In this specific case the compression is not of much help because of the small number of rows and their small size. The LOB table space was defined with LOG NO. Example 6-4 Output for Load job inserting into table with CLOB column DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = POLOR3LD DSNU050I DSNUGUTC - LOAD DATA LOG NO REPLACE INDDN SYSREC00 NOCOPYPEND DSNU650I -DB2G DSNURWI - INTO TABLE "PAOLOR3"."CLOB_BASE_TABLE" DSNU650I -DB2G DSNURWI - (DOCUMENT_NR POSITION(1) CHAR(10), DSNU650I -DB2G DSNURWI DESCRIPTION POSITION(11) CHAR(32), DSNU650I -DB2G DSNURWI DOCUMENT POSITION(43) CLOB) DSNU350I -DB2G DSNURRST - EXISTING RECORDS DELETED FROM TABLESPACE DSNU231I -DB2G DSNURBDC - DICTIONARY WITH 512 ENTRIES HAS BEEN SUCCESSFULLY BUILT FROM 21 ROWS FOR TABLESPACE PAOLOR3.CLOBBTS1 DSNU234I -DB2G DSNURWT - COMPRESSION REPORT FOR TABLE SPACE PAOLOR3.CLOBBTS1 1 KB WITHOUT COMPRESSION 1 KB WITH COMPRESSION 0 PERCENT OF THE BYTES SAVED FROM COMPRESSED DATA ROWS 0 PERCENT OF THE LOADED ROWS WERE COMPRESSED 72 BYTES FOR AVERAGE UNCOMPRESSED ROW LENGTH 72 BYTES FOR AVERAGE COMPRESSED ROW LENGTH 1 PAGES REQUIRED WITHOUT COMPRESSION 65 PAGES REQUIRED WITH COMPRESSION -6400 PERCENT OF THE DB2 DATA PAGES SAVED USING COMPRESSED DATA Chapter 6. Data administration with LOBs

95

DSNU304I -DB2G DSNURWT - (RE)LOAD PHASE STATISTICS - NUMBER OF RECORDS=21 FOR TABLE PAOLOR3.CLOB_BASE_TABLE DSNU302I DSNURILD - (RE)LOAD PHASE STATISTICS - NUMBER OF INPUT RECORDS PROCESSED DSNU300I DSNURILD - (RE)LOAD PHASE COMPLETE, ELAPSED TIME=00:00:08 DSNU381I -DB2G DSNUGSRX - TABLESPACE PAOLOR3.CLOBBTS1 IS IN COPY PENDING DSNU381I -DB2G DSNUGSRX - TABLESPACE PAOLOR3.CLOBATS1 IS IN COPY PENDING DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

6.2.2 DSNTIAUL If you select a LOB column in a list of field specifications when unloading a table, or select a LOB column by default (by omitting a list of field specifications), LOB data is materialized in the output, if the definition of the field is less than or equal to (<=) 32 KB. Any column greater than (>) 32 KB cannot be unloaded using this program. DSNTIAUL returns truncated data. Unloading data from an image copy data set is not available for LOB columns. DSNTIAUL works fine for unloading tables containing LOB columns with a size less than (<) 32 KB as shown in Example 6-5. Example 6-5 Output from Unload with LOB column < 32 KB DSN RUN PROGRAM(DSNTIAUL) PLAN(DSNTIB71) DSN END READY END DSNT490I SAMPLE DATA UNLOAD PROGRAM DSNT503I UNLOAD DATA SET SYSPUNCH RECORD LENGTH SET TO 80 DSNT504I UNLOAD DATA SET SYSPUNCH BLOCK SIZE SET TO 27920 DSNT503I UNLOAD DATA SET SYSREC00 RECORD LENGTH SET TO 87 DSNT504I UNLOAD DATA SET SYSREC00 BLOCK SIZE SET TO 27927 DSNT495I SUCCESSFUL UNLOAD 21 ROWS OF TABLE "PAOLOR3"."CLOB_BASE_TABLE

When using DSNTIAUL on LOB tables with columns > 32 KB, the output is truncated to about 32 KB. Below in Example 6-6 is the output from an unload of a LOB table with rows > 32 KB. Example 6-6 Output from DSNTIAUL where column length > 32 KB DSNT490I DSNT503I DSNT504I WARNING: WARNING: DSNT503I DSNT504I DSNT495I

SAMPLE DATA UNLOAD PROGRAM UNLOAD DATA SET SYSPUNCH RECORD LENGTH SET TO 80 UNLOAD DATA SET SYSPUNCH BLOCK SIZE SET TO 27920 DATA FROM COLUMN 4 WAS TRUNCATED TO 32756 BYTES FROM 2693248. DATA FROM COLUMN 4 WAS TRUNCATED TO 32668 BYTES FROM 32756. UNLOAD DATA SET SYSREC00 RECORD LENGTH SET TO 32756 UNLOAD DATA SET SYSREC00 BLOCK SIZE SET TO 32756 SUCCESSFUL UNLOAD 1 ROWS OF TABLE "PAOLO"."CLOB_BASE_TABLE"

6.2.3 UNLOAD The UNLOAD utility was introduced in DB2 V7. It can provide better performance, enhanced functionality, higher availability, improved concurrency, and low cost of operation, compared to previous methods used for unloading data.

96

Large Objects with DB2 for z/OS and OS/390

As we already have mentioned for DSNTIAUL, the UNLOAD utility does not act against LOB tables, because they are, in general, not available for access via SQL nor the UNLOAD utility. Every statement referring to LOB values has to be issued against the base table. You can use the UNLOAD utility for unloading your LOB data, but the maximum size is also represented by the maximum record length of your output file. When you want to truncate your data to fit in your output data set, you can use the syntax in Example 6-7. Example 6-7 Using UNLOAD with truncation UNLOAD TABLESPACE LOBDB.BASETS FROM BOOK_BASE_TABLE (BOOK_TEXT CLOB (10000) TRUNCATE)

The TRUNCATE keyword tells DB2 to truncate after the given number of bytes. If you are dealing with BLOBs, truncation of BLOB values normally results in unusable data in the case of images, movies or audio data. This data truncation occurs only when you explicitly specify the TRUNCATE option, this prevents you from being unaware of possible data truncation. If you do not specify the truncation option for your LOB data, the message reported in Example 6-8 is issued by DB2 whenever your composite record size is greater than 32 KB. Example 6-8 Using the UNLOAD utility on a base table with row size > 32 KB DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = UNLOAD DSNU050I DSNUGUTC - UNLOAD TABLESPACE PAOLO.CLOBBTS DSNU1218I -DB2G DSNUULIA - LOGICAL RECORD LENGTH OF OUTPUT RECORD EXCEEDED THE LIMIT FOR TABLE PAOLO.CLOB_BASE_TABLE DSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

When you specify BLOB, CLOB or DBCLOB as a field type, a four byte binary length field is placed in the output record prior to the actual data field. If your LOB column is nullable, a NULL indicator byte is placed before the length field. Any CCSID conversion is applied first, before the data is considered for encoding in output. For CLOBs, no conversion is applied if the subtype is BIT. For more information on data truncation and conversion purposes, refer to DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945. The UNLOAD utility does not support full image copies of LOB table spaces in input.

6.2.4 COPY Both full and incremental image copies are supported for a LOB table space, as well as SHRLEVEL REFERENCE, SHRLEVEL CHANGE, and the CONCURRENT options. COPY with CONCURRENT option copies empty or unformatted allocated data pages for a LOB table space. You can quiesce and copy both the base table space and the LOB table space at the same time to establish a recoverable point of consistency. Be aware that QUIESCE does not create a recoverable point for a LOB table space that contains LOBs defined with LOG NO. Another way is to run the COPY UTILITY for the base table space and the LOB table space in one UTILITY control statement as listed in Example 6-9.

Chapter 6. Data administration with LOBs

97

Example 6-9 Copy of base and LOB table space in one statement COPY TABLESPACE PAOLO.CLOBATS COPYDDN SYSREC00 FULL YES TABLESPACE PAOLO.CLOBBTS COPYDDN SYSREC01 FULL YES SHRLEVEL REFERENCE DSNU050I DSNUGUTC - COPY TABLESPACE PAOLO.CLOBATS COPYDDN SYSREC00 FULL YES TABLESPACE PAOLO.CLOBBTS COPYDDN SYSREC01 FULL YES SHRLEVEL REFERENCE DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE PAOLO.CLOBATS NUMBER OF PAGES=54 AVERAGE PERCENT FREE SPACE PER PAGE = 2.90 PERCENT OF CHANGED PAGES = 40.90 ELAPSED TIME=00:00:00 DSNU400I DSNUBBID - COPY PROCESSED FOR TABLESPACE PAOLO.CLOBBTS NUMBER OF PAGES=3 AVERAGE PERCENT FREE SPACE PER PAGE = 35.33 PERCENT OF CHANGED PAGES = 3.57 ELAPSED TIME=00:00:00 DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLO.CLOBATS DSNU428I DSNUBBID - DB2 IMAGE COPY SUCCESSFUL FOR TABLESPACE PAOLO.CLOBBTS DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

This will create a single RBA in SYSIBM.SYSCOPY for recovery purposes. Querying the SYSCOPY table you get the common RBA recorded with the above image copy as shown in Example 6-10. Example 6-10 Query of SYSCOPY for common RBA SELECT DBNAME,TSNAME,ICTYPE,ICDATE,ICTIME,HEX(START_RBA) FROM SYSIBM.SYSCOPY WHERE DBNAME = 'PAOLO' AND TSNAME LIKE 'CL%' ---------+---------+---------+---------+---------+---------+---------+---------+ DBNAME TSNAME ICTYPE ICDATE ICTIME COL1 ---------+---------+---------+---------+---------+---------+---------+---------+ PAOLO CLOBATS F 020319 142508 0002A9A86268 PAOLO CLOBBTS F 020319 142509 0002A9A86268 DSNE610I NUMBER OF ROWS DISPLAYED IS 2 DSNE616I STATEMENT EXECUTION WAS SUCCESSFUL, SQLCODE IS 100

If you take an inline image copy of a table with LOB columns, DB2 makes a copy of the base table space, but does not copy the LOB table spaces.

The SHRLEVEL options In general, when you use SHRLEVEL CHANGE for image copies of your LOB table space, somebody can interfere with your job and manipulate the content of the table you are currently backing up. Consider the following example: an image copy job backs up your LOB table space and it takes a couple of minutes, depending on the number and size of your LOBs stored in the auxiliary table. When somebody inserts a LOB value, some pages can be placed in the already backed up area of your table space while other pages can be stored in an area of your LOB table space your job has not backed up yet. When you have used LOG YES for the definition of the LOB table space, you can acquire a quiesce point after the image copy is taken to create a recoverable point in time. But when you have specified LOG NO for your LOB table space, you can be in trouble if you have to recover to an image copy taken with SHRLEVEL CHANGE. This means that you are not protected against an inconsistent status of your image copy when using SHRLEVEL CHANGE. We recommend using SHRLEVEL REFERENCE for copying the LOB table space, regardless if you take incremental or full image copies. 98

Large Objects with DB2 for z/OS and OS/390

6.2.5 REPORT TABLESPACESET The REPORT TABLESPACESET utility reports the LOB table spaces as part of the table space set that includes the base table space. So because of that REPORT TABLESPACESET can be used to identify the LOB table spaces that were implicitly created by DB2 when SQLRULES(STD) is used, as you can see in Example 6-11. Example 6-11 REPORT TABLESPACESET with CURRENT RULES STD created LOBs DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR2 0DSNU050I DSNUGUTC - REPORT TABLESPACESET TABLESPACE PAOLO.CLOBBTS DSNU587I -DB2G DSNUPSET - REPORT TABLESPACE SET WITH TABLESPACE PAOLO.CLOBBTS TABLESPACE TABLE DEPENDENT TABLE INDEXSPACE PAOLO.CLOBBTS PAOLO.CLOB_BASE_TABLE BASE TABLESPACE LOB TABLESPACE AUXILIARY TABLE INDEXSPACE PAOLO.CLOBBTS PAOLO.L1LX$XO8 PAOLO.CLOB_DOCUM1LXA0LID PAOLO.ICLOBRDO DSNU580I DSNUPORT - REPORT UTILITY COMPLETE - ELAPSED TIME=00:00:00 DSNU010I DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=0

INDEX

INDEX PAOLO.ICLOB_DOCUM1LXARNX

6.2.6 RUNSTATS When dealing with LOBs, you can run the RUNSTATS utility against the base table space, the LOB table space, or the index on the auxiliary table.

RUNSTATS on the base table space In SYSIBM.SYSCOLUMNS, the cardinality of a LOB column, COLCARDF, is determined by counting only non-null and non-zero length columns. HIGH2KEY and LOW2KEY are not applicable for the LOB column and will contain blanks. In SYSIBM.SYSCOLSTATS, the cardinality of a LOB column, COLCARDF, is determined by counting only non-null and non-zero length columns. HIGHKEY, LOWKEY, HIGH2KEY and LOW2KEY are not applicable for the LOB column and will contain blanks.

RUNSTATS on the index o the auxiliary table The statistics, NEAROFFPOSF and FAROFFPOSF in SYSIBM.SYSINDEXPART are not applicable for the index on the auxiliary table. For an index on the auxiliary table, the CARDF column of SYSIBM.SYSINDEXPART indicates the number of keys in the index which refer to LOBs. The statistics CLUSTERED and CLUSTERRATIO in SYSIBM.SYSINDEXPART are not applicable for the index on the auxiliary table.

RUNSTATS on the LOB table space RUNSTATS support is extended to LOB table spaces. The statistics gathered for a LOB table space are used to determine whether or not a LOB table space needs to be REORGed. The statistics in SYSIBM.SYSTABLESPACE will be updated. Statistics in SYSIBM.SYSTABLES will not be updated except for CARDF which contains the number of LOBs in the auxiliary table. Statistics in SYSIBM.SYSTABLEPART will not be updated except for CARD which will contain the number of LOBs. PERCACTIVE will contain -2 to indicate that this field is not updated for auxiliary tables. Statistics in SYSIBM.SYSCOLUMNS will not be updated. HIGH2KEY and LOW2KEY will contain blanks for the columns of the auxiliary table, COLCARDF will contain -2 to indicate that this field is not updated for auxiliary tables. The statistics for a LOB table space will be stored in a new catalog table called SYSIBM.SYSLOBSTATS. Its columns are described in Table 6-2.

Chapter 6. Data administration with LOBs

99

Table 6-2 SYSIBM.SYSLOBSTATS columns Column name

Description

AVGSIZE

Average size of a LOB in a LOB table space.

FREESPACE

Amount of free space in the LOB table space.

ORGRATIO

The ratio of organization in the LOB table space. A value of 1 indicates perfect organization of the LOB table space. The greater than 1 the value, the more disorganized is the LOB table space.

DBNAME

Database that contains the LOB table space named in NAME.

NAME

Name of the LOB table space.

STATSTIME

The time that RUNSTATS was executed.

Only the keywords in Table 6-3 are allowed if the table space specified by the TABLESPACE keyword is a LOB table space. Table 6-3 Allowed keywords and options Keyword

Valid options

SHRLEVEL

REFERENCE / CHANGE

REPORT

YES / NO

UPDATE

ALL / NONE

Note: Apply the PTF for PQ42864 to allow RUNSTATS on a LOB table space to produce correct statistics values for FREESPACE, ORGRATIO, PERCACTIVE, and PCTROWCOMP.

6.2.7 REORG The REORG utility for LOB table spaces is different from the normal table space REORG utility. In fact a REORG of a LOB table space is not compatible with any other utility: the LOB table space is unavailable to other applications during REORG processing. For REORG on non-LOB table spaces you may use OFFPOSLIMIT or INDREFLIMIT catalog query options to decide when to trigger your Reorg. If you specify the REPORTONLY option, REORG will produce a report detailing if a REORG is recommended; a REORG will not be performed. You cannot run REORG with the keyword REPORTONLY on a LOB table space. While LOAD allows LOG YES or NO for LOBs, REORG requires LOG YES. To get information on when to run a REORG on LOB table spaces you can view SYSIBM.SYSLOBSTATS. This table contains information about how the data in the table space is physically stored. The pertinent columns are AVGSIZE, FREESPACE, and ORGRATIO. Running RUNSTATS updates the SYSLOBSTATS table. If you then select from it, you get results similar to those shown in Example 6-12. 100

Large Objects with DB2 for z/OS and OS/390

Example 6-12 Table SYSIBM.SYSLOBSTATS L STATSTIME AVGSIZE FREESPACE ORGRATIO DBNAME NAME * * * * * * - -------------------------- ----------- ----------- -------- -------- -------* 2001-07-13-21.36.18.835699 0 156 0.00 DSN8D71L DSN8S71M 2001-07-13-21.36.19.107433 1210 140 2.00 DSN8D71L DSN8S71N 2001-07-13-21.36.18.536620 0 156 0.00 DSN8D71L DSN8S71L ******************************* END OF DB2 DATA *******************************

The definitions of the columns are: 򐂰 AVGSIZE AVGSIZE is the average size of all LOBs in the table space, or the total number of LOB bytes divided by the number of LOBs. For example, if you had an employee column for a picture of the employee saved as a LOB, the format used (GIF, JPEG, and so on) will affect the size. The typical value of size within the column is indicated by AVGSIZE. AVGSIZE does not contribute to triggering of any utility. Instead it can be used to estimate the data set sizes of work and sort data sets in LOB data set management. 򐂰 FREESPACE This value has no relationship with the FREESPACE parameter in the CREATE TABLESPACE statement. In this case FREESPACE is the number of kilobytes of available space in the LOB table space as reflected in the space maps, and it provides an indication of how many more LOBs can be added in the existing extents already allocated. 򐂰 ORGRATIO ORGRATIO is a measure of fragmentation or non-optimal organization of the LOBs stored in the LOB table space. A value of 1.0 is optimal. The more this value exceeds 1, the more your LOB table space is disorganized.

More information on ORGRATIO A LOB column always has an auxiliary index which locates the LOB within the LOB table space. Access path is not an issue, because LOB access is always done via an index probe using the auxiliary index. However, performance can be affected if LOBs are scattered into more physical pieces than necessary, thus involving more I/O to materialize. As you can see in Figure 6-1, there are four LOBs accessed via the auxiliary index. These LOBs are stored in chunks of pages. A chunk consists of 16 contiguous pages of LOB data. If the size of a LOB is smaller than a chunk (smaller than 16 pages), then it is expected to fit in one chunk. If the size is greater than a chunk (greater than 16 pages), then it is optimized to fit into the minimum number of chunks (LOB size divided by chunk size). If, because of fragmentation within chunks, LOBs are split up to store in more chunks than optimal, the ORGRATIO will increase. The small e at the bottom of a chunk in Figure 6-1 marks the end of a single LOB value.

Chapter 6. Data administration with LOBs

101

LOBs ORGRATIO - fragmented Auxiliary index rowid 1 rowid 2 rowid 3 rowid 4

E

chunk 1

E

chunk 2

E

E

chunk 3

chunk 4

chunk 5

Auxiliary table (LOB table space)

No access path statistics for LOBs ORGRATIO = (#extra chunk LOBs + #LOBs) / #LOBs = (2 + 4) / 4, or 1.5 Figure 6-1 Fragmented LOB table space

In our example, the first part of LOB 1 (accessed by rowid 1) is stored in chunk 1, the remaining bytes are placed in data pages of chunk number 2. Because of its size, LOB number 1 should fit into one chunk, but two chunks are used to store its value. This means that one extra chunk is used to store LOB number 1. Because of its size, LOB number 2 can at least be stored in two chunks which are also used by DB2 to store it. So there are no extra chunks for LOB number 2. The value of LOB number 3 should fit into one chunk and is also stored in one chunk which means that there are no extra chunks used for its storage. Things look different when you look at LOB number 4. The value behind LOB number 4 is expected to fit in two chunks, but DB2 uses three chunks. So there is another extra chunk used for LOB number 4. DB2 uses the following formula to calculate an appropriate value of ORGRATIO: ORGRATIO = (å extra chunks + å LOBs) / å LOBs If a LOB should fit in a number of n chunks, but n + m chunks are used to store its value, the number of m is referred to as extra chunks. After reorganization of the LOB space, Figure 6-2, the LOBs are placed in the optimal number of chunks. Since there are no extra chunks allocated, the ORGRATIO is 1.0 according to the formula shown below.

102

Large Objects with DB2 for z/OS and OS/390

LOBs ORGRATIO - after REORG Auxiliary index rowid 1 rowid 2 rowid 3 rowid 4

E

E

chunk 1

chunk 2

E

chunk 3

E

chunk 4

chunk 5

Auxiliary table (LOB table space)

Improved performance after REORG ORGRATIO = (#extra chunk LOBs + #LOBs) / #LOBs = (0 + 4) / 4, or 1.0 Figure 6-2 Non-fragmented LOB table space

Running REORG Running the REORG table space utility on a LOB table space will help increase the effectiveness of prefetch. For a LOB table space, REORG table space performs these actions: 򐂰 Removes embedded free space 򐂰 Attempts to make LOB pages contiguous Consider running REORG on the LOB table space when the value in the ORGRATIO column of SYSLOBSTATS is 2 or higher. The value is set by running the RUNSTATS utility. After inserting LOBs into your LOB table spaces and running the RUNSTATS utility, you may see values in ORGRATIO like 40.00 or 100.00. This depends on the size of the LOBs, and how many space map pages they span; small LOBs tend to show the ORGRATIO of 1. Running the REORG utility on these LOB table spaces that have been inserted with big LOBS will reset the value. Running the RUNSTATS utility on the ‘big’ reorged LOB table space will show the desired 1.00. REORG on a LOB table space will free embedded space and will attempt to make the LOB PAGES contiguous. Reorg will not do a delete and define for the LOB table space itself. It will not get rid of the empty data sets. If you want to adjust the primary allocation or to get rid of secondary extents, you can use RECOVER. That is, you change PRIQTY and SECQTY, then run RECOVER , and RECOVER will do a delete/define. However RECOVER will not get rid of secondary data sets. The only way to get rid of A002, A003, ... is by using an explicit drop/recreate of the object. If you use REORG SHRLEVEL NONE on a LOB table space and the LOB Manager determines that nothing needs to be done to the table space, no COPY pending status is set.

Chapter 6. Data administration with LOBs

103

However, if the LOB Manager indicates that changes are needed, REORG places the reorganized LOB table space or partition in COPY pending status. In this situation, perform a full image copy to reset the COPY pending status and to ensure that a backup is available for recovery.

Reclaim space for LOB table spaces The FREESPACE gives you an indication of how much allocated space is available for more LOBs. LOB table spaces can accumulate dead space that can be reclaimed during reorganization. If this dead space is not regularly reclaimed, it may result in acquiring more extents than needed to add additional LOBs. For LOB table spaces, an updated LOB will be written without reclaiming the old version of the LOB immediately. When FREESPACE approaches zero for a LOB table space, it is a good time to reclaim the space, but there are no direct statistics indicating how much will be reclaimed. You can calculate the percentage of free space for LOB table spaces based on the values of FREESPACE in SYSLOBSTATS and SPACEF in SYSTABLEPART:

(FREESPACE / SPACEF) * 100 We recommend that you reorganize LOB table spaces, if the percent of free space is less than 10%. REORG does honor the PCTFREE and FREEPAGE values specified at CREATE TABLESPACE time, but they do not make much sense with LOBs, and should be set to zero. REORG will not delete and redefine the data set during reorganization of a LOB table space. If the number of extents are a problem, they can be changed, if DB2 managed, by altering the PRIQTY and SECQTY values, and then running RECOVER TOCOPY to delete and redefine the data sets with the new space quantities.

REORG UNLOAD EXTERNAL The UNLOAD EXTERNAL option is not available for LOB table spaces and its use will result in an error message. When you try to use this option for a REORG on a base table, which is possible, it depends on the declared size of your LOB values if DB2 lets you do it or not. As long as the composite size of your columns does not exceed the maximum record length, you are allowed to do it. Otherwise the message shown in Example 6-13 is issued. Example 6-13 Trying to use REORG UNLOAD EXTERNAL on a base table row > 32 KB DSNU000I DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = REORG DSNU050I DSNUGUTC - REORG TABLESPACE PAOLO.CLOBBTS UNLOAD EXTERNAL DSNU297I -DB2G DSNURFTB - COMPOSITE RECORD SIZE TOO LARGE FOR PAOLO.CLOB_BASE_TABLE DSNU012I DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8

6.2.8 CHECK LOB The CHECK LOB online utility can be run against a LOB table space to identify any structural defects in the LOB table space and any invalid LOB values. You must first recover a LOB table space that is in RECOVER-pending status before running CHECK LOB. If you plan to run CHECK DATA on a base table space containing at least one LOB column, complete the following steps prior to running CHECK DATA:

104

Large Objects with DB2 for z/OS and OS/390

1. Run CHECK LOB on the LOB table space. 2. Run CHECK INDEX on the index on the auxiliary table prior to running CHECK DATA to ensure the validity of the LOB table space and index on the auxiliary table. 3. Run CHECK INDEX on the base table space indexes. If the LOB table space is in either the CHECK-pending or RECOVER-pending status, or if the index on the auxiliary table is in REBUILD-pending status, CHECK DATA will issue an error message and fail. Run the CHECK LOB online utility against a LOB table space that is marked CHECK pending (CHKP) to identify structural defects. If none is found, the CHECK LOB utility turns the CHKP status off. Run the CHECK LOB online utility against a LOB table space that is in auxiliary warning (AUXW) status to identify invalid LOBs. If none exists, the CHECK LOB utility turns AUXW status off. Run CHECK LOB after a conditional restart or a point-in-time recovery on all table spaces where LOB table spaces may not be synchronized. After successful execution, CHECK LOB resets the CHECK pending (CHKP) and auxiliary warning (AUXW) statuses.

6.2.9 CHECK DATA If you run CHECK DATA AUXERROR REPORT or INVALIDATE on a base table space containing at least one LOB column, the following errors may be reported: 򐂰 Orphan LOBs An orphan LOB column is a LOB found in the LOB table space but not referenced by the base table space. An orphan can result if you recover the base table space to a point in time prior to the insertion of the base table row or prior to the definition of the LOB column. An orphan can also result if you recover the LOB table space to a point in time prior to the deletion of a base table row. 򐂰 Missing LOBs A missing LOB column is a LOB referenced by the base table space, but the LOB is not in the LOB table space. A missing LOB can result if you recover the LOB table space to a point in time when the LOB column is not in the LOB table space. This could be a point in time prior to the first insertion of the LOB into the base table, or when the LOB column is null or has a zero length. 򐂰 Out-of-synch LOBs An out-of-synch LOB error occurs when DB2 detects a LOB that is found in both the base table and the LOB table space, but the LOB in the LOB table space is at a different level. A LOB column is also out-of-synch if the base table LOB column is null or has a zero length, but the LOB is found in the LOB table space. An out-of-synch LOB can occur anytime you recover the LOB table space or the base table space to a prior point in time. 򐂰 Invalid LOBs An invalid LOB is an uncorrected LOB column error found by a previous execution of CHECK DATA AUXERROR INVALIDATE. Note: Apply PTF UQ64673 for APAR PQ46137 to eliminate several errors while executing CHECK DATA.

Chapter 6. Data administration with LOBs

105

Depending on the AUXERROR clause specified, the following actions are performed: 򐂰 With AUXERROR REPORT DB2 sets the base table space to the auxiliary CHECK-pending (ACHKP) status. If CHECK DATA encounters only invalid LOB columns and no other LOB column errors, the base table space is set to the auxiliary warning status. 򐂰 With AUXERROR INVALIDATE DB2 sets the base table LOB column to an invalid status, and sets the base table space to the auxiliary warning (AUXW) status. You can use SQL to update a LOB column in the AUXW status, however, any other attempt to access the column will result in a -904 SQL return code. If you use exception tables, the exception table for the base table must have a similar LOB column and a LOB table space for each LOB column. If an exception is found, DB2 moves the base table row with its LOB column to the exception table, and moves the LOB column into the exception table's LOB table space. If you specify DELETE YES, DB2 deletes the base table row and the LOB column. An auxiliary table cannot be an exception table. A LOB column check error is not included in the exception count. A row with a LOB column check error only does not participate in exception processing. You must complete all LOB column definitions for a base table before running CHECK DATA. A LOB column definition is not complete until the LOB table space, auxiliary table and index on the auxiliary table have been created. If any LOB column definition is not complete, CHECK DATA will fail and issue error message DSNU075E. If you terminate CHECK LOB during the CHECKLOB phase, it sets the table space to CHECK-pending status. Otherwise, CHECK LOB resets the CHECK-pending status at the end of the phase if no errors are detected. You can restart a CHECK LOB utility job, but it starts from the beginning again. Auxiliary warning (AUXW) status is set on when at least one base table LOB column has an invalidated LOB as a result of running CHECK DATA AUXERROR INVALIDATE. An attempt to retrieve an invalidated LOB results in a -904 SQL return code. The RECOVER utility also sets AUXW status if it finds an invalid LOB column. Invalid LOB columns may result from the following actions (all three must apply): 򐂰 LOB table space was defined with LOG NO. 򐂰 LOB table space was recovered. 򐂰 LOB was updated since the last image copy. If you run CHECK LOB, and LOB table space errors are found, the table space is placed in CHECK-pending status. Complete the following tasks to remove the CHECK-pending status: 򐂰 Correct any defects found in the LOB table space either through SQL update/delete, or by using the REPAIR utility. 򐂰 Run CHECK LOB again, or run the REPAIR utility, in order to reset CHECK-pending or AUXW status. Table 6-4 summarizes the actions for resetting the AUXW status.

106

Large Objects with DB2 for z/OS and OS/390

Table 6-4 Resetting AUXW flags Resetting auxiliary warning status Status

Abbreviation

Object Affected

Corrective Action

Notes

Auxiliary Warning

AUXW

Base table space

1) Update or delete invalid LOBs through SQL 2) Run CHECK DATA utility to verify the validity of LOBs and reset the AUXW status.

1, 2, 3

LOB table space

1) Update or delete invalid LOBS through SQL. 2) Run CHECK LOB utility to verify the validity of LOBs and reset the AUXW Status. Alternatively you can run the REPAIR utility to set NOLOBCHKP. Be aware that using the REPAIR utility to delete invalid LOBS may cause the base tables and the Index on the AUX table to reference invalid LOBS.

1

Notes 1) A base table space or LOB table space in the AUXW status is available for processing by SQL, even though it contains invalid LOBS However, an attempt to retrieve an invalid LOB returns a -904. 2) DB2 can access all rows of a base table space that is in the AUXW status. SQL can update the invalid LOB column and delete base table rows, but the value of the LOB column cannot be retrieved. If DB2 attempts to access an invalid LOB column, a -904 SQL code is returned. The AUXW status remains on the base table space even when SQL deletes or updates the last invalid LOB column 3) If CHECK DATA AUXERROR REPORT encounters only invalid LOB columns and no other LOB column errors, the base table space is set to auxiliary warning status.

CHECK LOB issues message DSNU743I whenever a LOB value is invalid. The violation is identified by: 򐂰 The row ID and version number of the LOB 򐂰 A reason code for the error 򐂰 The page number where the error was found You can resolve LOB violations by using the UPDATE or DELETE SQL statements to update the LOB column or delete the row associated with the LOB (use the rowid given in message DSNU743I). If CHECK LOB issues either message DSNU785I or DSNU787I, it has detected a logical inconsistency within the LOB table space. Contact IBM Support Center for assistance with diagnosing and resolving the problem. Table 6-5 summarizes the actions to reset the ACHKP status. Table 6-5 Resetting AUX CHECK pending status Resetting auxiliary check-pending status Status

Abbreviation

Object affected

Corrective Action

Notes

Auxiliary Check Pending

ACHKP

Base table space

Run the Check Data utility with the Appropriate scope option to verify the validity of LOBS and reset ACHKP status.

1)

Notes 1) A base table space in the ACHKP status is unavailable for processing by SQL.

Chapter 6. Data administration with LOBs

107

6.2.10 CHECK INDEX When checking an auxiliary table index, CHECK INDEX verifies that each LOB is represented by an index entry. To run this utility, the privilege set of this process must include one of the following authorities: 򐂰 STATS privilege for the database 򐂰 DBADM, DBCTRL, or DBMAINT authority for the database 򐂰 SYSCTRL or SYSADM authority An ID with installation SYSOPR authority can also execute CHECK LOB.

6.2.11 DSN1COPY The standalone utility DSN1COPY works on LOB table spaces just as with other types, with the exception that, when working with LOB table spaces, the parm field LOB must be specified. LOB specifies that the SYSUT1 data set is a LOB table space. Empty pages in the table space are copied, but no error messages are issued. You cannot specify the SEGMENT and INLCOPY options with the LOB parameter. DB2 attempts to determine if the input data set is a LOB data set. If you specify the LOB option but the data set is not a LOB data set, or if you omit the LOB option but the data set is a LOB data set, DB2 issues an error message and terminates. Example 6-14 shows the error you receive when not using the LOB parameter for a LOB table space. Example 6-14 DSN1COPY error for LOB table space DSN1999I START OF DSN1COPY FOR JOB PAOLOR3C PH01S01 DSN1954I DSN1COPY PARAMETER PAGESIZE OR LOB IS MISSING OR INCORRECTLY SPECIFIED DSN1993I DSN1COPY TERMINATED, 0 PAGES PROCESSED

We used DSN1COPY to restore data to a different base and auxiliary table from a full image copy data set. The JCL for the LOB table space DSN1COPY restore is reported in Example 6-15. Example 6-15 JCL for DSN1COPY OBIDXLAT for LOB table space //PH01S01 EXEC PGM=DSN1COPY, // PARM=('FULLCOPY,RESET,LOB,32K,OBIDXLAT') //STEPLIB DD DSN=DB2G7.SDSNLOAD,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUDUMP DD SYSOUT=* //SYSUT1 DD DSN=PAOLOR3.IMAGE.CLOBATS1.IC0322,DISP=OLD //SYSUT2 DD DSN=DB2V710G.DSNDBC.PAOLOR3.CLOBATS2.I0001.A001,DISP=OLD //SYSXLAT DD * 307 307 5 14 6 15 *********************************************************************************** DSN1COPY IS PROCESSED WITH THE FOLLOWING OPTIONS: NO CHECK/NO PRINT/32K/FULLCOPY /NON-SEGMENT/NUMPARTS = 0/ OBIDXLAT/NO VALUE/ RESET/ /LOB/PIECESIZ= / DSSIZE= INPUT DSNAME = PAOLOR3.IMAGE.CLOBATS1.IC0322 , SEQ

108

Large Objects with DB2 for z/OS and OS/390

OUTPUT DSNAME = DB2V710G.DSNDBC.PAOLOR3.CLOBATS2.I0001.A001 , VSAM DSN1COPY COMPLETED SUCCESSFULLY, 00000041 PAGES PROCESSED

6.2.12 REPAIR When dealing with LOBs, the LOCATE table space statement locates data to be repaired within a table space. When you specify LOCATE ROWID for a LOB table space, with the delete option, the LOB specified by ROWID is deleted with its index entry. All pages occupied by the LOB are converted to free space. The DELETE statement will not remove any reference to the deleted LOB from the base table space. One LOCATE statement is required for each unit of data to be repaired. Several LOCATE statements can appear after each REPAIR statement. With ROWID X'byte-string' you can specify that the data to be located is a LOB in a LOB table space. The byte-string is the row ID that identifies the LOB column. The most likely source of a row ID is an orphaned LOB that is reported by the CHECK LOB utility. If you specify the ROWID keyword, the specified table space must be a LOB table space. With VERSION X'byte-string' you can specify that the data to be located is a LOB in a LOB table space. The byte-string is the version number that identifies the version of the LOB column. The most likely source of a version number is an orphaned LOB that is reported by the CHECK LOB utility, or an out-of-synch LOB that is reported by the CHECK DATA utility. If you specify the VERSION keyword, the specified table space must be a LOB table space.

6.2.13 RECOVERY In some ways, planning for LOB recovery is similar to that for user defined referential integrity since you have to remember that there is a relationship between a table with a LOB column and the associated LOB table space. It is true that tables involved in referential integrity relationships must be considered as part of the same table space set for recovery purposes. Similarly, a LOB table space and its associated base table space are part of a table space set. That set, too, is reported using REPORT table space.

Recovering to a prior point in time You can recover your LOB table space to a prior point in time, using TOLOGPOINT, TORBA, or TOCOPY. Ideally, you have taken copies of the LOB table space and the related base table space at a quiesce point. If you then recover to the quiesce point, you avoid having the LOB or base table space marked as check pending. Thus, if the LOB table space was defined with LOG NO, any log records written for the LOB table space between its last image copy and the recover-to-point means the AUX WARNING will be turned on. Then you have to run the CHECK LOB utility to find any invalid LOBs. For more details see 6.3, “Recovery scenario” on page 111.

Chapter 6. Data administration with LOBs

109

Recovering to the current point in time Recovering to the current point in time for a base table containing a LOB column is not any different form the past. DB2 will apply the appropriate image copy to the base table space and read the log since copy time until present, redoing all the changes. This is also true for the LOB table space. However, if the LOB table space is defined with LOG NO and log records must be applied to the LOB, the LOB is invalid and DB2 will set AUX Warning (AUXW) for that LOB table space.

Recovering LOB PAGES on the Logical Page List If there are LPL entries for a LOB table space the same procedure has to take place as for regular table spaces (START DATABASE command with the SPACENAM option). But if the LOB table space is defined with LOG NO and that log data is needed, then LOB CHECK PENDING will be turned on for that table space. Therefore, you will have to run the CHECK LOB utility to identify which LOBs are invalid.

In general You should quiesce and copy both the base table space and the LOB table space at the same time to establish a recoverable point of consistency. Be aware that QUIESCE does not create a recoverable point for a LOB table space that contain LOBs defined with LOG NO. The RECOVER utility sets AUXW status if it finds an invalid LOB column. Invalid LOB columns may result from the following actions (all three must apply): 򐂰 LOB table space was defined with LOG NO 򐂰 LOB table space was recovered 򐂰 LOB was updated since the last image copy

Object states after recovery After describing and reviewing all the different utilities you may run in your LOB environment, the following table will give you a list of all the different states of the different objects you may receive when you are dealing with recovery scenarios in a LOB environment. See Table 6-6. Table 6-6 Object status after different recovery scenarios Determining object status after recovery

110

Object

Recovery type

Base table space status

Index on auxiliary table status

LOB table space status

Base table space

Current RBA or LRSN

none

none

none

Base table space

Point in Time

CHKP

none

none

Index on Auxiliary table

Current RBA or LRSN

none

none

none

Index on Auxiliary table

Point in Time

none

CHKP

none

LOB table space

Current RBA or LRSN, LOB table space defined with LOG (YES)

none

none

none

LOB table space

Current RBA or LRSN, LOB table space defined with LOG(NO)

none

none

AUXW (1)

Large Objects with DB2 for z/OS and OS/390

Determining object status after recovery LOB table space

TOCOPY, copy was taken with SHARELEVEL REFERENCE

CHKP

RBDP

none

LOB table space

TOCOPY, copy was taken with SHARELEVEL CHANGE

CHKP

RBDP

CHKP, AUXW

LOB table space

TORBA or TOLOGPOINT (not a quiesce point)

CHKP

RBDP

CHKP,AUXW

LOB table space

TORBA or TOLOGPOINT (at a quiesce point)

CHKP

RBDP

none

Note: (1) If at any time a log record is applied to a LOB table space that results in a LOB being marked invalid, the LOB table space is set to AUXILIARY Warning Status.

6.3 Recovery scenario In the previous sections we have described the details and dependencies of using utilities regarding LOBs. As an example we will now walk you through a recovery scenario showing the steps to take for a successful recovery. This includes the LOB base table space, the LOB table space and the auxiliary index. Also it will give you an idea of what needs to be performed in similar situations. With this information you can preventively set up a scenario for your recovery case when dealing with LOBs.

Setting up the recovery environment To get started, we first implement the DB2 objects we need. How to create LOBs is described in detail in 3.2, “Storing LOBs” on page 16. For our purposes, we build a database (PAOLO), a base table space (CLOBBTS) with its base table (CLOB_BASE_TABLE), the LOB table space (CLOBATS) with the auxiliary table (CLOB_AUX_TABLE) and the auxiliary index (CLOBAIX) which goes with it. See Figure 6-3.

Chapter 6. Data administration with LOBs

111

D ata B ase: PA O LO Ba se Table Space:

C LO B B T S B ase Table C LO B_BASE_TA BL E

C ol 1

C ol 2

LO B C ol 1

Vala

Valb

A B C D EF G ...

Valc

Vald

1234567...

LO B C ol 1 F lags

. .. . ..

LO B Table Space:

C LO B AT S R ow ID

LO B

A B C D E F G ...

1234567...

Value

Picture 1

Picture 2

R ow ID LO B Table A uxiliary In d ex: CL O B A IX

C L O B _ A U X _TA B LE

Figure 6-3 Recovery environment

Creating a recovery situation In order to recreate a reasonable recovery situation we recover to Copy the LOB table space (LOBATS), that was defined with LOG NO (this is recommended for performance reasons). However, after the last full image copy, LOBs have been accessed via insert and so the base table, the auxiliary table and the auxiliary index have been updated. We have used the last full image copy to recover the LOB table space to a previous point in time. See Figure 6-4.

112

Large Objects with DB2 for z/OS and OS/390

Data Base: PAOLO B ase Table Space:

C LO B B TS B ase Table CLO B _B AS E_TA BLE

Col 1

Col 2

LO B Col 1

Vala

Valb

ABCD EFG...

Valc

Vald

1234567...

LOB C ol 1 Flags

Recover w ith a full im age copy

.. . .. .

LOB Table Space:

C LO B ATS Row ID

LO B

ABCD EFG...

Value

Picture 1

Gone

Row ID LOB Table Auxiliary Index: C LO BAIX

CLOB_A UX_TABLE

Figure 6-4 Recovering the auxiliary table with full image copy

The recover has left the base table and index at a different point in time, compared to the auxiliary table. Because of the special relationship of base and auxiliary table, the row entries are now more on the base table and the auxiliary index than in the aux table, since inserts have occurred. A display on our database, after recovering the LOB table space, shows the status reported in Example 6-16. Example 6-16 Display database after recovery, using last full image copy DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----CLOBATS LS RW CLOBBTS TS RW,ACHKP,COPY CLOBAIX IX RW,RBDP ******* DISPLAY OF DATABASE PAOLO ENDED **********************

The display also gives us the information that the table space (CLOBBTS) is in aux check pending (ACHKP). This requests to run CHECK DATA on the auxiliary table. It also shows that the entire auxiliary index is in rebuild pending status (RBDP).

Chapter 6. Data administration with LOBs

113

Rebuilding the auxiliary index First we focus on the auxiliary index. As the status indicated, we have to run a Rebuild Index utility (REBUILD INDEX (PAOLO.CLOBAIX)), in order to synchronize the index with the current contents of the auxiliary table. See Figure 6-5.

D ata Base: PAO LO B ase Table Space: CLO BB TS Base Table CLOB _B ASE_TAB LE

C ol 1

C ol 2

LOB C ol 1

Vala

Valb

ABC D EFG ...

Valc

Vald

LOB C ol 1 Flags

1234567...

.. . .. .

LO B Table Space: C LO B ATS R ow ID

LOB

ABC D EFG...

Value

Picture 1

Row ID LO B Table

Rebuild index utility

A uxiliary Index: C LO B AIX

C LOB _A UX _TA B LE

Figure 6-5 Rebuilding the index

After successfully running the rebuild index utility, the DISPLAY DATABASE command gives us the status information listed in Example 6-17. Example 6-17 Display database after Rebuild Index DSNT360I DSNT361I

-DB2G *********************************** -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----CLOBATS LS RW CLOBBTS TS RW,ACHKP,COPY CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************

114

Large Objects with DB2 for z/OS and OS/390

Running the Check Data utility As you have seen, we still have the aux check pending (ACHKP,COPY) status left on the base table (CLOBBTS). We then run the CHECK DATA utility on the base table with SCOPE(AUXONLY) and AUXERROR(INVALIDATE). The scope keyword AUXONLY is used because only the base table needs checking. See the check data utility job output in Example 6-18. Example 6-18 Check Data utility output DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = PAOLOR3 DSNUGUTC - CHECK DATA table space PAOLO.CLOBBTS SCOPE(AUXONLY) AUXERROR(INVALIDATE) SORTDEVT SYSDA DSNUKDST - CHECKING TABLE PAOLO.CLOB_BASE_TABLE DSNUGSOR - SORT PHASE STATISTICS NUMBER OF RECORDS=101 ELAPSED TIME=00:00:00 DSNUGSOR - SORT PHASE STATISTICS NUMBER OF RECORDS=1 ELAPSED TIME=00:00:00 DSNUKERK - TABLE=PAOLO.CLOB_BASE_TABLE COLUMN=DOCUMENT IS MISSING IN INDEX PAOLO.CLOBAIX ROWID=X'41C48DDCC1CED26F260401D37018010000000000032D' VERSION=X'0001' DSNUKDAT - CHECK TABLE PAOLO.CLOB_BASE_TABLE COMPLETE, ELAPSED TIME=00:00:00 -DB2G DSNUKRDN - TABLE=PAOLO.CLOB_BASE_TABLE COLUMN=DOCUMENT WAS SET INVALID ROWID=X'41C48DDCC1CED26F260401D37018010000000000032D' -DB2G DSNUGSRX - TABLESPACE PAOLO.CLOBBTS IS IN COPY PENDING -DB2G DSNUGSRX - TABLESPACE PAOLO.CLOBBTS IS IN AUX WARNING STATE DSNUK001 - CHECK DATA COMPLETE,ELAPSED TIME=00:00:02 DSNUGBAC - UTILITY EXECUTION COMPLETE, HIGHEST RETURN CODE=4

After running the Check Data utility, the status of the base table space (CLOBBTS) has changed from ACHKP,COPY to COPY,AUXW, as listed in Example 6-19. The CHECK DATA also gives us the information we need to clean up the base table row that still was pointing to an invalid LOB. Example 6-19 Display database after Check Data DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----CLOBATS LS RW CLOBBTS TS RW,COPY,AUXW CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************

Removing the copy pending and deleting the invalid ROWID Before we can delete the invalid ROWID, we have to run a full image copy to remove the copy pending flag on the BASE TABLE. Then the identified ROWID within the CHECK DATA output, as shown in Figure 6-6, can be deleted from the base table.

Chapter 6. Data administration with LOBs

115

Data Base: PAOLO Base Table Space: CLOBBTS Base Table CLOB_BASE_TABLE

Col 1

Col 2

LOB Col 1

Vala

Valb

ABCDEFG...

LOB Col 1 Flags

...

SQL: delete from clob_base_table where row_id = rowid (x'.....')

LOB Table Space: CLOBATS Row ID

LOB Value

ABCDEFG...

Picture 1

Row ID LOB Table Auxiliary Index: CLOBAIX

CLOB_AUX_TABLE

Figure 6-6 Delete row from base table

This is done by using normal SQL, pointing to the invalid ROWID in the SQL where clause and the statement look like: DELETE FROM BASE_TABLE WHERE ROW_ID = ROWID(X'41C48DDCC1CED26F260401D37018010000000000032D’)

Solving the auxiliary warning on the base table Now we still have to complete the recovery process by solving the AUXW status on the base table. This can be done running the CHECK DATA SCOPE(AUXONLY) AUXERR(REPORT) again, like we have done before in Example 6-18. After this execution, the display database shows that everything is in order again, as shown in Example 6-20. Example 6-20 Display database at the end DSNT360I -DB2G *********************************** DSNT361I -DB2G * DISPLAY DATABASE SUMMARY * GLOBAL DSNT360I -DB2G *********************************** DSNT362I -DB2G DATABASE = PAOLO STATUS = RW DBD LENGTH = 16142 DSNT397I -DB2G NAME TYPE PART STATUS PHYERRLO PHYERRHI CATALOG PIECE -------- ---- ---- ------------------ -------- -------- -------- ----CLOBATS LS RW CLOBBTS TS RW CLOBAIX IX RW ******* DISPLAY OF DATABASE PAOLO ENDED **********************

116

Large Objects with DB2 for z/OS and OS/390

Full image copy is reasonable with LOG NO The last step we need to complete our recover scenario is to take a full image copy of the base and the auxiliary table. Because the LOB table was created with LOG(NO) only a full image copy with SHRLEVEL REFERENCE is the correct backup for the LOB environment. See Figure 6-7.

Data Base: PAO LO Base Table Space: CLOBBTS B ase Table CLO B_BASE_TA BLE

C ol 1

C ol 2

LOB C ol 1

Vala

Valb

ABCD EFG ...

LOB C ol 1 Flags

...

Take a full im age copy of the base table space and the LOB table space

..

FIC

FIC

LOB Table Space: CLOBATS R ow ID

LOB

ABC DEFG...

Value

Picture 1

R ow ID LOB Table Auxiliary Index: CLOBAIX

CLOB_AUX_TA BLE

Figure 6-7 Take a full image copy

6.4 Operational scenario The most prevalent way to get LOB data onto the mainframe is through file transfer (FTP) from a non-host based platform and writing a program which runs on the host taking in this file to insert the data into the LOB table. Another possibility is using a program running on a non-host platform with file reference variables pointing to files on the workstation and using DRDA to insert it into the mainframe based database. Another means is through the use of the DB2 Extenders, which use UDFs to accomplish the insert. The extender model uses DRDA to connect workstations to the mainframe through a normal n-tier architecture.

Chapter 6. Data administration with LOBs

117

The evolution of n-tier architecture has demanded that data from the mainframe become available to workstations. You can enable a DRDA connection from the workstation to any member of the DB2 family using DB2 Connect. There are three flavors of DB2 Connect, PE (Personal Edition), EE (Enterprise Edition), or EE Unlimited Edition. DB2 Connect provides extensive application programming tools such as OLE DB, JDBC, SQLJ, DB2 CLI, and embedded SQL. DB2 Connect integrates with both Java and Microsoft models for developing Web applications. When using DB2 Connect PE, it is built in the normal 2- tier approach. This may be desirable if the environment has native TCP/IP support available, and an intermediate server is not desirable. When using DB2 connect in the EE configuration, it is a normal 3-tier architecture which connect LAN based workstations to the mainframe host databases. Some good uses for this are when the host system doesn't support native TCP/IP and needs support for JAVA APPLETS, or supporting workstations on platforms where PE cannot exist (UNIX). The Unlimited edition of EE avoids the necessity of counting users which must happen when using EE. It is an MSU based licensing model, which lends itself well to Web based applications.

Backup considerations Data administration with LOBs is challenging, especially when you have to deal with LOG NO definitions on your LOB table spaces. DB2’s recovery process is built on log data sets by definition, but using large objects allows you to turn off logging mechanisms or even forces you to turn them off when you use them in a size bigger than 1 GB. Backing up a high availability scenario having a LOG YES LOB table space can be compared with copying and recovering referential integrity tables from the logical point of view, except the amount of data is probably much higher for a single table space. When your scenario requests high availability and you have to specify LOG NO for the LOB table space for whatever reasons and it is not necessary to apply updates during the day on your LOBs, you can consider collecting your LOB updates during the day and process them in your batch window. When you take a full image copy of your base table space and the LOB table space after your batch job is done, you are prepared again for the day. This helps you minimizing outage times during the day when you have to recover a LOG NO LOB table space.

118

Large Objects with DB2 for z/OS and OS/390

7

Chapter 7.

Performance with LOBs In this chapter we discuss topics directly and indirectly related to LOBs performance. The chapter is structured as follows: 򐂰 LOB materialization 򐂰 Buffer pools and group buffer pools 򐂰 LOBs performance considerations

© Copyright IBM Corp. 2002

119

7.1 LOB materialization LOB materialization from the point of view of DB2 is the function DB2 uses to place a LOB value into contiguous storage in virtual storage. Materialization will happen also on disk, but we are especially interested in the cases where the LOB materializes in buffer pools, data spaces, or the user (allied) address space. Where the LOB is materialized depends on the way the LOB data is retrieved. In general, because LOB values can be very large, DB2 will do the utmost to try to avoid the DB2 materialization of LOB data until it is absolutely necessary. DB2 will materialize the LOB in the following different cases: 򐂰 When your application calls a user-defined function with a LOB as an argument 򐂰 When it moves a LOB into or out of a stored procedure 򐂰 When it assigns a LOB host variable to a LOB locator host variable 򐂰 When it converts a LOB from one CCSID to another With or without the use of a locator, retrieving a LOB value will always go through the buffer pools associated with the LOB table space. Basically, from the DB2 point of view, if a LOB needs to be materialized in storage, it will be placed in a data space. The amount of storage used in data spaces for LOB materialization depends upon a number of factors. They include the size of the LOB, the number of LOBs in a statement that need to be materialized, and the use of cursors to hold position in an application. Figure 7-1 provides an overview of the materialization process.

DB2 DDF Address Space

Data Spaces

DB2 DBM1 Address Space Local Buffer Pool

Stored Procedure Address Space

Data Manager

LOB Manager

Buffer Manager

User Address Space

Disk environment

Figure 7-1 DB2 materialization overview picture

120

Large Objects with DB2 for z/OS and OS/390

The different cases of materialization In order to give you a better picture of LOB materialization and when it will occur, we describe different scenarios. In general you can UPDATE, INSERT, DELETE, or SELECT a LOB value. As described in 4.2, “LOB locators” on page 46, there are good reasons for using LOCATORs when processing LOB values. Beside other benefits, a LOCATOR is recommended if you are careful about storage allocation within your users’ address space, overall performance, and useful handling your LOB data. Materialization within your DB2 data spaces depends on the way the LOB value is accessed.

SELECT First we assume you have two applications that will select a LOB for further processing. This could be printing a LOB that contains a book or storing it into a non DB2 data set. However, let us say we have application A using a LOCATOR and application B not using a LOCATOR, but rather a host variable. From the point of view of DB2 LOB materialization, application A and application B will not materialize the LOB in DB2 data spaces when they only select a LOB value. See Figure 7-2.

User A d d re s s S p a c e

D B 2 D B M 1 A d d re s s

B

Space L o c a l B u ffe r P o o l

Data Manager

LOB Manager

A

Buffer Manager

U ser A d d re s s S p a c e

D is k e n v iro n m e n t

Figure 7-2 A SELECT application and LOB materialization

As you can see in these cases, application B may require the complete LOB data. Therefore the private user address space may require, based on the size of the LOB, a huge amount of storage within its user address space. Application A uses a locator and may just use the space it needs for the chunks of data pointed to through the defined locator.

Chapter 7. Performance with LOBs

121

In the case where you have applications within a distributed environment and if they are selecting LOB values via the network, involving the DB2 DDF address space, or if you are using STORED PROCEDURES, in all these cases the selected LOB data will not be materialized within a DB2 data space. They will internally use LOCATORs and move the data, in small chunks, from DISK through the buffer pool and the DDF address space, or the STORED PROCEDURES address space.

INSERT Assume a scenario where your application A and B INSERT into your LOB table space. From the point of view of materialization, it will show a different picture. Now there are data spaces involved. The materialization in DB2 data spaces will occur if you are using LOCATORs, and it will not if you do not use them. Still, not using LOCATORS, like in application B, will cause you to use more storage in your private user address space and again depending on the size of your LOB, this could be quite storage consuming. See Figure 7-3. . Data Spaces User Address Space

DB2 DBM1 Address Space

A

Local Buffer Pool

User Address Space

Data Manager

LOB Manager

Buffer Manager

B

Disk environment

Figure 7-3 An INSERT application and LOB materialization

If your application will INSERT LOB values via the network (this implicates the use of DB2’s DDF address space), or it will use a STORED PROCEDURE, or converts from one CCSID to another, LOB materialization will occur in a DB2 data space.

DELETE and UPDATE As in the SELECT case, the DELETE case will also not materialize the LOB value: instead it just de-allocates the pages where the LOB is stored. Because the UPDATE case implicitly will INSERT and DELETE, the materialization of the LOB within the data space is the same as with the INSERT case.

122

Large Objects with DB2 for z/OS and OS/390

7.1.1 Using data spaces for LOBs If DB2 needs to materialize LOBs in storage they will be placed in a data space. So now we focus on the amount of storage used in data spaces for LOBs that need to be materialized. This depends upon a number of factors. They include the size of the LOBs, the quantity of LOBs in a statement that needs to be materialized, and if you use cursors to hold position in an application. As mentioned, LOBs are stored in data spaces for various situations. Again this will appear when a LOB is a parameter of a UDF, when it is needed to be converted, or when passing LOBs in a distributed environment and also when it has to be moved into a stored procedure. If you want to see the number of data spaces created by DB2 (for instance as a result of materialization), you can use the MVS console command: D A,xxxxDBM1 (xxxx = subsystem id)

As you can see in Example 7-1, the MVS console command will give you the number and the name of the data spaces DB2 has created. Example 7-1 MVS display command IEE115I 15.39.29 2002.067 ACTIVITY 481 JOBS M/S TS USERS SYSAS INITS ACTIVE/MAX VTAM OAS 00005 00038 00008 00031 00024 00008/00030 00019 DB2GDBM1 DB2GDBM1 IEFPROC NSW S A=0194 PER=NO SMC=000 PGN=N/A DMN=N/A AFF=NONE CT=223.764S ET=67.22.51 WUID=STC02493 USERID=STC WKL=SYSTEM SCL=SYSSTC P=1 RGP=N/A SRVR=NO QSC=NO ADDR SPACE ASTE=03EAB500 DSPNAME=DB2GD000 ASTE=4DA3AF80 DSPNAME=DB2GD001 ASTE=005C4080 DSPNAME=DB2GD002 ASTE=005C4100

If you may want to monitor the maximum storage used for LOBs in a data space, there is the IFCID 2 record for accounting and the IFCID 3 record for statistics. Both records contain the field QXSTLOBV, that provides the values. If you have activated the statistics trace with IFCID 2 and the accounting traces with IFCID 3 in your DB2 subsystem, you will find the values within DB2 PM accounting and statistics record printout. In the report you will find the values in the miscellaneous part. The values shown there are in kilobytes as you can see in the Example 7-2 printout. Example 7-2 DB2 PM accounting miscellaneous values MISCELLANEOUS AVERAGE ------------------- -------MAX STOR LOB VALUES 102.4K

TOTAL -------512021

You can find the values within DB2 PM statistics record printout. They are also pointed out in miscellaneous. The values shown there are in megabytes as you can see in Example 7-3. Example 7-3 DB2 PM statistics miscellaneous values MISCELLANEOUS -----------------------------BYPASS COL : 0.00 MAX SQL CASCAD LEVEL: 0.00

Chapter 7. Performance with LOBs

123

MAX STOR LOB VALUES :

1470.00

DB2 subsystem parameters for LOBs As mentioned DB2 will create data spaces as needed. The LOB Manager will keep track of the amount used and manages the space within the data space. There are limits and they are specified in subsystem parameters in DSNZPARM. You will find them on the installation panel DSNTIP7 as shown in Example 7-4. They are defined as a maximum storage per user for LOB values in kilobytes (LOBVALA) and as a maximum storage per DB2 subsystem for LOB values in megabytes (LOBVALS). In DB2 V7 both ZPARMs can be changed without stopping DB2, refer to -SET SYSPARM in DB2 UDB for OS/390 and z/OS Version 7 Command Reference, SC26-9934, for details.

__________________________________________________________________________________ DSNTIP7 ===> _

INSTALL DB2 - SIZES PANEL 2

Check numbers and reenter to change: 1 USER LOB VALUE STORAGE

===> 2048

Max storage per user for LOB values in kilobytes Max storage per system for LOB values in megabytes Maximum tokens at any time. 0-50

2 SYSTEM LOB VALUE STORAGE ===> 2048 3 MAXIMUM LE TOKENS

===> 20

PRESS: ENTER to continue RETURN to exit HELP for more information __________________________________________________________________________________ Figure 7-4 Panel DSNTIP7 showing storage LOB values Table 7-1 Two subsystem parameters Parameter

Macro

Panel

Values

LOBVALA

DSNSYSP

DSNTIP7

1-2097152 KB

LOBVALS

DSNSYSP

DSNTIP7

1-51200 MB

򐂰 The value LOBVALA field establishes an upper limit for the amount of storage each user can have for storing LOB values. The value specified gives the numbers of kilobytes. 򐂰 The value LOBVALS field establishes an upper limit for the amount of storage per system that can be used for storing LOB values. The value specified gives the numbers of megabytes. Because these definitions can have an impact on your operating system, specifically on paging and auxiliary storage, and depend on how heavy is the use of LOBs within your DB2 subsystem, the values of these settings should be chosen with care. Before setting the values of these new ZPARMs, you should consult your z/OS system programmer.

124

Large Objects with DB2 for z/OS and OS/390

If you run into a resource unavailable situation (SQLCODE -904) with data space pools, the reason codes you can expect are 00C900Dx (x = 1, 2 and 3). The reason codes mean respectively: user limit exceeded, system limit exceeded, and out of space condition in the data space.

7.2 Buffer pools and group buffer pools This section gives you an overview of buffer pool and group buffer pool considerations you have to be aware of when you are using large LOBs in production or non production environments.

Buffer pools Depending upon how you have planned to use LOBs in your shop, you will most likely want to put LOB data in separate buffer pools, so as to not interfere with data used by online transactions or other work. If the LOBs have their own buffer pools, you have the capability of setting different thresholds than with your other buffer pools, without affecting other work. For example, you may want to set the virtual pool sequential steal threshold to 100%, taking full advantage of the sequential access, knowing that this buffer pool is only used for LOB pages. Also minimizing the buffer pool wide deferred write threshold (DWQT) almost to 0, this will force to asynchronously write buffers out of the buffer pool. At the data set level, the vertical deferred write threshold (VDWQT), will force write upon a data set level. It is expressed as a percentage of the virtual pool size for a single data set. This value should always be less than DWQT. In regards to dealing with LOB buffer pools, all three are worthwhile looking into. So again, it is not only meaningful but also a good advise to have your LOB buffer pools isolated from your other ‘normal’ buffer pools in your environment. See Figure 7-5.

BP3 BP2

VPSEQT

Buffer Pool

Random l

Sequential

Buffer

Buffer Pool

Pool

BP20 Index

Buffer

DWQT Pool

Separate

BP0

System

BP22

BP12

BP10

BP1

VPSIZE

BP32K1

BP32K

VPSEQT

VDWQT DWQT VDWQT

Example of usual Buffer Pools

LOB Buffer Pools

Separation of LOB Buffer Pools Figure 7-5 Separation of LOB buffer pools

Chapter 7. Performance with LOBs

125

Considerations for data sharing The GBPCACHE SYSTEM option is allowed only for LOBs. For GBPCACHE SYSTEM page sets, the only pages that are written to the group buffer pool are LOB space map pages. All other data pages are written directly to disk, similar to GBPCACHE NONE page sets. SYSTEM is the default for LOB table spaces. See Figure 7-6.

D B 2 d a ta s h a rin g e n v iro n m e n t

G GB BP P CACHE ES S Y S TT E M fo fo rr LL O B B GR RO OU UP P BU U FF FF E R P O OO OL

C o u p lin g F a c ility

C o u p lin g F a c ility

ssnm DB M 1

ssnm D B M 1

B u ffe r P o o ls

B u ffe r P o o ls

Figure 7-6 LOB group buffer pool

As mentioned GBPCACHE SYSTEM is the default for a LOB table space and we recommended to use it. In a data sharing environment, if GBPCACHE CHANGED or GBPCACHE ALL is used instead for a LOG NO LOB table space, then a coupling facility failure could result in the table space being marked GRECP. Any kind of recovery of the table space will mark the LOB values as invalid and place the table space in the AUXW state. Again for LOB table spaces, choose GBPCACHE SYSTEM to avoid having large LOB values overwhelm the group buffer pool. Also, for LOB table spaces with the LOG NO attribute, GBPCACHE SYSTEM ensures that LOB values are written to DASD by commit time, thereby avoiding possible recovery problems caused by missing log data.

Storage estimate for caching LOB space maps Example 7-4 shows the formula for estimating storage in the Coupling Facility for group buffer pools that cache LOB space map data (GBPCACHE SYSTEM). Example 7-4 Calculation for LOB space maps Data_entries Data(MB) Dir_entries Dir(MB) GBP(MB)

126

= = = = =

(U * D / 10) * R Data_entries * P / 1024 Data_entries + (U * (HP + VP)) 1.1 * Dir_entries * 0.2 / 1024 Data(MB) + Dir(MB)

Large Objects with DB2 for z/OS and OS/390

RATIO

= MIN(Dir_entries / Data_entries, 255)

Coming up with the formula, there are different values used and they have to be explained. Starting with the Data_entries = (U * D / 10) * R: The value U is the estimated degree of data sharing. The value D is based upon the number of data pages written to disk per second and the value of 1/10 is the estimate of LOB system pages that are written for every LOB data page. The value R is the average page residency time in the group buffer pool in seconds. Note about the value D: Do not use the number of pages written to the group buffer pool as D. Instead it must be a count of distinct pages. This value can be determined by using field QBSTPWS from IFCID 0002. You can find it using the DB2 PM and looking into the statistics report (PAGES WRITTEN field). This will give you the right value. The Data(MB) = Data_entries * P / 1024: The value P is the page size (4, 8, 16, or 32) depending on the local buffer pool page size. At last the Dir_entries = Data_entries + (U * (HP + VP)): where HP The number of data pages defined for the hiperpool (the sum across all the members). VP The number of data pages defined for the virtual pool (the sum across all the data sharing members). Because the calculations are a little intricate, the following will step through a model. Assume that you have a two-member data sharing group for which you have determined the following information. See Example 7-5. Example 7-5 Calculation of group buffer pool The degree of data sharing is moderate (.7) There are 10 disk writes per second for across both members, peak rate The space map page is resident in the group buffer pool for 120 seconds The page size is 32 KB Member 1 is configured with a virtual pool of 20000 buffers and a hiperpool of 70000 buffers Member 2 is configured with a virtual pool of 10000 buffers and a hiperpool of 20000 buffers The calculation is as follows: Data_entries = ((.7 * 10)/ 10) * 120 = 84 Data(MB) = 84 * 32 / 1024 = 2.6 MB Dir_entries = 84 + (.7 * (90000 + 30000)) = 84084 Dir(MB) = 1.1 * 84084 * 0.2 / 1024 = 18.6 MB GBP(MB) = 2.6 MB + 18.6 MB = 21.2 MB RATIO = MIN (84084 / 84, 255) = 255

The above calculation indicates that the group buffer pool should be defined with an INITSIZE of 21.2 MB. The ratio is greater than the maximum value, which is not unusual with GBPCACHE(SYSTEM), so use the command ALTER GROUPBUFFERPOOL to change the ratio to 255.

7.3 LOBs performance considerations In this section we look at read and update performance, and provide recommendations. The measurements shown were implemented with DB2 V6 in 1999 and were included in DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351. They are shown here because they still provide meaningful comparisons. More measurements are currently under way with DB2 V7 and more current disk technology, the results will be included as soon as available.

Chapter 7. Performance with LOBs

127

7.3.1 Logging with LOBs To reduce the volume of logging, you can specify LOG NO in your CREATE LOB TABLESPACE statement. This suppresses writing redo records. There are no UNDO records for LOB updates (except for system pages, space map) even with LOG YES. LOBs always insert the new value at a different place and delete the old one at commit, marking the old space as free. To give you an idea of the amount of data being written to the log, look at Example 7-6 which shows the DSN1LOGP output for a LOB insert of a using LOG YES. Example 7-6 Inserting a 1 MB LOB with LOG YES URID(0002AB431314) LRSN(B76312AC4AE6) DBID(0130) OBID(000D) PAGE(00000001) TYPE( REDO ) SUBTYPE(UPDATE SPACE MAP) CLR(NO) PROCNAME(DSNISGSU) URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130) OBID(000D) PAGE(00000001) TYPE( REDO ) SUBTYPE(CURRENT LAST ENTRY IN SPACE MAP PAGE) CLR(NO) PROCNAME(DSNISGSU) URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130) OBID(000D) PAGE(00000002) TYPE( REDO ) SUBTYPE(FORMAT PAGE OR MODIFY SPACE MAP) CLR(NO) PROCNAME(DSNISGRT) URID(0002AB431314) LRSN(B76312AC4AE7) DBID(0130) OBID(000D) PAGE(00000002) TYPE( UNDO REDO ) SUBTYPE(INSERT IN A DATA PAGE) CLR(NO) PROCNAME(DSNISGRT) URID(0002AB431314) LRSN(B76312AC4AE8) DBID(0130) OBID(000D) PAGE(00000001) TYPE( UNDO REDO ) SUBTYPE(UPDATE SPACE MAP) CLR(NO) PROCNAME(DSNISGSU) LRSN(B76312AC4D71) DBID(0130) OBID(0010) TYPE(PAGE SET CONTROL) SUBTYPE(PAGE SET OPEN) URID(0002AB431314) LRSN(B76312AC4EF2) DBID(0130) OBID(0010) PAGE(00000005) TYPE( UNDO REDO ) SUBTYPE(ALLOC/DEALLOC OF SPACE IN LOB SPACE MAP PAGE) CLR(NO) PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EF3) DBID(0130) OBID(0010) PAGE(00000001) TYPE( UNDO REDO ) SUBTYPE(LOB HIGH LEVEL SPACE MAP PAGE UPDATE) CLR(NO) PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EF5) DBID(0130) OBID(0010) PAGE(00000005) TYPE( UNDO REDO ) SUBTYPE(ALLOC/DEALLOC OF SPACE IN LOB SPACE MAP PAGE) CLR(NO) PROCNAME(DSNOALLO) URID(0002AB431314) LRSN(B76312AC4EFC) DBID(0130) OBID(0010) PAGE(00000015) TYPE( REDO ) SUBTYPE(LOB MAP CHANGE) CLR(NO) PROCNAME(DSNOFLMP) URID(0002AB431314)

128

LRSN(B76312AC4EFD)

Large Objects with DB2 for z/OS and OS/390

DBID(0130)

OBID(0010) PAGE(00000015) TYPE( REDO ) SUBTYPE(LOB DATA PAGE CHANGE) CLR(NO) PROCNAME(DSNOLINS) URID(0002AB431314) LRSN(B76312AC4F0B) DBID(0130) OBID(0010) PAGE(00000016) TYPE( REDO ) SUBTYPE(LOB DATA PAGE CHANGE) CLR(NO) PROCNAME(DSNOLINS) [ LOB DATA PAGE CHANGE REPEATS n TIMES ] URID(0002AB431314) LRSN(B76312ACD1FE) OBID(0014) PAGE(00000003) TYPE( UNDO SUBTYPE(TYPE 2 INDEX UPDATE) CLR(NO) PROCNAME(DSNKINSL)

DBID(0130) REDO )

LRSN(B76312D1B66E) DBID(0130) OBID(0014) TYPE(PAGE SET CONTROL) SUBTYPE(PAGE SET WRITE)

The log entries marked as bold text, occurring n times where n depends on the size of your LOB value, represent the additional overhead for specifying LOG YES for the LOB table space. Every LOB DATA PAGE CHANGE log entry contains data of a single LOB data page with those values you insert into your LOB column. You have as many LOB DATA PAGE CHANGE log entries in your log as needed for storing your LOB value. You will not find the bold text log entries if you have created your LOB table space using LOG NO. This means that only system pages for the LOB value are written to the log, no LOB data is written to the log. The amount of data to be logged can grow rapidly when you use the LOG YES clause.

7.3.2 LOBs processing The maximum size of a LOB column is 2 GB. You can also use LOBs to store data, for example, long characters, which do not fit entirely within DB2’s largest page size of 32 KB. Although you can manipulate LOB columns like any other data type, there are a number of issues: 򐂰 They are subject to the same restrictions as long strings. Substrings of LOBs which are less than 256 bytes can be CAST as CHAR to avoid these restrictions. 򐂰 Acquiring buffers in a program to accommodate a large LOB can be difficult. Since LOBs are essentially long strings, you can use string functions to manipulate them and parts of them. For example, SUBSTR(LOB,1,200) will retrieve the first 200 bytes of the LOB. This can more easily be managed in an application program. LOBs are inserted, selected, deleted, and updated using standard SQL, although there are some special considerations: 򐂰 A LOB locator, which is a 4-byte value stored in a host variable, can be used to reference the LOB, and can be used wherever you would use a LOB. The locator essentially acts as a pointer to the LOB. For full details, refer to DB2 UDB for OS/390 and z/OS Version 7 Application Programming and SQL Guide, SC26-9933. 򐂰 The LOB update operation does not update in place, it performs a LOB delete followed by an insert. 򐂰 The LOB delete is a logical delete: it flags the space as available. Chapter 7. Performance with LOBs

129

7.3.3 LOBs read performance We have measured the efficiency of reading data from tables that contain LOB data. We created four different tables. The first was the reference case and consisted of a non-LOB 20 KB character column. The table space page size had to be 32 KB. Operations performed on this table provided the baseline against which we could compare the efficiency of LOB processing. See Figure 7-7.

LOBs read performance Select into :HV KB/sec

non-LOB 20 KB 0.025 (0.0006) 800.0

Select into locator

20 KB 0.027 (0.0008) 740.7

200 KB 0.117 (0.0023) 1,709.4

2 MB 0.867 (0.0175) 2,306.8

0.020 (0.0007)

0.029 (0.0007)

0.046 (0.0007)

All measurements in seconds Elapsed time is on the top line, CPU time shown in brackets Processor is LPAR of RX5, disks are RAMAC 2 Large LOBs can be processed (KB/sec) more efficiently than small For 2 MB LOBs, locators offer superior performance and lower CPU overhead: nearly 19 times faster elapsed 25 time less CPU required

We recommend you use LOB locators Figure 7-7 Read performance

The 20 KB LOB table was conceptually identical to the first one except that the 20 KB character column was defined as a CLOB. We created a LOB table space and auxiliary table and index to support this. For our LOB table spaces, we used a 32 KB page size, so that comparisons with the non-LOB table were reasonable. The other two tables were defined with 200 KB and 2 MB LOBs, respectively. When we retrieved LOB data into the host application, we repeatedly called SUBSTR to “walk” down the entire length of the column. We used the same technique to process LOB data with and without the use of LOB locators. Any differences in the performance of processing LOBs can, therefore, be attributed solely to the use of locators or the processing of LOBs into host variables. The diagram above shows the elapsed and CPU time taken to process non-LOB and LOB data of different lengths and shows the following results: 򐂰 LOB processing is more efficient for larger LOB columns. The time taken to process a select of 20 KB worth of LOB data into a host variable is 0.027 seconds. In other words DB2 can process 740.7 KB LOB data per second for a 20 KB LOB column. To select a 2 MB LOB into a host variable (that is, 100 times more data) does not take 100 times longer; rather, DB2 can process 2306.8 KB LOB data per second for a 2 MB LOB column.

130

Large Objects with DB2 for z/OS and OS/390

򐂰 The efficiency by which large LOBs can be processed is increased enormously by using LOB locators. With 20 KB of LOB data, there is only a small improvement in performance using LOB locators. At 2 MB we saw that it took nearly 19 times longer and used 25 times more CPU to select a LOB into a host variable rather than into a LOB locator. 򐂰 You get very consistent responses and CPU times reading LOB data using locator variables regardless of the size of the LOB column. These measurements lead us to recommend that you use LOB locators when selecting LOB data.

7.3.4 Comparing SQL accounting profiles There are good reasons for the use of LOBs, but there are also good reasons to consider using VARCHAR instead, if possible. Even when dealing with the wide variety of additional data types, such as audio, video or mixed text, if they are below 32 KB in size, you should consider the possibility of storing them into a VARCHAR column. The main reasons for still wanting to use VARCHAR or VARGRAPHIC, for your small (<32KB) multimedia objects, are that there is added complexity with LOBs, and that there are more SQL calls executed to process a LOB than when processing a VARCHAR column. As an example, we have setup an environment with a simple base table and its associated LOB. The LOB is 32 KB in size. On the other side we have a normal table that contains the same structure as the base table, except that it has a VARCHAR column. The VARCHAR column can also have 32 KB. For our comparison we now use two different batch application programs, both using host variables. One selects the LOB and the other selects the VARCHAR column into a host variable, as shown in Example 7-7. Example 7-7 Select into host variable EXEC SQL SELECT DOCUMENT INTO :LO-DOCUMENT FROM PAOLO.CLOB_BASE_TABLE WHERE DOCUMENT_NR = :LO-DOCUMENT-NR END-EXEC

We now look at the accounting trace output provided by DB2 PM and reported in Figure 7-8. The values shown for DB2 Class 2 elapsed time clearly indicates which strategy to use for your small (<32 KB) objects. Example 7-8 Accounting trace output SELECTING A LOB AVERAGE APPL(CL.1) DB2 (CL.2) ----------- ---------- ---------ELAPSED TIME 0.803155 0.557689 CPU TIME 0.033644 0.012224 _____________________________________ SELECTING A VARCHAR AVERAGE APPL(CL.1) DB2 (CL.2) ------------ ---------- ---------ELAPSED TIME 0.536118 0.284875 CPU TIME 0.061166 0.040178

Chapter 7. Performance with LOBs

131

7.3.5 LOBs write performance We also measured the performance of update, delete, and insert operations against LOB data. The scenario we used was the same as for LOB reads: one non-LOB table with a 20 KB CHAR as a control for the 20 KB LOB table, and tables with LOB sizes of 200 KB and 2 MB. The processor in an RX5 LPAR and LOG NO was specified for the LOB runs. We examined the cost of an insert operation using a host variable compared with using a LOB locator with different sized LOBs. We also measured the cost of a delete and an update. See Figure 7-8.

LOBs write performance NON-LOB 20 KB Insert with :hv

0.011 (0.0007)

Insert with locator Update (see notes) Delete

0.0036 (0.0008) 0.03 (0.001)

20 KB

200 KB

0.025 (0.0012) 0.028 (0.0012) 0.027 (0.0013) 0.024 (0.001)

0.219 (0.0033) 0.187 (0.0028) 0.223 (0.0033) 0.035 (0.001)

2 MB 2.029 (0.0194) 2.08 (0.0199) 2.229 0.021) 0.045 (0.001)

All measurements in seconds Elapsed time is on the top line, CPU time shown in brackets 2 MB insert takes about 2 seconds insert more efficient as length of LOB data increases Delete is very quick - logical delete Update is roughly the same as an insert internally consists of insert+delete so this makes sense

Figure 7-8 Write performance

LOB insert It is more expensive to process 20 KB of LOB data, even if a LOB locator is used, than processing an equivalent volume of non-LOB data. As with select processing, the greater the LOB length, the greater the efficiency of insert processing — to insert 100 times the data does not require 100 times the resource. The relative improvement of insert efficiency with respect to LOB size results, however, are much less dramatic than for select processing. The reason is that the cost of preformatting disk space to accommodate the newly inserted LOB increases with the size of the LOB and it accounts for most of the elapsed time. There is relatively little difference in insert performance if a LOB locator is used.

LOB update Internally a LOB update is equivalent to a LOB insert and a LOB delete. Most of the cost of a LOB update can, therefore, be attributed to the cost of the LOB insert.

132

Large Objects with DB2 for z/OS and OS/390

LOB delete A LOB delete is a logical delete, and so is relatively cheap and gives consistently good performance. The CPU cost was independent of the size of the LOB column (to 2MB). The larger the LOB column, the greater the number of space map pages that have to be updated to reflect the logically deleted row. There is, therefore, a relationship between elapsed time and LOB size for deletes. The larger the LOB column, the greater the efficiency of the delete operation. DB2 can delete 833 KB per second if the LOB is 20 KB in length. With a 2 MB LOB, DB2 can delete 45,511 KB per second — more than 50 times faster.

7.3.6 IFCIDs enhancements for LOBs If you have defined LOBs within your system, the need of tracking their performance and watching over all resources used by your application retrieving LOB data, can become very important to you. The use of LOBs may increase in the near future and getting to know some main counters within the IFCID environment can help you there. Because the IFCIDs are part of your DB2 traces, you may have to start them. By chance you already have them active in your DB2 system. There is a collection of the current IFCIDs providing information regarding LOBs. See Table 7-2.

Table 7-2 Current IFCID providing information regarding LOBs IFCID IFCID 2

IFCID 3

IFCID 18

IFCID 20

Fields

Description

DB2 statistics record QXSTLOBV

Maximum storage used for LOB values

QXCRATB

Number of CREATE AUXILIARY TABLE statements

QXHLDLOC

Number of HOLD LOCATOR statements

QXFRELOC

Number of FREE LOCATOR statements

DB2 accounting record QXSTLOBV

Maximum storage used for LOB values

QWACLRN

Number of log records written

QWACLRAB

Total number of bytes of log records written

Ends sequential scan, index scan or insert. The additional pages scanned in a LOB table space and for a count of the number of LOBs updated. Other fields in IFCID 18 are only applicable to the base table. QW0018PL

Additional pages scanned in a LOB table space

QW0018UL

Count of LOB data pages updated (either by an SQL INSERT or an SQL UPDATE)

Summary of page, row and LOB locks held and lock escalation. QW0020TP

Maximum number of page, row and LOB locks held

QW0020PL

Maximum number of either page, row or LOB locks held for the thread

QW0020F5

LOB table space

QW0020R3

LOB lock

Chapter 7. Performance with LOBs

133

IFCID IFCID 21

IFCID 23

Fields

Detail lock trace LOB lock type. QW0021ML

LOB lock type (value '30 'x)

QW0021KX

ID of resource for LOB locks

QW0021K6

Row ID

QW0021K7

Version number

Record utility start information QW0023PH

IFCID 24

IFCID 58

IFCID 62

CHECKLOB is a new phase for the CHECK LOB utility

Record utility start information QW0025PH

IFCID 44

CHECKLOB is a new phase for the CHECK LOB utility

Records utility object or phase change record QW0024PH

IFCID 25

Description

CHECKLOB is a new phase for the CHECK LOB utility

Records lock suspension. LOB lock type record utility start information QW0044ML

LOB lock type (value '30 'x)

QW0044KX

ID of resource for LOB locks

QW0044K6

Row ID

QW0044K7

Version number

End of SQL statement execution. Add fields for the additional pages scanned in a LOB table space and for a count of the number of LOBs updated. Other fields in IFCID 58 are only applicable to the base table. QW0058PL

Additional pages scanned in a LOB table space

QW0058UL

Count of LOB data pages updated (either by an SQL INSERT or an SQL UPDATE)

DDL (and other) execution statement start QW0062CX

Create auxiliary table. Value x'F2'

QW0062HL

Hold locator. Value x'CE'.

QW0062FL

Free locator. Value x'CF'

IFCID 105 Maps the internal DBID and OBID to the database and table space name. also maps the DBID and OBID of the LOB table space and index on the auxiliary table to the LOB table space name and name of the index on the auxiliary table. DDL (and other) execution statement start. IFCID 106 System Initialization Parameters QWP1LVA

Bytes for LOB values - per user

QWP1LVS

Bytes for LOB values - per system

IFCID 107 Open/close information. Also records open and close for LOB table spaces and indexes on auxiliary tables IFCID 141 Records grants and revokes.

134

Large Objects with DB2 for z/OS and OS/390

IFCID

Fields

Description

QW0140BS

For a LOB table space

QW0140BT

For an auxiliary table

IFCID 150 Lock information for a given agent. QW0150ML

LOB lock type (value '30 'x)

QW0150KX

ID of resource for LOB locks

QW0150K6

Row ID

QW0150K7

Version number

IFCID 172 Deadlock trace. QW0172MO

LOB lock type (value '30 'x)

QW0172KX

ID of resource for LOB locks

QW0172K6

Row ID

QW0172K7

Version number

IFCID 185 Data capture information deadlock trace. QW0185ST

For a LOB column, this is the data type of the LOB

QW0185LE

For a LOB column, this is the length of the indicator column

IFCID 196 Time-out trace. Lock types are not explicitly specified for this trace QW0196FR

This value represents a LOB lock '30 'X

QW0196KX

ID of resource for LOB locks

QW0196K6

Row ID

QW0196K7

Version number

IFCID 306 Records log records. Note that when LOG NO has been specified for a LOB table space, log records will not be written for the value of the LOB. IFCID 321 Trace record to trace the beginning of a force-at-commit. IFCID 322 Trace record to trace the end of a force-at-commit. QW0322NP

Number of pages written

Traces to use with LOBs 򐂰 IFCID 58 – QW0058PL - pages scanned in LOB table space – QW0058UL - pages updated in LOB table spaces 򐂰 IFCID 321 and 322 – 321 - begin force-at-commit – 322 - end force-at-commit – belong to the same classes as other I/O wait events: • Accounting classes 3 and 8 • Monitor classes 3 and 8 • Performance class 4

Chapter 7. Performance with LOBs

135

򐂰 QXSTLOBV – Maximum storage used for LOBs in data spaces – Value is in kilobytes for accounting – Value is in megabytes for statistics

7.3.7 LOBs recommendations 򐂰 Use LOBS for what they are intended — large (>32 KB) objects. On balance, we observed that the larger the LOB, the greater the efficiency. LOB insert (and therefore update) processing is expensive for small length LOB columns. We recommend, therefore, that you use LOBs only for the purpose for which they were intended — processing large objects over 32 KB in size. You must take into account that there is only 1 LOB per page; if you have specified a pagesize of 32 KB, and a row of 100 bytes, the overhead in dealing with each row in terms of I/O and CPU cannot be negligible. 򐂰 Use the correct page size For really large LOBs, page size must be 32 KB. Since DB2 places only one LOB per page, space can be wasted for small LOBs (less than the page size). Use 4 KB page for LOB tablespace to minimize wasted disk space and potentially much higher I/O time if both small and large LOBs exist, as it is found to be a very common situation. 򐂰 Use locators to manipulate LOBs. LOB locators process LOB data consistently and more efficiently than using host variables, and avoid application buffering problems. 򐂰 Do not use LOBS as a means of normalizing data. We do not recommend that you use LOBs as a technique to obtain performance improvements that normalization and proper physical design can achieve. Note that: – Any improvements are entirely application dependent. – At the physical design stage, you should have been able to identify the columns that are infrequently accessed and the relationship across the columns. Depending on the application, there may be a net benefit in placing some columns into its own base table, keyed on employee number and holding the 20 KB VARCHAR and possibly other infrequently accessed columns. – Do not use LOBs as a means to deactivate REDO logging. The additional cost, particularly when processing small LOBs, means you are likely to create a different bottleneck from the perceived one of logging that you were trying to solve. Use LOBs only for their intended purpose. 򐂰 Expect overhead when running LOAD You should be aware that the LOAD utility processes LOBs using insert mode processing. To estimate very roughly the elapsed time for LOAD, assume that the cost to load each row is equivalent to the cost of randomly inserting every row. 򐂰 Tune VSAM and GRS with data sharing In data sharing environments, we observed some contention during drop LOB table space and unusually high Coupling Facility (CF) CPU activity. We could attribute this to VSAM making use of GRS. When we specified GRS=STAR, these problems were eliminated. We recommend that you investigate the GRS setting in data sharing environments if you are observing high CF CPU activity.

136

Large Objects with DB2 for z/OS and OS/390

A

Appendix A.

Sample jobs output In this appendix we provide the output of sample jobs used to illustrate the examples included in this redbook. It is structured as follows: 򐂰 DDL for a LOB environment 򐂰 Sample of DB2 PM accounting trace output 򐂰 Using file reference variables

© Copyright IBM Corp. 2002

137

A.1 DDL for a LOB environment This sample is the output of the DDL that creates the environment for our sample programs. DSNTIAD

- SAMPLE DYNAMIC SQL PROGRAM 2.0

CREATE TABLESPACE CLOBBTS IN PAOLO USING STOGROUP PAOLOSG1 PRIQTY 1000 SECQTY 100 ERASE NO BUFFERPOOL BP12 COMPRESS YES SEGSIZE 64 LOCKSIZE ANY CLOSE NO DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE TABLE PAOLO.CLOB_BASE_TABLE ( DOCUMENT_NR CHAR(10) NOT , DESCRIPTION CHAR(32) NOT , ROW_ID ROWID NOT , DOCUMENT CLOB(1G) NOT ) IN PAOLO.CLOBBTS DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION

NULL NULL NULL NULL

COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE LOB TABLESPACE CLOBATS IN PAOLO USING STOGROUP PAOLOSG1 PRIQTY 50000 SECQTY 50000 ERASE NO BUFFERPOOL BP32K LOCKSIZE LOB CLOSE NO LOG NO DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE AUXILIARY TABLE PAOLO.CLOB_AUX_TABLE IN PAOLO.CLOBATS STORES PAOLO.CLOB_BASE_TABLE COLUMN DOCUMENT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION CREATE TYPE 2 UNIQUE INDEX PAOLO.CLOBAIX ON PAOLO.CLOB_AUX_TABLE USING STOGROUP PAOLOSG1 PRIQTY 100 SECQTY 10 FREEPAGE 10 PCTFREE 10 BUFFERPOOL BP11 CLOSE NO

138

Large Objects with DB2 for z/OS and OS/390

WITH DEFAULT WITH DEFAULT GENERATED ALWAYS WITH DEFAULT

DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION COMMIT DSNT400I SQLCODE = 000, SUCCESSFUL EXECUTION

A.2 Sample of DB2 PM accounting trace output Within this DB2 PM accounting output you will find special counters related to the LOB processing and performance. Look at the miscellaneous part of the output, you will find some highlighted counters that may be of interest to you when analyzing LOBs performance. 1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

DB2G N/P N/P DB2G V7

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

ORDER: PLANNAME SCOPE: MEMBER

2-14 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

PLANNAME: SAMPLE3 ELAPSED TIME DISTRIBUTION ---------------------------------------------------------------APPL !===========================================> 86% DB2 !==> 4% SUSP !=====> 10% AVERAGE -----------ELAPSED TIME NONNESTED STORED PROC UDF TRIGGER

APPL(CL.1) ---------24:30.4256 24:30.4256 0.000000 0.000000 0.000000

DB2 (CL.2) ---------3:29.97658 3:29.97658 0.000000 0.000000 0.000000

IFI (CL.5) ---------N/P N/A N/A N/A N/A

CPU TIME AGENT NONNESTED STORED PRC UDF TRIGGER PAR.TASKS

4.398020 4.398020 4.398020 0.000000 0.000000 0.000000 0.000000

1.660921 1.660921 1.660921 0.000000 0.000000 0.000000 0.000000

N/P N/A N/P N/A N/A N/A N/A

SUSPEND TIME AGENT PAR.TASKS

N/A N/A N/A

2:29.51135 2:29.51135 0.000000

N/A N/A N/A

NOT ACCOUNT. DB2 ENT/EXIT EN/EX-STPROC EN/EX-UDF DCAPT.DESCR. LOG EXTRACT.

N/A N/A N/A N/A N/A N/A

58.804310 101.20 0.00 0.00 N/A N/A

N/A N/A N/A N/A N/P N/P

GLOBAL CONTENTION L-LOCKS ------------------------------------L-LOCKS PARENT (DB,TS,TAB,PART) CHILD (PAGE,ROW) OTHER 1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

DB2G N/P N/P DB2G V7

CLASS 2 TIME DISTRIBUTION -----------------------------------------------------------CPU !> 1% NOTACC !==============> 28% SUSP !===================================> 71%

CLASS 3 SUSPENSIONS -------------------LOCK/LATCH(DB2+IRLM) SYNCHRON. I/O DATABASE I/O LOG WRITE I/O OTHER READ I/O OTHER WRTE I/O SER.TASK SWTCH UPDATE COMMIT OPEN/CLOSE SYSLGRNG REC EXT/DEL/DEF OTHER SERVICE ARC.LOG(QUIES) ARC.LOG READ STOR.PRC SCHED UDF SCHEDULE DRAIN LOCK CLAIM RELEASE PAGE LATCH NOTIFY MSGS GLOBAL CONTENTION COMMIT PH1 WRITE I/O ASYNCH IXL REQUESTS TOTAL CLASS 3

AVERAGE TIME -----------0.000000 0.000000 0.000000 0.000000

AV.EVENT -------0.00 0.00 0.00 0.00

AVERAGE TIME -----------1:28.359418 3.145753 3.145753 0.000000 0.000000 0.000000 58.006180 40.082702 0.078285 0.021327 17.741402 0.082464 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 2:29.511350

AV.EVENT -------55.80 71.20 71.20 0.00 0.00 0.00 15.20 1.00 0.20 0.20 13.20 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 142.20

HIGHLIGHTS -------------------------#OCCURRENCES : 5 #ALLIEDS : 5 #ALLIEDS DISTRIB: 0 #DBATS : 0 #DBATS DISTRIB. : 0 #NO PROGRAM DATA: 0 #NORMAL TERMINAT: 0 #ABNORMAL TERMIN: 5 #CP/X PARALLEL. : 0 #IO PARALLELISM : 0 #INCREMENT. BIND: 0 #COMMITS : 0 #ROLLBACKS : 5 #SVPT REQUESTS : 0 #SVPT RELEASE : 0 #SVPT ROLLBACK : 0 MAX SQL CASC LVL: 0 UPDATE/COMMIT : 0.20 SYNCH I/O AVG. : 0.044182

GLOBAL CONTENTION P-LOCKS ------------------------------------P-LOCKS PAGESET/PARTITION PAGE OTHER

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG ORDER: PLANNAME SCOPE: MEMBER

AVERAGE TIME -----------0.000000 0.000000 0.000000 0.000000

PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

AV.EVENT -------0.00 0.00 0.00 0.00

2-15 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

PLANNAME: SAMPLE3 SQL DML AVERAGE TOTAL -------- -------- -------SELECT 0.00 0 INSERT 0.20 1 UPDATE 0.00 0 DELETE 0.00 0 DESCRIBE

0.00

0

SQL DCL TOTAL -------------- -------LOCK TABLE 0 GRANT 0 REVOKE 0 SET CURR.SQLID 0 SET HOST VAR. 247 SET CUR.DEGREE 0

SQL DDL CREATE DROP ALTER ---------- ------ ------ -----TABLE 0 0 0 CRT TTABLE 0 N/A N/A DCL TTABLE 0 N/A N/A AUX TABLE 0 N/A N/A INDEX 0 0 0 TABLESPACE 0 0 0

LOCKING AVERAGE ---------------------- -------- ---TIMEOUTS 0.00 DEADLOCKS 0.00 ESCAL.(SHARED) 0.00 ESCAL.(EXCLUS) 0.00 MAX PG/ROW LOCKS HELD 0.60 LOCK REQUEST 87.60

Appendix A. Sample jobs output

139

DESC.TBL PREPARE OPEN FETCH CLOSE

0.00 0.00 0.00 0.00 0.00

0 0 0 0 0

DML-ALL

0.20

1

NORMAL TERM. AVERAGE --------------- -------NEW USER 0.00 DEALLOCATION 0.00 APPL.PROGR. END 0.00 RESIGNON 0.00 DBAT INACTIVE 0.00 RRS COMMIT 0.00

SET RULES SET CURR.PATH SET CURR.PREC. CONNECT TYPE 1 CONNECT TYPE 2 SET CONNECTION RELEASE CALL ASSOC LOCATORS ALLOC CURSOR HOLD LOCATOR FREE LOCATOR DCL-ALL

DATA CAPTURE ----------------IFI CALLS MADE RECORDS CAPTURED LOG RECORDS READ ROWS RETURNED RECORDS RETURNED DATA DESC. RETURN TABLES RETURNED DESCRIBES

1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

TOTAL -------0 0 0 0 0 0

AVERAGE -------N/P N/P N/P N/P N/P N/P N/P N/P

0 0 0 0 0 0 0 0 0 0 0 0 247

DATABASE STOGROUP SYNONYM VIEW ALIAS PACKAGE PROCEDURE FUNCTION TRIGGER DIST TYPE TOTAL RENAME TBL COMMENT ON LABEL ON

ABNORMAL TERM. ----------------APPL.PROGR. ABEND END OF MEMORY RESOL.IN DOUBT CANCEL FORCE

TOTAL -------N/P N/P N/P N/P N/P N/P N/P N/P

TOTAL -------5 0 0 0

DATA SHARING ------------------GLOBAL CONT RATE(%) FALSE CONT RATE(%) L-LOCKS XES RATE(%) LOCK REQ - PLOCKS UNLOCK REQ - PLOCKS CHANGE REQ - PLOCKS LOCK REQ - XES UNLOCK REQ - XES CHANGE REQ - XES SUSPENDS - IRLM SUSPENDS - XES SUSPENDS - FALSE INCOMPATIBLE LOCKS NOTIFY MSGS SENT

DB2G N/P N/P DB2G V7

0 0 0 0 0 N/A 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 N/A N/A N/A N/A 0 0 N/A N/A

0 0 0 0

0

0

IN DOUBT -------------APPL.PGM ABEND END OF MEMORY END OF TASK CANCEL FORCE

AVERAGE -------N/C N/C N/C 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

TOTAL -------N/A N/A N/A 0 0 0 0 0 0 0 0 0 0 0

UNLOCK REQUEST QUERY REQUEST CHANGE REQUEST OTHER REQUEST LOCK SUSPENSIONS IRLM LATCH SUSPENSIONS OTHER SUSPENSIONS TOTAL SUSPENSIONS

TOTAL -------0 0 0 0

86.80 0.00 0.00 0.00 54.80 0.00 0.00 54.80

DRAIN/CLAIM -------------DRAIN REQUESTS DRAIN FAILED CLAIM REQUESTS CLAIM FAILED

AVERAGE -------0.00 0.00 1.60 0.00

QUERY PARALLELISM ---------------------------MAXIMUM MEMBERS USED MAXIMUM DEGREE GROUPS EXECUTED RAN AS PLANNED RAN REDUCED ONE DB2-COORDINATOR = NO ONE DB2-ISOLATION LEVEL ONE DB2-DCL TEMPORARY TABLE SEQUENTIAL-CURSOR SEQUENTIAL-NO ESA SORT SEQUENTIAL-NO BUFFER SEQUENTIAL-ENCLAVE SERVICES MEMBER SKIPPED (%) DISABLED BY RLF REFORM PARAL-CONFIG REFORM PARAL-NO BUF

AVERAGE -------N/A N/A 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 N/C 0.00 0.00 0.00

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

ORDER: PLANNAME SCOPE: MEMBER

----

----

2-16 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

PLANNAME: SAMPLE3 STORED PROCEDURES ----------------CALL STATEMENTS ABENDED TIMED OUT REJECTED

AVERAGE -------0.00 0.00 0.00 0.00

LOGGING ------------------LOG RECORDS WRITTEN TOT BYTES WRITTEN

AVERAGE SU -----------CPU AGENT NONNESTED STORED PRC UDF TRIGGER PAR.TASKS

CLASS 1 -------------50588.20 50588.20 50588.20 0.00 0.00 0.00 0.00

BP0 --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE

140

AVERAGE -------12.60 7051.60

TOTAL -------0 0 0 0 TOTAL -------63 35258

UDF --------EXECUTED ABENDED TIMED OUT REJECTED

ROWID ------------DIRECT ACCESS INDEX USED TS SCAN USED

CLASS 2 -------------19104.60 19104.60 19104.60 0.00 0.00 0.00 0.00

AVERAGE TOTAL -------- -------98.56 N/A 0.00 N/A 41.80 209 0.00 0 0.00 0 0.00 0

AVERAGE -------0.00 0.00 0.00 0.00

TOTAL -------0 0 0 0

TRIGGERS ----------------STATEMENT TRIGGER ROW TRIGGER SQL ERROR OCCUR

AVERAGE -------0.00 0.00 0.00

TOTAL -------0 0 0

DYNAMIC SQL STMT -----------------REOPTIMIZATION NOT FOUND IN CACHE FOUND IN CACHE IMPLICIT PREPARES PREPARES AVOIDED STMT INVALID (MAX) STMT INVALID (DDL)

AVERAGE -------0.00 0.00 0.00 0.00 0.00 0.00 0.00

BP11 --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE

Large Objects with DB2 for z/OS and OS/390

AVERAGE -------0.00 0.00 0.00

RID LIST ------------------USED FAIL-NO STORAGE FAIL-LIMIT EXCEEDED TOTAL -------0 0 0 0 0 0 0

AVERAGE TOTAL -------- -------42.86 N/A 0.00 N/A 1.40 7 0.00 0 0.60 3 0.20 1

TOTAL -------0 0 0

AVERAGE -------0.00 0.00 0.00

TOTAL -------0 0 0

MISCELLANEOUS AVERAGE TOTAL ------------------- -------- ------MAX STOR LOB VALUES 102.4K 512021

BP12 --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE

AVERAGE -------- ----100.00 N/C 0.80 0.00 0.80 0.00

SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED 1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

0.00 0.00 0.00 0.20 0.60 0.00 0.00 0.00 0.00 0.00

0 0 0 1 3 0 0 0 0 0

DB2G N/P N/P DB2G V7

SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

0.80 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

4 0 0 0 0 0 0 0 0 0

SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

ORDER: PLANNAME SCOPE: MEMBER

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2-17 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

PLANNAME: SAMPLE3 TOT4K --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

AVERAGE -------96.82 0.00 44.00 0.00 1.40 0.20 0.80 0.00 0.00 0.20 0.60 0.00 0.00 0.00 0.00 0.00

TOTAL -------N/A N/A 220 0 7 1 4 0 0 1 3 0 0 0 0 0

SAMPLE3 -----------------TYPE

VALUE -----------------PACKAGE

LOCATION COLLECTION ID PROGRAM NAME

DB2G COLL000 SAMPLE3

OCCURRENCES SQL STMT - AVERAGE SQL STMT - TOTAL STOR PROC EXECUTED UDF EXECUTED USED BY STOR PROC USED BY UDF USED BY TRIGGER SUCC AUTH CHECK 1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

SAMPLE3 -----------------ELAP-CL7 TIME-AVG CPU TIME AGENT PAR.TASKS SUSPENSION-CL8 AGENT PAR.TASKS NOT ACCOUNTED AVG.DB2 ENTRY/EXIT DB2 ENTRY/EXIT

5 49.60 248 0 0 0 0 0 0

DB2G N/P N/P DB2G V7

BP32K --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

AVERAGE -------97.79 0.00 3278.00 0.00 3210.20 0.00 72.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

TIMES -----------3:29.976568 1.660909 1.660909 0.000000 2:29.432129 2:29.432129 0.000000 58.883529 99.20 496

CPU SERVICE UNITS AGENT PAR.TASKS

TOTAL -------N/A N/A 16390 0 16051 0 363 0 0 0 0 0 0 0 0 0

TOTAL --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

SAMPLE3 -----------------LOCK/LATCH SYNCHRONOUS I/O OTHER READ I/O OTHER WRITE I/O SERV.TASK SWITCH ARCH.LOG(QUIESCE) ARCHIVE LOG READ STORED PROC SCHED UDF SCHEDULE DRAIN LOCK CLAIM RELEASE PAGE LATCH NOTIFY MESSAGES GLOBAL CONTENTION TOTAL CL8 SUSPENS.

19104.60 19104.60 0.00

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

AVERAGE TIME -----------1:28.359418 3.145753 0.000000 0.000000 57.926959 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 2:29.432129 PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

ORDER: PLANNAME SCOPE: MEMBER

*** GRAND TOTAL *** ELAPSED TIME DISTRIBUTION ---------------------------------------------------------------APPL !============================================> 89% DB2 !=> 3% SUSP !====> 8% AVERAGE -----------ELAPSED TIME NONNESTED STORED PROC UDF TRIGGER

APPL(CL.1) ---------14:54.4918 14:54.4918 0.000000 0.000000 0.000000

DB2 (CL.2) ---------1:35.51485 1:35.51485 0.000000 0.000000 0.000000

IFI (CL.5) ---------N/P N/A N/A N/A N/A

CPU TIME AGENT NONNESTED STORED PRC UDF TRIGGER

2.002305 2.002305 2.002305 0.000000 0.000000 0.000000

0.756192 0.756192 0.756192 0.000000 0.000000 0.000000

N/P N/A N/P N/A N/A N/A

AVERAGE -------97.77 0.00 3322.00 0.00 3211.60 0.20 73.40 0.00 0.00 0.20 0.60 0.00 0.00 0.00 0.00 0.00 AVG.EV -----55.80 71.20 0.00 0.00 15.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 142.00

TO -----

16 16

TIME/E -------1.58 0.04

3.86

1.05

2-18 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

CLASS 2 TIME DISTRIBUTION -----------------------------------------------------------CPU !> 1% NOTACC !==============> 28% SUSP !===================================> 71%

CLASS 3 SUSPENSIONS -------------------LOCK/LATCH(DB2+IRLM) SYNCHRON. I/O DATABASE I/O LOG WRITE I/O OTHER READ I/O OTHER WRTE I/O SER.TASK SWTCH UPDATE COMMIT OPEN/CLOSE SYSLGRNG REC EXT/DEL/DEF OTHER SERVICE

AVERAGE TIME -----------40.163372 1.429888 1.429888 0.000000 0.000000 0.000000 26.434087 18.286116 0.035584 0.010161 8.064274 0.037953

AV.EVENT -------25.36 32.36 32.36 0.00 0.00 0.00 12.73 1.00 0.09 0.18 6.00 5.45

HIGHLIGHTS -------------------------#OCCURRENCES : 11 #ALLIEDS : 11 #ALLIEDS DISTRIB: 0 #DBATS : 0 #DBATS DISTRIB. : 0 #NO PROGRAM DATA: 4 #NORMAL TERMINAT: 6 #ABNORMAL TERMIN: 5 #CP/X PARALLEL. : 0 #IO PARALLELISM : 0 #INCREMENT. BIND: 0 #COMMITS : 7

Appendix A. Sample jobs output

141

PAR.TASKS

0.000000

0.000000

N/A

SUSPEND TIME AGENT PAR.TASKS

N/A N/A N/A

1:08.02735 1:08.02735 0.000000

N/A N/A N/A

NOT ACCOUNT. DB2 ENT/EXIT EN/EX-STPROC EN/EX-UDF DCAPT.DESCR. LOG EXTRACT.

N/A N/A N/A N/A N/A N/A

26.731315 49.27 0.00 0.00 N/A N/A

N/A N/A N/A N/A N/P N/P

GLOBAL CONTENTION L-LOCKS ------------------------------------L-LOCKS PARENT (DB,TS,TAB,PART) CHILD (PAGE,ROW) OTHER 1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

SELECT INSERT UPDATE DELETE

0.00 0.09 0.00 0.00

0 1 0 0

DESCRIBE DESC.TBL PREPARE OPEN FETCH CLOSE

0.00 0.00 0.09 0.00 0.00 0.00

0 0 1 0 0 0

DML-ALL

0.18

2

SQL DCL

142

AVERAGE -------N/P N/P N/P N/P N/P N/P N/P N/P

TOTAL

LOCK TABLE GRANT REVOKE SET CURR.SQLID SET HOST VAR. SET CUR.DEGREE SET RULES SET CURR.PATH SET CURR.PREC. CONNECT TYPE 1 CONNECT TYPE 2 SET CONNECTION RELEASE CALL ASSOC LOCATORS ALLOC CURSOR HOLD LOCATOR FREE LOCATOR DCL-ALL

SQL DDL

TOTAL -------N/P N/P N/P N/P N/P N/P N/P N/P

0 0 0 0 248 0 0 0 0 0 0 0 0 0 0 0 0 0 248

CREATE

ALTER 0 N/A N/A N/A 0 0 0 0 N/A N/A N/A N/A 0 0 N/A N/A

TOTAL RENAME TBL COMMENT ON LABEL ON

0 0 0 0

1

0

ORDER: PLANNAME

2-19 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

LOCKING

AVERAGE

TIMEOUTS DEADLOCKS ESCAL.(SHARED) ESCAL.(EXCLUS) MAX PG/ROW LOCKS HELD LOCK REQUEST UNLOCK REQUEST QUERY REQUEST CHANGE REQUEST OTHER REQUEST LOCK SUSPENSIONS IRLM LATCH SUSPENSIONS OTHER SUSPENSIONS TOTAL SUSPENSIONS

IN DOUBT TOTAL -------------- -------APPL.PGM ABEND 0 END OF MEMORY 0 END OF TASK 0 CANCEL FORCE 0

AVERAGE -------N/C N/C N/C 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

AV.EVENT -------0.00 0.00 0.00 0.00

---------------------- -------- ---------0 N/A N/A N/A 1 0 0 0 0 0 0 0 0 0 0 0

TOTAL -------N/A N/A N/A 0 0 0 0 0 0 0 0 0 0 0

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

Large Objects with DB2 for z/OS and OS/390

DROP

0 0 0 0 0 0 0 0 0 0 0 N/A 0 0 0 0

ABNORMAL TERM. TOTAL ----------------- -------APPL.PROGR. ABEND 5 END OF MEMORY 0 RESOL.IN DOUBT 0 CANCEL FORCE 0

AVERAGE TIME -----------0.000000 0.000000 0.000000 0.000000

PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

TABLE CRT TTABLE DCL TTABLE AUX TABLE INDEX TABLESPACE DATABASE STOGROUP SYNONYM VIEW ALIAS PACKAGE PROCEDURE FUNCTION TRIGGER DIST TYPE

DATA SHARING ------------------GLOBAL CONT RATE(%) FALSE CONT RATE(%) L-LOCKS XES RATE(%) LOCK REQ - PLOCKS UNLOCK REQ - PLOCKS CHANGE REQ - PLOCKS LOCK REQ - XES UNLOCK REQ - XES CHANGE REQ - XES SUSPENDS - IRLM SUSPENDS - XES SUSPENDS - FALSE INCOMPATIBLE LOCKS NOTIFY MSGS SENT

#ROLLBACKS : 5 #SVPT REQUESTS : 0 #SVPT RELEASE : 0 #SVPT ROLLBACK : 0 MAX SQL CASC LVL: 0 UPDATE/COMMIT : 0.08 SYNCH I/O AVG. : 0.044182

GLOBAL CONTENTION P-LOCKS ------------------------------------P-LOCKS PAGESET/PARTITION PAGE OTHER

---------- ------ ------ ------

NORMAL TERM. AVERAGE TOTAL --------------- -------- -------NEW USER 0.00 0 DEALLOCATION 0.55 6 APPL.PROGR. END 0.00 0 RESIGNON 0.00 0 DBAT INACTIVE 0.00 0 RRS COMMIT 0.00 0

DB2G N/P N/P DB2G

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 70.45

ORDER: PLANNAME SCOPE: MEMBER

-------------- --------

LOCATION: GROUP: MEMBER: SUBSYSTEM:

AV.EVENT -------0.00 0.00 0.00 0.00

0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 1:08.027347

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

*** GRAND TOTAL *** SQL DML AVERAGE TOTAL

1

AVERAGE TIME -----------0.000000 0.000000 0.000000 0.000000

DB2G N/P N/P DB2G V7

DATA CAPTURE ----------------IFI CALLS MADE RECORDS CAPTURED LOG RECORDS READ ROWS RETURNED RECORDS RETURNED DATA DESC. RETURN TABLES RETURNED DESCRIBES

ARC.LOG(QUIES) ARC.LOG READ STOR.PRC SCHED UDF SCHEDULE DRAIN LOCK CLAIM RELEASE PAGE LATCH NOTIFY MSGS GLOBAL CONTENTION COMMIT PH1 WRITE I/O ASYNCH IXL REQUESTS TOTAL CLASS 3

0.00 0.00 0.00 0.00 0.64 44.36 41.27 0.00 0.82 0.00 24.91 0.00 0.00 24.91

DRAIN/CLAIM AVERAGE -------------- -------- ---DRAIN REQUESTS 0.09 DRAIN FAILED 0.00 CLAIM REQUESTS 3.18 CLAIM FAILED 0.00

QUERY PARALLELISM ---------------------------MAXIMUM MEMBERS USED MAXIMUM DEGREE GROUPS EXECUTED RAN AS PLANNED RAN REDUCED ONE DB2-COORDINATOR = NO ONE DB2-ISOLATION LEVEL ONE DB2-DCL TEMPORARY TABLE SEQUENTIAL-CURSOR SEQUENTIAL-NO ESA SORT SEQUENTIAL-NO BUFFER SEQUENTIAL-ENCLAVE SERVICES MEMBER SKIPPED (%) DISABLED BY RLF REFORM PARAL-CONFIG REFORM PARAL-NO BUF PAGE: REQUESTED FROM: TO: INTERVAL FROM:

AVERAGE -------N/A N/A 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 N/C 0.00 0.00 0.00

----

2-20 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2

DB2 VERSION: V7

SCOPE: MEMBER

*** GRAND TOTAL *** STORED PROCEDURES AVERAGE ----------------- -------CALL STATEMENTS 0.00 ABENDED 0.00 TIMED OUT 0.00 REJECTED 0.00 LOGGING ------------------LOG RECORDS WRITTEN TOT BYTES WRITTEN

AVERAGE SU -----------CPU AGENT NONNESTED STORED PRC UDF TRIGGER PAR.TASKS

TOTAL -------0 0 0 0

AVERAGE -------13.73 6169.82

CLASS 1 -------------23031.36 23031.36 23031.36 0.00 0.00 0.00 0.00

TOTAL -------151 67868

UDF --------EXECUTED ABENDED TIMED OUT REJECTED

1

LOCATION: GROUP: MEMBER: SUBSYSTEM: DB2 VERSION:

DB2G N/P N/P DB2G V7

TOTAL -------0 0 0 0

TRIGGERS ----------------STATEMENT TRIGGER ROW TRIGGER SQL ERROR OCCUR

AVERAGE -------0.00 0.00 0.00

TOTAL -------0 0 0

DYNAMIC SQL STMT -----------------REOPTIMIZATION NOT FOUND IN CACHE FOUND IN CACHE IMPLICIT PREPARES PREPARES AVOIDED STMT INVALID (MAX) STMT INVALID (DDL)

AVERAGE -------0.00 0.00 0.00 0.00 0.00 0.00 0.00

AVERAGE -------0.00 0.00 0.00

RID LIST ------------------USED FAIL-NO STORAGE FAIL-LIMIT EXCEEDED TOTAL -------0 0 0 0 0 0 0

BP11 AVERAGE TOTAL --------------------- -------- -------BPOOL HIT RATIO (%) 42.86 N/A HPOOL HIT RATIO (%) 0.00 N/A GETPAGES 1.40 7 GETPAGES-FAILED 0.00 0 BUFFER UPDATES 0.60 3 SYNCHRONOUS WRITE 0.20 1 SYNCHRONOUS READ 0.80 4 SEQ. PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.00 0 PAGES READ ASYNCHR. 0.00 0 HPOOL WRITES 0.00 0 HPOOL WRITES-FAILED 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 HPOOL READS 0.00 0 HPOOL READS-FAILED 0.00 0

TOTAL -------N/A N/A 300 0 36 1 4 0 0 1 3 0 0 0 0 0

ALL PROG -----------------TYPE

VALUE -----------------PACKAGE

LOCATION COLLECTION ID PROGRAM NAME

'BLANK' 'BLANK' ALL PROG 7 36.14 253 0

BP32K --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED ALL PROG -----------------ELAP-CL7 TIME-AVG CPU TIME AGENT PAR.TASKS SUSPENSION-CL8 AGENT PAR.TASKS NOT ACCOUNTED AVG.DB2 ENTRY/EXIT DB2 ENTRY/EXIT

AVERAGE -------97.79 0.00 3278.00 0.00 3210.20 0.00 72.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

TIMES -----------2:30.091872 1.187084 1.187084 0.000000 1:46.842769 1:46.842769 0.000000 42.062020 73.14 512

AVERAGE -------0.00 0.00 0.00

TOTAL -------0 0 0

BP12 AVERAGE --------------------- -------- ----BPOOL HIT RATIO (%) 100.00 HPOOL HIT RATIO (%) N/C GETPAGES 0.80 GETPAGES-FAILED 0.00 BUFFER UPDATES 0.80 SYNCHRONOUS WRITE 0.00 SYNCHRONOUS READ 0.00 SEQ. PREFETCH REQS 0.00 LIST PREFETCH REQS 0.00 DYN. PREFETCH REQS 0.00 PAGES READ ASYNCHR. 0.00 HPOOL WRITES 0.00 HPOOL WRITES-FAILED 0.00 PAGES READ ASYN-HPOOL 0.00 HPOOL READS 0.00 HPOOL READS-FAILED 0.00 PAGE: REQUESTED FROM: TO: INTERVAL FROM: TO:

ORDER: PLANNAME SCOPE: MEMBER

AVERAGE -------97.67 0.00 50.00 0.00 6.00 0.17 0.67 0.00 0.00 0.17 0.50 0.00 0.00 0.00 0.00 0.00

TOTAL -------0 0 0

MISCELLANEOUS AVERAGE TOTAL ------------------- -------- ------MAX STOR LOB VALUES 46547.36 512021

DB2 PERFORMANCE MONITOR (V7) ACCOUNTING REPORT - LONG

*** GRAND TOTAL *** TOT4K --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

OCCURRENCES SQL STMT - AVERAGE SQL STMT - TOTAL STOR PROC EXECUTED

ROWID ------------DIRECT ACCESS INDEX USED TS SCAN USED

CLASS 2 -------------8697.73 8697.73 8697.73 0.00 0.00 0.00 0.00

BP0 AVERAGE TOTAL --------------------- -------- -------BPOOL HIT RATIO (%) 98.96 N/A HPOOL HIT RATIO (%) 0.00 N/A GETPAGES 48.17 289 GETPAGES-FAILED 0.00 0 BUFFER UPDATES 4.83 29 SYNCHRONOUS WRITE 0.00 0 SYNCHRONOUS READ 0.00 0 SEQ. PREFETCH REQS 0.00 0 LIST PREFETCH REQS 0.00 0 DYN. PREFETCH REQS 0.17 1 PAGES READ ASYNCHR. 0.50 3 HPOOL WRITES 0.00 0 HPOOL WRITES-FAILED 0.00 0 PAGES READ ASYN-HPOOL 0.00 0 HPOOL READS 0.00 0 HPOOL READS-FAILED 0.00 0

AVERAGE -------0.00 0.00 0.00 0.00

TO: 03/18/02 23:19:3

TOTAL -------N/A N/A 16390 0 16051 0 363 0 0 0 0 0 0 0 0 0

TOTAL --------------------BPOOL HIT RATIO (%) HPOOL HIT RATIO (%) GETPAGES GETPAGES-FAILED BUFFER UPDATES SYNCHRONOUS WRITE SYNCHRONOUS READ SEQ. PREFETCH REQS LIST PREFETCH REQS DYN. PREFETCH REQS PAGES READ ASYNCHR. HPOOL WRITES HPOOL WRITES-FAILED PAGES READ ASYN-HPOOL HPOOL READS HPOOL READS-FAILED

ALL PROG -----------------LOCK/LATCH SYNCHRONOUS I/O OTHER READ I/O OTHER WRITE I/O SERV.TASK SWITCH ARCH.LOG(QUIESCE) ARCHIVE LOG READ STORED PROC SCHED UDF SCHEDULE DRAIN LOCK

AVERAGE TIME -----------1:03.113870 2.246966 0.000000 0.000000 41.481932 0.000000 0.000000 0.000000 0.000000 0.000000

2-21 NOT SPECIFIED NOT SPECIFIED 03/18/02 22:36:2 03/18/02 23:19:3

AVERAGE -------97.78 0.00 2781.67 0.00 2681.17 0.17 61.17 0.00 0.00 0.17 0.50 0.00 0.00 0.00 0.00 0.00 AVG.EV -----39.86 50.86 0.00 0.00 11.14 0.00 0.00 0.00 0.00 0.00

TO -----

16 16

TIME/E -------1.58 0.04

3.72

Appendix A. Sample jobs output

143

UDF EXECUTED USED BY STOR PROC USED BY UDF USED BY TRIGGER SUCC AUTH CHECK

0 0 0 0 0

CPU SERVICE UNITS AGENT PAR.TASKS

13654.29 13654.29 0.00

CLAIM RELEASE PAGE LATCH NOTIFY MESSAGES GLOBAL CONTENTION TOTAL CL8 SUSPENS.

0.000000 0.000000 0.000000 0.000000 1:46.842769

0.00 0.00 0.00 0.00 101.86

1.04

ACCOUNTING REPORT COMPLETE

A.3 Using file reference variables You can use the file reference variable as input to the extender UDFs. In this section we have included an example of loading a video clip from a file on the workstation which is stored as a BLOB column at the host. The table and columns are enabled to handle AVI extenders. This is shown in Example A-1. Example: A-1 Enabled Extender tables get extender status in mmdbsys.employee EXTENDER TABLESPACE COLUMN -------- ----------------------------------- ------------------DB2VIDEO MMDBSYS.EMPLOYAD,,MMDBSYS.EMPLOYVL, VIDEO DB2IMAGE MMDBSYS.DB2EXTND,,MMDBSYS.DB2LXTND, PICTURE DB2AUDIO MMDBSYS.EMPLOYAA,,MMDBSYS.EMPLOYAL, AUDIO DMB0025I: Table "mmdbsys.employee" is enabled for "3" extenders. db2ext =>

We tested the insert using DB2 Connect command center. The results from the connect statement are: Database Connection Information Database server = DB2 OS/390 7.1.1 SQL authorization ID = PAOLOR3 Local database alias = DB2G

We issued an INSERT from a client file to a BLOB column at the host using a relative file name. Example A-2 shows the command and the results. Example: A-2 Use of file reference --------------- Command Entered --------------------------------------------insert into mmdbsys.employee (name, video) values ('XT video',DB2VIDEO(CURRENT SERVER, 'nitecry.avi', 'AVI', 1,'test')) ----------------------------------------------------------------------------DB20000I The SQL command completed successfully

If you specify a relative file name, the extenders will use the directory specifications in various client and server environment variables as a search path to resolve the file name. In our example the path to the files stored on the Window NT workstation was setup with the installation of the AVI extender software.

144

Large Objects with DB2 for z/OS and OS/390

Environment variable description 򐂰 DB2MMPATH It is used to resolve source file name for store, retrieve, and update operations. Our path is: c:\dmb\samples 򐂰 DB2MMSTORE It is used to resolve target file name for store and update operations. Our path is: c:\dmb\samples 򐂰 DB2MMTEMP It is used to resolve file name for operations that create temporary files. Our path is: c:\dmb\tmp These parameters are set up in My Computer->Settings->Control Panel->Environment.

Appendix A. Sample jobs output

145

146

Large Objects with DB2 for z/OS and OS/390

B

Appendix B.

Unicode implementation DB2 V7 supports UNICODE conversion from the host encoding scheme to UNICODE 367. DB2 utilizes OS/390 services for the code conversion. This appendix describes the procedures to install and activate the 1140 (Euro English) to UNICODE 367 conversion. A full description of the OS/390 UNICODE support can be found at: http://www.ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

Here we show as an example only the CCSID 1140 to 367 conversion installation.

© Copyright IBM Corp. 2002

147

B.1 Generate the conversion table The JCL in Example B-1 creates the conversion table and output to DD SYSIMG. Please note that the ER option must be supplied to the CONVERSION command. The ER option contains two characters, representing the technique-search-order ; respectively, E for enforced subset and R for roundtrip. Example: B-1 JCL to create the conversion table //CUNJIUTL JOB (POK,999),'UNICODE',MSGLEVEL=(1,1),MSGCLASS=T, // CLASS=A,NOTIFY=&SYSUID //******************************************************************* //* * //* LICENSED MATERIALS - PROPERTY OF IBM * //* * //* 5647-A01 * //* * //* (C) COPYRIGHT IBM CORP. 2000 * //* * //* STATUS = HUNI2A0 * //* * //******************************************************************* //* * //* IMAGE GENERATOR * //* CHANGED TO USE ER !!! * //******************************************************************* //CUNMIUTL EXEC PGM=CUNMIUTL //STEPLIB DD DSN=SYS1.LINKLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //TABIN DD DISP=SHR,DSN=SYS1.SCUNTBL //SYSIMG DD DSN=PAOLOR1.IMAGES(CUNIMG01),DISP=SHR //SYSIN DD * /******************************************** * INPUT STATEMENTS FOR THE IMAGE GENERATOR * ********************************************/ CASE NORMAL; CONVERSION 1140,367,ER; CONVERSION 367,1140,ER;

/* ENABLE TOUPPER AND TOLOWER */ /* EBCDIC EURO -> UNICODE */ /* UNICODE -> EBCDIC EURO */

/*

The output of the job is shown in Example B-2. Please note that the job creates the conversion table indirectly, since no direct conversion from 1140 to 367 is currently available. Example: B-2 Output from the generation job CUN1000I OS/390 SUPPORT FOR UNICODE VERSION 2.8.0 CUN1001I PROCESSING STARTED ON 06/25/2001 AT 14:21:56 Source Listing ----+----1----+----2----+----3----+----4----+----5----+----6---1 2 /******************************************** 3 * INPUT STATEMENTS FOR THE IMAGE GENERATOR * 4 ********************************************/ 5 6 CASE NORMAL; /* ENABLE TOUPPER AND TOLOWER */

148

Large Objects with DB2 for z/OS and OS/390

7 8 9 10

CONVERSION 1140,367,ER; CONVERSION 367,1140,ER;

/* EBCDIC EURO -> UNICODE */ /* UNICODE -> EBCDIC EURO */

Statement Report --+----1----+----2----+----3----+----4----+----5----+----6---1 CONVERSION 1140,367,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 01140-00367-ER */ /* 01140-01200-R using CUNRN5PH */ /* 01200-00367-X using CUNXPGB0 */ 2 CONVERSION 367,1140,ER; CUN0999I NO TABLE FOUND. GENERATING A FORCED INDIRECT /* 00367-01140-ER */ /* 00367-01200-R using CUNRB0PG */ /* 01200-01140-E using CUNEPHN5 */ 3 CASE NORMAL; /* to-upper using CUNANUUP */ /* to-lower using CUNANULO */ CUN1014I CUN1015I CUN1016I CUN1017I CUN1002I

INPUT READ 10 RECORDS STATEMENTS PROCESSED 3 STATEMENTS FLAGGED 0 GENERATED IMAGE SIZE 98 PAGES PROCESSING ENDED. HIGHEST RETURN CODE WAS 0

B.2 Activate the UNICODE table These are the steps to activate the UNICODE conversion table: 򐂰 Copy the conversion table (the highlighted module in Example B-1 on page 148) into SYS1.PARMLIB as CUNIMG01. 򐂰 Create a new member in SYS1.PARMLIB, member name CUNUNI01. The contents of CUNUNI01 are shown in Example B-3. Example: B-3 Contents of CUNUNI01 in SYS1.PARMLIB /**********************************************************/ /* */ /* CUNUNIXX - UNICODE CONVERSION CONTROL PARAMETERS */ /* */ /**********************************************************/ /* ESTABLISH A NEW ENVIRONMENT */ /**********************************************************/ /* REQUIRED KEYWORD REALSTORAGE */ /* MAXIMAL USED PAGES OF REAL STORAGE, MIN=0 MAX=524287 */ /* WHERE 0 MEANS NO EXPLICITE LIMIT (=524287) */ /**********************************************************/ REALSTORAGE 51200; /* E.G. 200 MB */ /**********************************************************/ /* REQUIRED KEYWORD IMAGE WITH */ /* REQUIRED PARAMETER: MEMBER NAME */ /* THIS MEMBER MUST BE PLACED IN A DATA SET FROM THE */ /* LOGICAL PARMLIB CONCATENATION (DEF'D IN LOADXX) */ /**********************************************************/ IMAGE CUNIMG01;

Appendix B. Unicode implementation

149

򐂰 Activate the new UNICODE table with command T UNI=01. The SYSLOG output is reported in Example B-4. Example: B-4 Activate the new UNICODE table T UNI=01 IEE252I MEMBER CUNUNI01 FOUND IN SYS1.PARMLIB 537 CUN2036I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) WILL BE DELETED. ARE YOU SURE? (Y/N) R 537,Y IEE600I REPLY TO 537 IS;Y CUN2038I INACTIVE CONVERSION ENVIRONMENT (UNICODE1) 282 WAS SUCCESSFULLY DELETED CUN2020I START LOADING CONVERSION IMAGE CUNIMG01 IEE252I MEMBER CUNIMG01 FOUND IN SYS1.PARMLIB CUN2022I LOADING CONVERSION IMAGE CUNIMG01 FINISHED: 401434 BYTES LOADED CUN2034I SET UNI COMMAND SUCCESSFULLY EXECUTED IEE536I UNI VALUE 01 NOW IN EFFECT

򐂰 Verify the UNICODE table activation with command D UNI,ALL. The SYSLOG output is reported in Example B-5. Example: B-5 UNI=ALL output in SYSLOG D UNI,ALL IEF196I IEF237I 3D30 ALLOCATED TO SYS00108 IEF196I IEF285I SYS1.LINKLIB KEPT IEF196I IEF285I VOL SER NOS= Z01RE1. CUN3000I 14.26.27 UNI DISPLAY 303 ENVIRONMENT: CREATED 06/22/2001 AT 10.02.01 MODIFIED 06/25/2001 AT 14.24.25 IMAGE CREATED 06/25/2001 AT 14.21.56 SERVICE: CUNMCNV CUNMCASE STORAGE: ACTIVE 98 PAGES INACTIVE 98 PAGES SINCE 06/25/2001 AT 14.24.25 LIMIT 51200 PAGES CASECONV: NORMAL CONVERSION: 01140-00367-ER 00367-01140-ER

150

Large Objects with DB2 for z/OS and OS/390

C

Appendix C.

Recent maintenance In this appendix we provide feedback on the implementation of the environment and products. It consists of a list of the opened requirements and the APARs recommended. 򐂰 Open requirements 򐂰 APARs

© Copyright IBM Corp. 2002

151

C.1 Requirements In Table C-1 we briefly describe the current restrictions encountered during our tests. Their solutions have been submitted as requirements with the Fully Integrated Tool Set Marketing/Office requirements (FITSMO) application, a set of Lotus Notes databases, and a Web Interface, used to gather requirements from external customers and business partners to improve products and policies. Table C-1 Requirements Number

Area

Description

MR0322024656

Ease of use

A program needs to be created to run on the ‘connected workstation’ to push the data up to the mainframe and insert it if you want to use the file reference variables.

MR0322023113

Explain and auxiliary tables

If an SQL statement visually accesses the base table (when selecting a LOB column), the explain output does not show any access to the auxiliary table nor the auxiliary index. Only the access path for the base table is shown. This may confuse those people who do not know that a base table and a LOB column is accessed by the SQL statement.

MR032605920

Acceptability

AUXW state after recovery - only reset by REPAIR utility and program. Needs to optionally reset at CHECK LOB time, with LOB values set to null. Also EXCEPTION tables like for RI would help.

MR0322021925

REORG space reclaim

REORG to do normal REORG, i.e. release space (remove deleted objects physically).

MR0322023637

Accessibility

LOB Load to support > 32 KB (maybe in a different format, n times 32,760 bytes would build one row).

MR0322023750

Usability

Partitioning of LOB table spaces would correspond to base table design.

MR0322024927

Usability

POSSTR enhancement to support not just the 1st but the nth occurrence of a string.

MR0322026439

Performance

Compression on LOBs (maybe only CLOBs and DBCLOBs).

C.2 LOB related maintenance In Table C-2 we briefly describe the DB2 LOB related APARS that we came across during our project with DB2 V7. Table C-2 DB2 V7 LOB related APARs APAR

Area

Description

PQ42864

Runstats

PTF UQ54223 available.

PQ54511

DB2 Extenders

UQ63855 being prepared. It Enables AVI Extenders with DB2 V7.

PQ46137

152

CHECK DATA with LOBs

Large Objects with DB2 for z/OS and OS/390

It fixes several important problems and provides better messages. PTF UQ64673 is available for DB2 V7; UQ64672 for DB2 V6.

APAR

Area

Description

PQ53571

DB2 striped data set enhancements

UQ64357 for DB2 V7. VSAM striping is allowed on table space and index space with 4 KB page size, and disallowed on those with 8, 16, or 32 KB page sizes. Preformat quantity is changed with striping.

PQ58306

LOB utilities

UQ64118. It fixes errors due to inconsistent LOB data after forward log recovery of a LOB table space defined LOG NO.

PQ59820

LOB support

LOAD performance and bad page number. PTF UQ65857is available for DB2 V7, UQ65856 is available for DB2 V6.

(1)

Note: (1) The APAR is due to a remote possibility of loss of integrity at write time for stripes involving pages larger than 4 KB. If you want to use striping with LOBs this restriction is not penalizing performance and is not in contrast with the general recommendation of using 32 KB pages for large LOBs. Our opinion, supported by the lab measurements, is that the very small penalty in a little extra CPU time when concatenating 4 KB pages is really negligible, while I/O time remains the same. So it might be worthwhile changing to 4 KB pages if you want to exploit striping, even for large LOBs. Furthermore, a large majority of LOB users seem to have either small LOBs or a large variation in size, therefore justifying 4 KB pages in any case.

Appendix C. Recent maintenance

153

154

Large Objects with DB2 for z/OS and OS/390

D

Appendix D.

Additional material This redbook refers to additional material that can be downloaded from the Internet as described below.

Locating the Web material The Web material associated with this redbook is available in softcopy on the Internet from the IBM Redbooks Web server. Point your Web browser to: ftp://www.redbooks.ibm.com/redbooks/SG246571

Alternatively, you can go to the IBM Redbooks Web site at: ibm.com/redbooks

Select the Additional materials and open the directory that corresponds with the redbook form number, SG24-6571.

Using the Web material The additional Web material that accompanies this redbook includes the following files: File name LOBDDL.TXT BIND.TXT SAMPLE1.TXT SAMPLE1.JCL SAMPLE2.TXT SAMPLE2.JCL SAMPLE3.TXT SAMPLE3.JCL SAMPLE4.TXT SAMPLE4.JCL

© Copyright IBM Corp. 2002

Description DDL for creating the LOB environment needed by the samples BIND PACKAGE and BIND PLAN for sample programs Sample program for inserting LOB data using one large host variable JCL to run program SAMPLE1 Sample program for inserting LOB data using one large host variable and a single locator chain JCL to run program SAMPLE2 Sample program for inserting LOB data using one large host variable and two locator chains JCL to run program SAMPLE3 Sample program for unloading LOB data using one large host variable JCL to run program SAMPLE4

155

SAMPLE5.TXT SAMPLE5.JCL SAMPLE6.TXT SAMPLE6.JCL SAMPLE7.TXT SAMPLE7.JCL

Sample program for unloading LOB data using locators and a host variable of 1 MB JCL to run program SAMPLE5 Sample program for deleting a special part of a LOB using locators JCL to run program SAMPLE6 Sample program for updating a special part of a LOB using locators JCL to run program SAMPLE7

All the above files are contained in: LOBSAMPLES.zip Zipped Code Samples

System requirements for downloading the Web material The following system configuration is recommended: Hard disk space: Operating System: Processor: Memory:

8 MB minimum Windows 95 or NT or 2000 Intel 386 or higher 16 MB

How to use the Web material Create a subdirectory (folder) on your workstation, and unzip the contents of the Web material zip file into this folder.

156

Large Objects with DB2 for z/OS and OS/390

Abbreviations and acronyms AIX

Advanced Interactive eXecutive from IBM

DSC

dynamic statement cache, local or global

APAR

authorized program analysis report

DTT

declared temporary tables

ARM

automatic restart manager

EA

extended addressability

ASCII

American National Standard Code for Information Interchange

EBCDIC

extended binary coded decimal interchange code

BLOB

binary large object

ECS

enhanced catalog sharing

CCA

client configuration assistant

ECSA

extended common storage area

CCSID

coded character set identifier

EDM

CD

compact disk

environment descriptor management

CEC

central electronics complex

ERP

enterprise resource planning

CF

coupling facility

ESA

Enterprise Systems Architecture

CFCC

coupling facility control code

ESS

Enterprise Storage Server

CFRM

coupling facility resource management

ETR

CLI

call level interface

external throughput rate, an elapsed time measure, focuses on system capacity

CLOB

character large object

FDT

functional track directory

CLP

command line processor

FTP

File Transfer Program

CPU

central processing unit

GB

gigabyte (1,073,741,824 bytes)

CSA

common storage area

GBP

group buffer pool

CTT

created temporary table

GRS

global resource serialization

DAD

document access definition

GUI

graphical user interface

DASD

direct access storage device

HA

Host adapter

DB2 PM

DB2 performance monitor

HFS

Hierarchical File System

DBAT

database access thread

HPJ

high performance Java

DBCLOB

double byte character large object

I/O

input/output

DBD

database descriptor

IBM

International Business Machines Corporation

DBID

database identifier

ICF

integrated catalog facility

DBMS

database management system

ICF

integrated coupling facility

DBRM

database request module

ICMF

internal coupling migration facility

DCL

data control language

IFCID

DDCS

distributed database connection services

instrumentation facility component identifier

IFI

instrumentation facility interface

DDF

distributed data facility

IPLA

IBM Program Licence Agreement

DDL

data definition language

IRLM

internal resource lock manager

DLL

dynamic load library manipulation language

IRWW

IBM Relational Warehouse Workload

DML

data manipulation language

ISPF

DNS

domain name server

interactive system productivity facility

DRDA

distributed relational database architecture

ISV

independent software vendor

© Copyright IBM Corp. 2002

157

ITR

internal throughput rate, a processor time measure, focuses on processor capacity

RS

read stability

RSM

Relational Resource Manager

RTS

real time statistics

RVA

RAMAC Virtual Array

ITSO

International Technical Support Organization

IVP

installation verification process

SDK

software developers kit

JDBC

Java Database Connectivity

SMIT

System Management Interface Tool

JFS

journaled file systems

SVL

IBM Silicon Valley Laboratory

JIT

Just in time (Java compiler)

TCB

Task control block

JNI

Java Native Interface

USS

UNIX System Services

JVM

Java Virtual Machine

WAS

WebSphere Application Service

KB

kilobyte (1,024 bytes)

WLM

Workload Manager

LCU

logical control unit

LOB

large object

LPAR

Logical Partition

LPL

logical page list

LRECL

logical record length

LRSN

log record sequence number

LVM

logical volume manager

MB

megabyte (1,048,576 bytes)

MSM

Multidimensional Storage Manager

NPI

non partitioning index

NVS

Non Volatile Storage

ODB

object descriptor in DBD

ODBC

Open Data Base Connectivity

OLAP

Online Analytical Processing

OS/390

Operating System/390

PAV

Parallel Access Volume

PDS

partitioned data set

PIB

parallel index build

PSID

pageset identifier

PSP

preventive service planning

PTF

program temporary fix

PUNC

possibly uncommitted

QBIC

query by image content

QMF

Query Management Facility

RACF

Resource Access Control Facility

RBA

relative byte address

RECFM

record format

RID

record identifier

ROT

rule of thumb

RR

repeatable read

RRS

resource recovery services

RRSAF

resource recovery services attach facility

158

Large Objects with DB2 for z/OS and OS/390

Abbreviations and acronyms

159

160

Large Objects with DB2 for z/OS and OS/390

Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM Redbooks For information on ordering these publications, see “How to get IBM Redbooks” on page 162. 򐂰 DB2 for z/OS and OS/390 Version 7 Using the Utilities Suite, SG24-6289 򐂰 DB2 for z/OS and OS/390 Version 7 Performance Topics, SG24-6129 򐂰 DB2 UDB Server for OS/390 and z/OS Version 7 Presentation Guide, SG24-6121 򐂰 DB2 UDB Server for OS/390 Version 6 Technical Update, SG24-6108 򐂰 DB2 UDB for OS/390 Version 6 Performance Topics, SG24-5351 򐂰 DB2 for z/OS Application Programming Topics, SG24-6300

Other resources These publications are also relevant as further information sources: 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Installation Guide, GC26-9936 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Command Reference, SC26-9934 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Messages and Codes, GC26-9940 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Utility Guide and Reference, SC26-9945 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Programming Guide and Reference for Java, SC26-9932 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Administration Guide, SC26-9931 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Application Programming and SQL Guide, SC26-9933 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Release Planning Guide, SC26-9943 򐂰 DB2 UDB for OS/390 and z/OS Version 7 SQL Reference, SC26-9944 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Text Extender Administration and Programming, SC26-9948 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Data Sharing: Planning and Administration, SC26-9935 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Image, Audio, and Video Extenders, SC26-9947 򐂰 DB2 UDB for OS/390 and z/OS Version 7 ODBC Guide and Reference, SC26-9941 򐂰 DB2 UDB for OS/390 and z/OS Version 7 XML Extender Administration and Reference, SC26-9949 򐂰 DB2 UDB for OS/390 and z/OS Version 7 Diagnosis Guide and Reference, LY37-3740 򐂰 DB2 UDB Replication Guide and Reference Version 7, SC26-9920

© Copyright IBM Corp. 2002

161

Referenced Web sites These Web sites are also relevant as further information sources: 򐂰 DB2 for z/OS and OS/390 http://ibm.com/software/data/db2/os390/

򐂰 DB2 for z/OS and OS/390 Version 7 books http://www.ibm.com/software/data/db2/os390/v7books.html

򐂰 UNICODE http://ibm.com/servers/s390/os390/bkserv/latest/v2r10unicode.html

򐂰 DB2 Image, Audio and Video Formats http://www.ibm.com/software/data/db2/extenders/imgfmt.htm

򐂰 Developing Java applications using DB2 Image and Audio Extenders http://www7b.boulder.ibm.com/dmdd/library/techarticle/cox/0201cox.html

򐂰 XML Extender WIZARD http://www.ibm.com/software/data/db2/extenders/xmlext/downloads.html

򐂰 DAD examples http://www.ibm.com/software/data/pubs/papers/db2webservices/db2webservices.pdf

򐂰 Building an XML application, writing a DTD http://www.ibm.com/developerworks/library/buildappl/writedtd.html

򐂰 z/OS UNIX System Services http://www.s390.ibm.com/products/oe

How to get IBM Redbooks You can order hardcopy Redbooks, as well as view, download, or search for Redbooks at the following Web site: ibm.com/redbooks

You can also download additional materials (code samples or diskette/CD-ROM images) from that site.

IBM Redbooks collections Redbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.

162

Large Objects with DB2 for z/OS and OS/390

Index Numerics 00C900Dx 125

A ACHKP 113 adding a LOB column 30 adding a ROWID 69 ALTER GROUPBUFFERPOOL 127 application programming 3 ASCII 15 Audio Extender 75 aux check pending 113 aux table 18, 95 AUX WARNING state 24 auxiliary column 11, 18 auxiliary index 18, 28 DDL 29 auxiliary LOB table 11 auxiliary table 11 auxiliary warning 105 AUXW 24, 105 AVGSIZE 101

B backup and recovery 4 base table 11, 17, 20 Binary Large Objects 8 BIT 97 BLOB 8–9 browse an image 80 buffer pools 23, 25, 125

C caching LOB space maps 126 CAST 45 CCSID 8, 16 Character Large Objects 8 check constraints 59 CHECK DATA 105 CHECK-pending 105 CHKP 105 choosing a page size 25 chunk 60 client server solution 2 CLOB 8 COALESCE 45 collating 15 COLTYPE 92 compression 22, 59 CONCAT 44 concatenating two strings using locators 51 COPY 97

© Copyright IBM Corp. 2002

creating a LOB table space 22 creating a new table with ROWID 69 creating auxiliary table 28 creating LOBs 19 DDL statement 20 CS 56 CURRENT RULES 3, 22, 31 impact on CREATE and DROP 31 impact on cursors fetching LOBs 33 CURRENT RULES DB2 33 CURRENT RULES STD 33 cursor stability 56

D D UNI,ALL 150 DAD 86, 88 data conversion 14 Data Manager 19 data propagation 60 data sharing 4, 126 data space size per system 3 data space size per user 3 data spaces for LOBs 123 database management system xvii DB2 PM 123 DB2EXT.CLP 77 DB2IMAGE UDT 79 DBCLOB 8–9 DBMS xvii DBSNDB06 93 deferred write threshold 125 delete 133 deleting a LOB 25 deleting part of a LOB 52 DFSMS 27 displaying data spaces 123 displaying LOB objects 30 DM 19 DMBENVAR DD 76 DMBWLM1 76 document access definition 86 Document Type Definition 88 Double Byte Character Large Objects 8 DpropR 60 DSN8DLPL 94 DSNDB06 4 DSNTIP7 124 DSNZPARM 124 DSSIZE 27 DTD 88 DWQT 125

E ENABLE COLUMN 76

163

enabling extenders 77 encoding systems 14 ER option 148 exclusive LOB lock 62 Extended Markup Language 9 extenders 6

F FOR BIT DATA 9 FREE LOCATOR 47 FREESPACE 101

G GBPCACHE 27 GBPCACHE SYSTEM 27, 126 GENERATED ALWAYS 20 GENERATED BY DEFAULT 21, 30 GRS 136

H HOLD LOCATOR 49 host variable 10 HURBA 93

I IFCIDs 133 Image Copy 24 Image Extender 75 INCURSOR 94 INITSIZE 127 insert 132 inserting LOBs using a host-variable 35 using locators 36 intent exclusive 65 intent-share lock 63 internal resource lock manager 67 IRLM 67 IS 63 ISO-10646 15 ISOLATION (UR) 25, 64 IX 65

J Java stored procedures 4

L large objects 6–7 LENGTH 92 LENGTH2 92 LIKE 46 list 58 list prefetch 58 LOAD 94 LOAD utility loading LOBs 34

164

Large Objects with DB2 for z/OS and OS/390

loading a LOB column 58 LOB column 18 LOB database 20 LOB functions 44 LOB indicator 21 LOB locators 8 LOB lock 62 LOB lock serialization 64 LOB Manager 19 LOB manipulation 52 LOB map page 61 LOB materialization 120 LOB table space 17 DDL statement 22 LOBM 19 LOBs 8 allocation 26 delete 133 insert 132 performance considerations 127 processing 129 read performance 130 recommendations 136 update 132 write performance 132 LOBVALA 3, 124 LOBVALS 3, 124 locators 9, 46 COBOL syntax 11 concatenation 39 examples 50 precompiler conversion 11 types 11 when to use 49 locators across multiple units of work 49 locators and expressions 50 locators and materialization 50 locking 25, 62, 67 locks 59 locks with DELETE 65 locks with INSERT 65 locks with reads 63 locks with UPDATE 67 LOCKSIZE LOB 25 LOCKSIZE TABLESPACE 25 logging 3, 23, 128 logical data design 3 Logical Page List 110 LPL 110

M maintenance 2 map page 61 mass delete 65–66 mass delete statements 64 materialization 9, 120 DELETE and UPDATE 122 INSERT 122 SELECT 121 misusing LOBs 3

MMDBSYS 78 MVS considerations 3

N NOCOPYPEND 95 normalizing data 136 NULL 20

O object orientation 6 occurrence of a string 57 ORGRATIO 100–101 OS/390 Conversion Services 16

P padding 15 page size 25, 153 pageset structure 61 partitioned base table 18 partitioning 22 partitions 18 POSSTR 9, 45 PQ42864 100, 152 PQ46137 105 PQ49173 152 PQ51347 153 PQ51847 152 PQ53571 153 PQ54511 76 PQ59820 95, 153 PRIQTY 26

R RBDP 113 read performance 130 rebuild pending status 113 REBUILD-pending 105 RECLENGTH 93 recommendations for LOBs 136 record identifier 20 RECOVER-pending 105 Redbooks Web site 162 Contact us xix REDO logging 136 referring to a block of data inside of a CLO 51 REPORT TABLESPACESET 99 REPORT utility 22 RID 20 ROWID 8, 11, 18, 20, 68 adding the column 30 assigning 20 GENERATED BY DEFAULT 30

S SECQTY 26 SELECT SUBSTR 56 -SET SYSPARM 124

SHRLEVEL 98 singleton 23 singleton delete 65 S-LOB 25 S-LOB lock 59 space map pages 24 spanning pages 19 SQL accounting 131 SQL functions with LOBs 44 SQLCODE -171 11 SQLCODE -355 23 SQLCODE -423 49 SQLCODE -747 21 SQLCODE -767 29 SQLCODE -803 31 SQLCODE -904 24, 27, 125 STAR 136 STATUS 93 values X and I 93 store an image 80 STORES 28 striping 153 SUBSTR 9, 45, 130 SUBSTR(LOB,1,200) 129 summary of considerations 2 SYSIBM.SYSAUXRELS 92–93 SYSIBM.SYSCOLUMNS 92 SYSIBM.SYSJARCLASS_SOURCE 94 SYSIBM.SYSJARDATA 94 SYSIBM.SYSLOBSTATS 93 SYSIBM.SYSSTRINGS 15 SYSIBM.SYSTABLEPART 92–93 SYSIBM.SYSTABLES 93 SYSTABLEPART 93

T table space partition sizes 27 table space scans 19 TABLESTATUS 93 values L and P 93 TBNAME 92 text extenders 6 traces 135 triggers 6 TRUNCATE 97 TS 27

U UCS-2 16 UCS-4 16 UDFs 6 UDTs 6 undo log records 24 UNICODE 16 UNICODE 367 147 UNICODE support 15, 147 UNIX Systems Services 83 UNLOAD 96

Index

165

unload parts of LOB using locators 57 unloading a LOB 54 unloading a LOB using a host variable 54 unloading an entire LOB using locators 55 UPDATE 24, 132 updating a specific part of a LOB 53 UQ52836 84 UQ54223 152 UQ63855 76, 152 UQ64118 153 UQ64357 153 UQ64672 152 UQ64673 105, 152 UQ65856 153 UQ65857 153 user defined functions 6 user defined types 6 USS 76 UTF-16 16 UTF-8 16

V VALUE 45 VARCHAR 6 VARGRAPHIC 6 VDWQT 125 version number 28 vertical deferred write threshold 125 video extender 75

W WLM Application Environment 83 write performance 132

X X-LOB 25 X-LOB lock 59, 62 XML 7 UDFs 89 UDTs 89 XML and LOBs 84 XML collection 88 XML COLUMN 84, 86 XML Extender 81 XML EXTENDER UDFs 82 XML security 89 XML Toolkit 83 XML UDFs 81

166

Large Objects with DB2 for z/OS and OS/390

Large Objects with DB2 for z/OS and OS/390

(0.2”spine) 0.17”<->0.473” 90<->249 pages

Back cover

®

Large Objects with DB2 for z/OS and OS/390 Define LOBs, see how they work, and see how to store them

The amount of data to be managed by workstations, middleware systems and mainframes, is growing day by day. The times are changing, and so the requirements for a database management system (DBMS) are changing.

Manage LOBs in operational environments

With the incorporation of object-orientation into DB2 for OS/390, you can define new data types and functions. Some of the data objects you want to model may well be very large and complex. The foundation of object-relational extension, introduced with DB2 for OS/390 Version 6 has two major functions. One is based on support for large objects (LOB), and the other is support for user defined functions, user defined distinct types, and triggers. These large objects can contain text documents, images or even movies, and can be stored directly in the DBMS with sizes up to 2 gigabytes per object and 4.000 terabytes per LOB column.

Use LOBs in applications and DB2 Extenders

Normally, large objects are used and manipulated through Graphical User Interfaces from a workstation. So with the implementation of LOBs we can exploit the functionalities of the DB2 family and the capacity that DB2 for z/OS provides today. The introduction of these new data types implies some changes in the administration processes and programming techniques. In this IBM Redbook we describe the new data types, and provide some useful information on how to design and implement LOBs. We also offer examples of their use, programming considerations, and the new processes which are necessary for administration and maintenance.

INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information: ibm.com/redbooks SG24-6571-00

ISBN 0738425397