Veritas Volume Manager 1

VERITAS Volume Manager 4.0 Administration ES-310 Student Guide With Instructor Notes

Sun Microsystems, Inc. UBRM05-104 500 Eldorado Blvd. Broomfield, CO 80021 U.S.A. Revision D

December 19, 2003 10:15 am

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Sun, Sun Microsystems, the Sun logo, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java, Sun BluePrints, Sun Enterprise, SunOS, and SunSolve are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. and other countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc. UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd. Adobe is a registered trademark of Adobe Systems, Incorporated. PostScript is a trademark or a registered trademark of Adobe Systems, Incorporated, which may be registered in certain jurisdictions. ORACLE is a registered trademark of Oracle Corporation. The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledges the pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry. Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees who implement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements. U.S. Government approval might be required when exporting the product. RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) and FAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a). DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BE USED FOR REFERENCE PURPOSES IN CONJUNCTION WITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONE TRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOT RECOMMENDED. Export Control Classification Number (ECCN): 19 December 2001

Please Recycle

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés. Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution, et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit, sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a. Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licencié par des fournisseurs de Sun. Sun, Sun Microsystems, le logo Sun, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java, Sun BluePrints, Sun Enterprise, SunOS, et SunSolve sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. aux Etats-Unis et dans d’autres pays. Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc. aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par Sun Microsystems, Inc. UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd. Adobe est une marque enregistree de Adobe Systems, Incorporated. PostScript est une marque de fabrique d‘Adobe Systems, Incorporated, laquelle pourrait être déposée dans certaines juridictions aux Etats-Unis et dans d’autres pays. ORACLE est une marque déposée registre de Oracle Corporation. L’interfaces d’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés. Sun reconnaît les efforts de pionniers de Xerox pour larecherche et le développement du concept des interfaces d’utilisation visuelle ou graphique pour l’industrie de l’informatique. Sun détient une licence non exclusive de Xerox sur l’interface d’utilisation graphique Xerox, cette licence couvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui en outre se conforment aux licences écrites de Sun. L’accord du gouvernement américain est requis avant l’exportation du produit. LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, Y COMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON. CE MANUEL DE RÉFÉRENCE DOIT ÊTRE UTILISÉ DANS LE CADRE D’UN COURS DE FORMATION DIRIGÉ PAR UN INSTRUCTEUR (ILT). IL NE S’AGIT PAS D’UN OUTIL DE FORMATION INDÉPENDANT. NOUS VOUS DÉCONSEILLONS DE L’UTILISER DANS LE CADRE D’UNE AUTO-FORMATION.

Please Recycle

Table of Contents About This Course .............................................................Preface-xiv Course Goals........................................................................ Preface-xiv Course Map............................................................................ Preface-xv Topics Not Covered.............................................................Preface-xvi How Prepared Are You?....................................................Preface-xvii Introductions ..................................................................... Preface-xviii How to Use Course Materials ............................................ Preface-xix Conventions ........................................................................... Preface-xx Icons ............................................................................... Preface-xx Typographical Conventions ..................................... Preface-xxi Notes to the Instructor........................................................Preface-xxii Sun Storage Concepts .....................................................................1-1 Objectives ........................................................................................... 1-1 Disk Storage Administration Introduction .................................... 1-2 VxVM Software Installation .................................................... 1-2 VxVM Initialization .................................................................. 1-2 RAID Volume Design.............................................................. 1-3 RAID Volume Creation............................................................ 1-3 RAID Volume Administration................................................ 1-4 Interfaces for Sun Storage Devices .................................................. 1-5 SCSI Overview........................................................................... 1-5 SCSI Interface Implementation .............................................. 1-6 SCSI Interface Standards.......................................................... 1-7 SCSI Priority.............................................................................. 1-9 SCSI Phases and the Move to Fibre Channel ........................ 1-9 Fibre Channel Technology....................................................... 1-9 Fibre Channel-Arbitrated Loop ........................................... 1-10 Advantages of FC-AL............................................................. 1-10 Fibre Channel Compared to SCSI........................................ 1-11 RAID Technology ............................................................................ 1-12 Host-Based RAID (Software RAID Technology)................ 1-12 Controller-Based RAID (Hardware RAID Technology) ... 1-13

iv Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts..................................................................... 1-14 Hot Swapping.......................................................................... 1-14 Storage Area Networking ..................................................... 1-16 Multihost Storage Access...................................................... 1-21 Multipath Storage Access ..................................................... 1-23 Storage Configuration Identification ............................................ 1-28 Conducting Physical Inventory ............................................ 1-28 Displaying Storage Configurations ...................................... 1-28 Identifying Controller Addressing...................................... 1-30 Identifying Device Path Components.................................. 1-31 Identifying DMP Devices...................................................... 1-34 Storage Array Firmware ................................................................. 1-35 Fibre Channel HBA Cards ..................................................... 1-35 Verifying Fibre Channel HBA Firmware ........................... 1-36 Verifying SPARCstorage Array 100 Firmware.................. 1-37 Verifying Sun StorEdge A5x00 Array Firmware............... 1-38 Verifying Sun StorEdge T3 Array Firmware ..................... 1-39 Verifying Sun StorEdge A5x00 Disk Drive Firmware ...... 1-40 Firmware Upgrade Best Practices........................................ 1-41 Exercise: Recording Your Storage Configuration ....................... 1-42 Preparation............................................................................... 1-42 Task 1 – Reviewing Sun Storage Features ........................... 1-42 Task 2 – Identifying Host Adapter Configurations ........... 1-44 Task 3 – Identifying Storage Array Configurations.......... 1-45 Task 4 – Verifying Storage Interface Firmware Revisions ............................................................................... 1-46 Task 5 – Verifying Array Disk Drive Firmware Revisions ............................................................................... 1-47 Exercise Summary............................................................................ 1-48 Managing Data ................................................................................. 2-1 Objectives ........................................................................................... 2-1 Virtual Disk Management ................................................................ 2-2 Availability................................................................................. 2-2 Performance ............................................................................... 2-2 Scalability .................................................................................. 2-3 Maintainability .......................................................................... 2-3 RAID Technology Introduction ....................................................... 2-4 Supported RAID Standards..................................................... 2-4 RAID Terminology .................................................................. 2-5 RAID Level Common Features ........................................................ 2-6 Concatenation – RAID 0........................................................... 2-6 Striping – RAID 0 ...................................................................... 2-8 Mirroring – RAID 1................................................................ 2-10 Mirrored Stripe – RAID 0+1 ................................................. 2-12 Mirrored Concatenation – RAID 0+1 .................................. 2-14 Striped Mirror – RAID 1+0 ................................................... 2-15

v

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Concatenated Mirror – RAID 1+0........................................ 2-17 Striping With Distributed Parity – RAID 5 ........................ 2-19 Exercise: Optimizing System Configurations .............................. 2-21 Preparation............................................................................... 2-21 Task 1 – Reviewing Software RAID Features .................... 2-22 Task 2 – Identifying Availability and Performance Cabling.................................................................................. 2-25 Task 3 – Optimizing RAID-0 Volumes ............................... 2-26 Task 4 – Optimizing RAID-1 Volumes ............................... 2-27 Task 5 – Optimizing RAID-5 Volumes ............................... 2-28 Task 6 – Optimizing RAID 0+1 Volumes ........................... 2-29 Task 7 – Optimizing RAID 1+0 Volumes ........................... 2-30 Task 8 – Identifying Effective Storage Utilization............. 2-31 Task 9 – Selecting Disk Drives for Use................................. 2-32 Exercise Summary............................................................................ 2-33 VERITAS Volume Manager Installation ..........................................3-1 Objectives ........................................................................................... 3-1 Installation Planning.......................................................................... 3-2 System Downtime..................................................................... 3-2 Storage Configuration Assessment ........................................ 3-2 Upgrade Resources ................................................................... 3-3 Licensing .................................................................................... 3-3 Current System Checkpoint .................................................... 3-3 Backups....................................................................................... 3-3 Testing the New Configuration .............................................. 3-3 Researching Software Patches.......................................................... 3-4 Researching Current Patch Information................................ 3-4 Installing Patches ..................................................................... 3-6 Vendor Software Patches ......................................................... 3-6 Installing VxVM Software ................................................................ 3-7 Software Distribution Overview............................................. 3-7 Software Package Installation ................................................ 3-9 Software Installation User Interaction ................................ 3-10 Initializing VxVM Using the vxinstall Utility ......................... 3-13 The vxinstall Utility Dialogue .......................................... 3-13 Licensing Requirements......................................................... 3-14 Verifying Licensed Features ................................................. 3-15 VxVM Post-Installation Environment........................................... 3-16 VxVM System Files ................................................................. 3-16 System Startup Messages...................................................... 3-18 System Startup Processes...................................................... 3-19 System and User Executable Files ....................................... 3-21 Verifying Initial Disk Drive Status ....................................... 3-23 Preparing for VxVM Disk Drive Management............................ 3-24 Disk Drive Initialization Process .......................................... 3-24 Disk Drive Encapsulation Process........................................ 3-27

vi Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Protecting Storage Devices From Usage............................. 3-29 Global Exclusion .................................................................... 3-32 Installing the VEA............................................................................ 3-35 VEA Software Initialization.................................................. 3-36 VEA Client Software Startup ............................................... 3-37 Host Connection Window .................................................... 3-38 Resolving Low-Bandwidth Access Problems .................... 3-39 Using Basic VEA Features .............................................................. 3-40 Main Window Functional Areas........................................... 3-40 Resizing Display Panes .......................................................... 3-45 Modifying Preferences .......................................................... 3-46 Customizing the Grid Display ............................................. 3-47 Examining VEA Command Logs ......................................... 3-48 Using the VEA Search Tool .................................................. 3-49 Decoding VxVM Error Messages .................................................. 3-50 Exercise: Configuring VxVM.......................................................... 3-51 Preparation............................................................................... 3-51 Task 1 – Reviewing Key Lecture Points.............................. 3-52 Task 2 – Installing the VxVM Software .............................. 3-55 Task 3 – Verifying the VxVM System Files ........................ 3-57 Task 4 – Evaluating the Storage Configuration ................. 3-58 Task 5 – Installing the VEA Client Software ...................... 3-59 Task 6 – Starting the VEA Client Software......................... 3-60 Task 7 – Customizing the VEA GUI Appearance ............. 3-61 Task 8 – Navigating the VxVM Technical Manuals........... 3-61 Task 9 – Using the VxVM Error Numbering System........ 3-63 Exercise Summary............................................................................ 3-64 VERITAS Volume Manager Basic Operations ............................... 4-1 Objectives ........................................................................................... 4-1 VxVM Disk Group Functions........................................................... 4-2 Primary Functions of a Disk Group ....................................... 4-2 VxVM Disk Drives .................................................................... 4-3 Standard VxVM Disk Groups ................................................ 4-4 Shared VxVM Disk Groups .................................................... 4-5 Cross-Platform Data Sharing Disk Groups .......................... 4-6 VxVM Disk Group Operations ........................................................ 4-7 Verifying Disk Group Status ............................................................ 4-8 Using the vxdisk Command to Verify Disk Group Status........................................................................................ 4-8 Using the vxdg Command to Verify Disk Group Status..... 4-8 Administering Disk Groups Using the vxdiskadm Utility.......... 4-9 Functional Overview ............................................................. 4-10 Creating a New Disk Group................................................. 4-11 Removing a Disk Drive From a Disk Group...................... 4-12

vii

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs ........................................................................................... 4-13 Using the vxdiskunsetup Command ................................. 4-13 Initializing Disk Drives ......................................................... 4-14 Using the vxdg Command..................................................... 4-14 Adding and Removing Disk Drives.................................... 4-16 Importing and Deporting Disk Groups .............................. 4-17 Destroying a Disk Group ...................................................... 4-19 Renaming VxVM Disk Drives............................................... 4-19 Administering Disk Groups Using the VEA GUI ....................... 4-20 Creating a New Disk Group.................................................. 4-20 Adding and Removing Disk Drives.................................... 4-22 Deporting Disk Groups......................................................... 4-24 Importing Disk Groups ......................................................... 4-25 Destroying a Disk Group ...................................................... 4-26 Renaming VxVM Disk Drives.............................................. 4-27 Displaying VEA Object Properties ...................................... 4-28 Exercise: Performing VxVM Basic Operations ............................ 4-29 Preparation.............................................................................. 4-30 Task 1 – Reviewing Key Lecture Points.............................. 4-31 Task 2 – Verifying the VxVM Environment....................... 4-32 Task 3 – Verifying the Initial Disk Drive Status ................ 4-33 Task 4 – Setting the Default Disk Drive Format ................ 4-34 Task 5 – Initializing Disk Drives.......................................... 4-35 Task 6 – Creating New Disk Groups.................................... 4-36 Task 7 – Viewing Command Logs....................................... 4-38 Task 8 – Importing and Deporting Disk Groups................ 4-38 Task 9 – Destroying a Disk Group....................................... 4-40 Task 10 – Renaming Disk Drives .......................................... 4-40 Task 11 – Using the vxdiskadm Utility to Perform Basic Operations (Optional) ......................................................... 4-41 Task 12 – Verifying Ending Lab Status ............................... 4-42 Exercise Summary............................................................................ 4-43 VERITAS Volume Manager Volume Operations ............................5-1 Objectives ........................................................................................... 5-1 Interpreting Volume Structure Listings.......................................... 5-2 Subdisks...................................................................................... 5-2 Plexes .......................................................................................... 5-3 Volumes...................................................................................... 5-4 Volume Structure Examples................................................... 5-5 Volume Planning ............................................................................... 5-6 Volume Distribution................................................................. 5-6 Volume Naming Conventions ............................................... 5-8 Space Allocation Planning ....................................................... 5-8 Selecting Volume Types......................................................... 5-11

viii Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI ........................................ 5-13 Disk Selection Method............................................................ 5-13 Using the Disk Selection Form............................................. 5-14 Using the Volume Attributes Form..................................... 5-15 Using the Create File System Form ..................................... 5-16 Creating Volumes Using the vxassist Command.................... 5-17 The vxassist Command ...................................................... 5-17 Specifying Volume Size.......................................................... 5-17 Using vxassist Command Options .................................. 5-18 Modifying Volume Access Attributes........................................... 5-20 Verifying Volume Ownership............................................... 5-20 Modifying Volume Ownership and Permissions............... 5-20 Adding a UFS File System to Existing Volumes ......................... 5-21 Using the VEA GUI to Add a File System........................... 5-21 Adding a File System From the Command Line............... 5-23 Enabling the Solaris OS UFS Logging Feature ................... 5-24 Administering Volume Logs.......................................................... 5-25 Using DRLs .............................................................................. 5-25 Using RAID-5 Logs................................................................ 5-26 Planning Log Placement ....................................................... 5-27 Adding a Volume Log From the VEA GUI......................... 5-28 Adding a Volume Log From the Command Line ............. 5-29 Removing Volume Logs Using the VEA GUI.................... 5-30 Removing Volume Logs From the Command Line.......... 5-31 Using the VEA GUI to Analyze Volume Structures ................... 5-32 Displaying Volume Layout Details ...................................... 5-32 Viewing Disk Volume Mapping and Performance........... 5-33 Exercise: Creating Volumes and File Systems ............................. 5-34 Preparation............................................................................... 5-34 Task 1 – Reviewing Key Lecture Points.............................. 5-35 Task 2 – Creating a Volume.................................................. 5-37 Task 3 – Adding a Volume Mirror ...................................... 5-39 Task 4 – Adding a File System to a Volume........................ 5-41 Task 5 – Adding a DRL ......................................................... 5-43 Task 6 – Resizing a Volume and File System..................... 5-45 Task 7 – Creating a RAID-5 Volume ................................... 5-47 Task 8 – Analyzing Volumes Using the VEA GUI............ 5-49 Task 9 – Verifying Ending Lab Status ................................. 5-54 Exercise Summary............................................................................ 5-55 VERITAS Volume Manager Advanced Operations ....................... 6-1 Objectives ........................................................................................... 6-1 Boot Disk Encapsulation and Mirroring......................................... 6-2 Optimizing the Boot Disk Hardware Configuration ........... 6-2 Boot Disk Encapsulation Prerequisites ................................. 6-3 Encapsulating the System Boot Disk..................................... 6-4

ix

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Mirroring the System Boot Disk ............................................ 6-5 Verifying the Completed Boot Disk Configuration ............ 6-6 Creating a Best Practice Boot Disk Configuration......................... 6-8 Establishing Boot Disk Configuration Policies .................. 6-10 Modifying an Existing Boot Disk Configuration............... 6-11 Administering Hot Devices ............................................................ 6-13 Selecting Hot Device Operational Mode ............................. 6-13 Evaluating Hot-Device Configurations .............................. 6-16 Administering Hot Devices Using Command-Line Programs .............................................................................. 6-17 Administering Hot Devices Using the VEA GUI.............. 6-18 Controlling Relocation Recovery Time............................... 6-19 Evacuating a Disk Drive ................................................................. 6-20 Identifying Evacuation Conflicts .......................................... 6-20 Preparing for Evacuation ....................................................... 6-20 Identifying Suitable Evacuation Disk Drives..................... 6-21 Evacuation Using the vxevac Command .......................... 6-22 Evacuation Using the VEA GUI........................................... 6-23 Evacuation Using the vxdiskadm Utility ........................... 6-24 Moving Disk Drives Without Preserving Data ........................... 6-25 Moving a Disk Drive Using the Command Line................ 6-25 Moving a Disk Drive Using the VEA GUI ......................... 6-26 Moving Populated Disk Drives to a New Disk Group............... 6-27 Evaluating Disk Drive Involvement .................................... 6-27 Saving the Configuration....................................................... 6-28 Moving the Disk Drives to a New Disk Group .................. 6-29 Reloading the Volume Configuration.................................. 6-30 Backing Up and Restoring Disk Group Configurations ............ 6-31 Automatic Configuration Backup ........................................ 6-31 Manual Configuration Backup and Restore ...................... 6-33 Importing Disk Groups After a System Crash ............................ 6-36 Importing the rootdg Disk Group After a Crash ............. 6-37 Volume Snapshot Operations ........................................................ 6-38 Snapshot Process..................................................................... 6-38 Using the VEA GUI to Create a Snapshot .......................... 6-39 Using the Command Line to Create a Snapshot ............... 6-43 Online Volume Relayout ................................................................ 6-44 Volume Relayout Prerequisites............................................. 6-44 Volume Relayout Using the Command Line..................... 6-45 Volume Relayout Using the VEA GUI................................ 6-47 Creating Layered Volumes............................................................. 6-49 Layered Volume Disk Requirements ................................... 6-49 Evaluating Available Disk Space......................................... 6-50 Creating Layered Volumes From the Command Line ..... 6-51 Creating Layered Volumes Using the VEA GUI............... 6-52 Identifying Layered Volume Subcomponents................... 6-53

x Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration .............. 6-54 Primary ISP Components ...................................................... 6-54 Using Storage Pool Set Templates ....................................... 6-56 Using Storage Pool Templates ............................................. 6-58 Using Application Volume Templates ............................... 6-60 Creating Application Volumes Using the vxvoladm Command............................................................................. 6-62 Creating Application Volumes Using the VEA GUI ........ 6-63 Interpreting Application Volume Configurations ............ 6-65 Replacing Failed Disk Drives ......................................................... 6-66 Failure Behavior ...................................................................... 6-66 Evaluating Failure Severity ................................................... 6-67 General Disk Drive Replacement Process .......................... 6-70 Exercise: Performing Advanced Operations................................ 6-72 Preparation............................................................................... 6-72 Task 1 – Reviewing Key Lecture Points.............................. 6-73 Task 2 – Encapsulating the System Boot Disk ................... 6-76 Task 3 – Mirroring the System Boot Disk ............................ 6-78 Task 4 – Performing an Online Volume Relayout............. 6-80 Task 5 – Evacuating a Disk Drive ........................................ 6-82 Task 6 – Moving a Populated Volume................................. 6-82 Task 7 – Performing a Snapshot Backup ............................ 6-84 Task 8 – Creating a Layered Volume .................................. 6-85 Task 9 – Replacing a Failed Disk Drive .............................. 6-86 Task 10 – Using Intelligent Storage Provisioning .............. 6-88 Task 11 – Configuring a Best Practice Boot Disk............... 6-91 Exercise Summary............................................................................ 6-93 VERITAS File System Basic Operations........................................ 7-1 Objectives ........................................................................................... 7-1 Basic VxFS Features ........................................................................... 7-2 Extent-Based Space Allocation................................................ 7-2 File System Intent Logging ..................................................... 7-3 Installing the VxFS Software ............................................................ 7-4 Creating VxFS File Systems .............................................................. 7-5 Extended VxFS Mount Options ....................................................... 7-6 Intent Log Behavior .................................................................. 7-6 Error Handling Behavior ......................................................... 7-7 Other VxFS Mount Options..................................................... 7-7 Online File System Administration ................................................. 7-8 Online Defragmentation .......................................................... 7-8 Online Resizing ......................................................................... 7-8 Online Backup and Restore ..................................................... 7-8

xi

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS.............................................................. 7-9 Preparation................................................................................. 7-9 Task 1 – Reviewing Key Lecture Points.............................. 7-10 Task 2 – Installing the VxFS Software................................. 7-11 Task 3 – Creating a VxFS File System ................................. 7-13 Task 4 – Resizing a VxFS File System .................................. 7-14 Task 5 – Defragmenting a VxFS File System....................... 7-15 Task 6 – Backing Up and Restoring a VxFS File System ... 7-16 Task 7 – Using VxFS Extended Mount Options ................. 7-17 Exercise Summary............................................................................ 7-19 VERITAS Volume Manager Performance Management ................8-1 Objectives ........................................................................................... 8-1 Performance Improvement Techniques ......................................... 8-2 Data Assignment Strategies..................................................... 8-2 Volume Structure Strategies.................................................... 8-4 Read Policy Strategies ............................................................. 8-6 Hardware Configuration Strategies ....................................... 8-7 Using Performance Analysis Tools ................................................. 8-8 Gathering Volume Performance Statistics ............................ 8-8 Gathering Application Performance Statistics................... 8-11 RAID-5 Write Performance ............................................................ 8-12 Read-Modify-Write Operations ............................................ 8-12 Reconstruct-Write Operations............................................... 8-13 Full-Stripe Write Operations ................................................. 8-14 Exercise: Demonstrating Performance Differences..................... 8-15 Preparation............................................................................... 8-15 Task 1 – Reviewing Key Lecture Points.............................. 8-16 Task 2 – Performing a RAID-5 Write Performance Test .. 8-18 Task 3 – Performing a Striped Volume Write Performance Test ........................................................................................ 8-21 Exercise Summary............................................................................ 8-22

xii Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface

About This Course Course Goals Upon completion of this course, you should be able to: ●

Install and initialize VERITAS Volume Manager (VxVM) software

●

Define VxVM objects

●

Describe public and private regions

●

Start and customize the Volume Manager Storage Administrator VERITAS Enterprise Administrator (VEA) graphical user interface (GUI)

●

Perform operations using the command-line interface

●

Perform disk and volume operations

●

Create redundant array of independent disk (RAID) volumes

●

Set up dirty-region logs (DRLs)

●

Perform common file system operations using the VEA GUI

●

Create new disk groups, remove disks from group, move disks between disk groups, and deport and import disk groups between servers

●

Simulate disk failure and complete a disk recovery

●

Create and mange hot-spare pools

●

Manage and disable the hot-relocation feature

●

Perform basic performance analysis

Preface-xiv Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Course Map

Course Map The following course map enables you to see what you have accomplished and where you are going in reference to the course goals.

Overview Sun Storage Concepts

Managing Data

VERITAS Volume Manager Introduction VERITAS Volume Manager Installation

VERITAS Volume Manager Basic Operations

VERITAS Volume Manager Operations VERITAS Volume Manager Volume Operations

VERITAS Volume Manager Advanced Operations

VERITAS File System Basic Operations

VERITAS File System Operations VERITAS Volume Manager Performance Management

Preface-xv

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Topics Not Covered

Topics Not Covered This course does not cover the following topics. Many of these topics are covered in other courses offered by Sun Educational Services: ●

Solaris™ Operating System (Solaris OS) administration – Covered in SA-239: Intermediate System Administration for the Solaris™ 9 Operating Environment and SA-299: Advanced System Administration for the Solaris™ 9 Operating Environment

●

Storage system maintenance – Covered in SM-250: Sun™ Software RAID Storage Systems Maintenance

Refer to the Sun Educational Services catalog for specific information and registration.

About This Course Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xvi

How Prepared Are You?

How Prepared Are You? To be sure you are prepared to take this course, can you answer yes to the following questions? ●

●

●

Preface-xvii

Can you perform the following Solaris 9 OS administration operations? ●

Configure environmental variables

●

Control system run levels

●

Use basic OpenBoot™ commands

Can you physically configure the following Sun systems and peripherals? ●

Desktop or server-class systems

●

Sun storage arrays

Can you use the following UNIX® administrative commands? ●

chmod, chown, and chgrp

●

format

●

man

●

mkdir

●

mkfs/newfs

●

mount

●

patchadd

●

pkgadd and pkgrm

●

vi

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Introductions

Introductions Now that you have been introduced to the course, introduce yourself to the other students and the instructor, addressing the following items: ●

Name

●

Company affiliation

●

Title, function, and job responsibility

●

Experience related to topics presented in this course

●

Reasons for enrolling in this course

●

Expectations for this course

About This Course Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xviii

How to Use Course Materials

How to Use Course Materials To enable you to succeed in this course, these course materials employ a learning module that is composed of the following components:

Preface-xix

●

Goals – You should be able to accomplish the course goals after finishing this course and meeting all of its objectives.

●

Objectives – You should be able to accomplish the objectives after completing a portion of instructional content. Objectives support goals and can support other higher-level objectives.

●

Lecture – The instructor presents information specific to the objectives of the module. This information helps you learn the knowledge and skills necessary to succeed with the activities.

●

Activities – The activities take on various forms, such as exercises, self-checks, discussions, and demonstrations. Activities are used to facilitate mastery of an objective.

●

Visual aids – The instructor might use several visual aids to convey a concept, such as a process, in a visual form. Visual aids commonly contain graphics, animation, and video.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Conventions

Conventions The following conventions are used in this course to represent various training elements and alternative learning resources.

Icons

!

Discussion – Indicates a small-group or class discussion on the current topic is recommended at this time.

?

Note – Indicates additional information that can help students but is not crucial to their understanding of the concept being described. Students should be able to understand the concept or complete the task without this information. Examples of notational information include keyword shortcuts and minor system adjustments. Caution – Indicates that there is a risk of personal injury from a nonelectrical hazard, or risk of irreversible damage to data, software, or the operating system. A caution indicates that the possibility of a hazard (as opposed to certainty) might happen, depending on the action of the user.

About This Course Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xx

Conventions

Typographical Conventions Courier is used for the names of commands, files, directories, programming code, and on-screen computer output; for example: Use ls -al to list all files. system% You have mail. Courier bold is used for characters and numbers that you type; for example: To list the files in this directory, type: # ls

Courier italics is used for variables and command-line placeholders that are replaced with a real name or value; for example: To delete a file, use the rm filename command.

Courier italics bold is used to represent variables whose values are to be entered by the student as part of an activity; for example: Type chmod a+rwx filename to grant read, write, and execute rights for filename to world, group, and users. Palatino italics is used for book titles, new words or terms, or words that you want to emphasize; for example: Read Chapter 6 in the User’s Guide. These are called class options.

Preface-xxi

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Notes to the Instructor

Notes to the Instructor For classroom setup suggestions, refer to the ES310_revC_setup.txt file. The ES310_revC_setup.txt file contains specific setup instructions about: ●

General equipment configuration for this course

●

Patch download instructions

●

VxVM requirements and download instructions

●

Supported hardware

●

Required lab files

About This Course Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface-xxii

Module 1

Sun Storage Concepts Objectives Upon completion of this module, you should be able to: ●

Describe the major disk storage administration tasks

●

Describe Sun storage interface types

●

Describe available RAID technologies including:

●

●

●

●

Host-based RAID technology

●

Controller-based RAID technology

Describe disk storage concepts that are common to many storage installations including: ●

Hot swapping

●

Storage area networking

●

Multihost access

●

Multipath access

Identify storage configurations including: ●

Conducting physical inventory

●

Displaying storage configurations

●

Identifying controller addresses

●

Decoding logical device paths

Verify storage array firmware revisions

1-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Administration Introduction

Disk Storage Administration Introduction Installing, configuring, and managing a VxVM installation requires both software and hardware knowledge.

VxVM Software Installation Installing the VxVM software is essentially the same as installing any Solaris OS application. You can use either the standard command-line or GUI installation tools.

VxVM Initialization When you install VxVM, at least one disk drive must be brought under VxVM control using the vxinstall utility. You can either encapsulate a disk, which preserves existing data on the disk, or you can initialize a disk, which effectively destroys existing data.

Required Hardware Knowledge The VxVM installation process is always the same, regardless of the system platform or storage technology used. However, you must be able to identify storage array device addresses and differentiate them from other types of disk storage addresses. If you are not familiar with the device address strategy in your particular installation, you might accidentally initialize the wrong disk drives. This error could destroy valuable data, including the operating system.

1-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Administration Introduction

RAID Volume Design Generally, RAID volume structures are designed with one or more of the following goals in mind: ●

Cost savings

●

Performance

●

Availability

●

Maintainability

In most cases, compromises are made when choosing among cost savings, performance, availability, and maintainability.

Required Hardware Knowledge You must have a thorough understanding of interface types, addressing schemes, and internal hardware structure to achieve design goals. It is possible to design virtual volume structures without this background knowledge, but the result might perform poorly and might not have the reliability that is required for your application.

RAID Volume Creation You can create RAID volume structures using the VxVM GUI or command-line programs. Command-line programs are sometimes a better choice to use, especially when volume creation must be automated using script files. You can configure the GUI to display command-line equivalents for each operation.

Required Hardware Knowledge Even though you might not be responsible for the design of your VxVM volume structures, you must still be familiar with most aspects of your particular storage devices.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-3

Disk Storage Administration Introduction

RAID Volume Administration In larger installations, the most common VxVM administrative task is identifying and replacing failed disk drives. At the most basic level, this task involves the use of a single VxVM utility, the vxdiskadm utility. For certain storage platforms, you must use another utility, luxadm, during the disk replacement process. Most VxVM administrative tasks require analyzing error messages using VxVM utilities, such as vxprint and vxdisk, along with some basic Solaris OS commands.

Required Hardware Knowledge Administering RAID volumes requires a number of hardware-related skills including:

1-4

●

Decoding device error messages

●

Relating device addresses to physical devices

●

Following hardware removal procedures that are appropriate for each particular disk storage technology

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

Interfaces for Sun Storage Devices Sun storage devices are accessed through one of two basic interface types: ●

Small Computer System Interface (SCSI)

●

Fiber-optic

Each of the basic interface types has two or more variations, which have evolved over a period of several years. The interfaces have improved in the following areas: ●

Data transfer speed

●

Data transfer latency

●

Interface cable lengths

SCSI Overview SCSI was initially implemented in the 1980s as a way of making the interface between the host computer system and the disks independent of the computer manufacturer. Prior to the introduction of SCSI, all the computer manufacturers had their own way of connecting the host computer system to the disk drives. SCSI introduced the idea of intelligent disk drives where the host computer system requested the transfer of a block of data from the disk. The host system had no need to know the underlying disk geometry. It issued a request to the disk for the transfer of a block of data. The shift of intelligence from the host computer system to the disk allowed the same disk to be used by different manufacturers, which ultimately led to cheaper, faster, and larger disk drives. The connection between the host system was by the SCSI bus, for which a set of standards was agreed upon. The speed and data capacity of the SCSI bus has been increased to allow for the higher demands of today’s servers. One of the earliest problems faced with SCSI was the differing cable lengths from the host system to the disk drives themselves. For the SCSI bus to reliably operate over differing cable lengths, two electrical connections methods were defined: Single-ended (for short connection lengths) and differential (for connection over longer cables).

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-5

Interfaces for Sun Storage Devices

SCSI Interface Implementation Both single-ended SCSI and differential SCSI are currently used with Sun storage products.

Single-Ended SCSI As shown in Figure 1-1, each bit is transmitted using one signal, referenced to 0 volts. This allows cable lengths up to six meters with standard SCSI-1 devices.

Figure 1-1

Single-Ended SCSI

Differential SCSI As shown in Figure 1-2, the data bits are sent using two equal and opposite voltages. These allow the signal to travel farther without degradation. Differential SCSI allows cable lengths up to 25 meters.

inv

Figure 1-2

1-6

Differential SCSI

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

add

Interfaces for Sun Storage Devices

SCSI Interface Standards A partial list of the SCSI-1, SCSI-2, and SCSI-3 standards currently used by Sun are shown in Table 1-1. Other SCSI standards are used by different manufacturers. Table 1-1 Sun SCSI Standards Name (Type)

Clock Speed

Data Width

Data Rate (Command Rate)

Number of Devices

SCSI (SCSI-1)

5 MHz

8 bits

5 Mbytes/sec

8

Fast SCSI (SCSI-2)

10 MHz

8 bits

10 Mbytes/sec

8

Wide SCSI (SCSI-2)

5 MHz

16 bits

10 Mbytes/sec

16

Fast/Wide SCSI (SCSI-2)

10 MHz

16 bits

20 Mbytes/sec

16

Ultra Differential Wide SCSI (SCSI-3)

20 MHz

16 bits

40 Mbytes/sec

16

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-7

Interfaces for Sun Storage Devices

SCSI-3 Standards Currently, only recent Sun systems such as the Ultra™ 60 workstation, Ultra 80 workstation, and the Sun Blade™ 1000 workstations, have SCSI-3 compatible interfaces. The related specifications are: ●

Ultra 60 workstation – 68-pin connector, 40-Mbyte/sec Ultra SCSI (SCSI-3), two channels (synchronous)

●

Ultra 80 workstation – 68-pin connector, 40-Mbyte/sec Ultra SCSI (SCSI-3), two channels (synchronous)

●

Sun Blade 1000 workstation – One single-ended 68-pin SCSI connector (16-bit Ultra/Fast/Wide, 40-Mbyte/sec), internal disks are Fibre Channel

Table 1-2 shows some of the current SCSI-3 standards. Table 1-2

SCSI-3 Standards High Voltage Differential Length

Low Voltage Differential Length

Targets

Type

Speed

Width

SingleEnded Length

SCSI 3U

20 MB/sec

8 bits

1.5 m

25 m

–

8

SCSI 3 U

20 MB/sec

8 bits

3m

–

–

4

SCSI 3 FW

20 MB/sec

16 bits

6m

25 m

–

16

SCSI 3 UW

40 MB/sec

16 bits

–

25 m

–

16

SCSI 3 UW

40 MB/sec

16 bits

1.5 m

–

–

8

SCSI 3 UW

40 MB/sec

16 bits

3m

–

–

4

SCSI 3 U2

40 MB/sec

8 bits

–

12 m

12 m

8

SCSI 3 U2

80 MB/sec

16 bits

–

25 m

25 m

2

SCSI 3 U2W

80 MB/sec

16 bits

–

12 m

12 m

16

SCSI 3 U2W

80 MB/sec

16 bits

–

25 m

25 m

2

1-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

SCSI Priority The bus arbitration mechanism for SCSI uses the SCSI target ID to determine priority. Narrow SCSI has target addresses 0–7. Target 7 has highest priority (usually the ID of the controller), and target 8 has the lowest. Performance can be affected through injudicious use of SCSI target addresses. Wide SCSI uses target addresses 0–15 in the following priority: 7, 6, 5, 4, 3, 2, 1, 0, 15, 14, 13, 12, 11, 10, 9, 8. Address 7 is the highest priority.

SCSI Phases and the Move to Fibre Channel Although it appears as if SCSI has improved performance eight-fold, this is an exaggeration. SCSI has six phases, one of which is data transfer to or from the disk. Other phases include sense (status), command transfer, and bus arbitration. SCSI supports backward compatibility, so a narrow, slow disk can run off a fast-wide controller. This capability is achieved by having the administration of SCSI run at 5 megahertz (MHz) on narrow cabling. Each device is allowed to negotiate upwards. That is, to be fast, ultra fast, wide, and so on. The data transfer phase benefits from the improved clock speed, but command transfer does not. It takes about 5 milliseconds (ms) on each form of SCSI. The last 10 years have witnessed a 10,000-fold increase in computer performance. At the same time, requirements are increasing for more robust, highly available, disaster-tolerant computing resources. Computer resources continue to be pushed to their limits.

Fibre Channel Technology Fibre Channel technology is the answer to the growing problems of SCSI-based peripherals. Fibre Channel is a high-performance serial interconnect standard designed for bidirectional, point-to-point communications between servers, storage systems, workstations, switches, and hubs. It offers a variety of benefits over other link-level protocols, including efficiency, high performance, scalability, simplicity, ease of use and installation, and support for popular high-level protocols. Fibre Channel also offers a higher level of security and reliability through electrical isolation. There is no electromagnetic (EM) influence.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-9

Interfaces for Sun Storage Devices

Fibre Channel-Arbitrated Loop An important enhancement to Fibre Channel technology has been the development of Fibre Channel-Arbitrated Loop (FC-AL), which was developed specifically to meet the needs of storage interconnects. Using a loop topology, FC-AL can support basic configurations and sophisticated arrangements of hubs, switches, servers, and storage systems. Furthermore, by using SCSI protocols over the much faster, more robust Fibre Channel link, FC-AL provides higher levels of performance without requiring expensive and complex changes to existing device drivers and firmware.

Advantages of FC-AL The FC-AL development effort is part of the American National Standards Institute/International Organization for Standardization (ANSI/ISO) accredited SCSI-3 standard. This standard helps to prevent the creation of non-conforming, incompatible implementations. Virtually all major system vendors are implementing FC-AL, as are all major disk drive and storage system vendors. FC-AL operates on both fiber-optic cable and copper wire, and it can be used for more than just disk input/output (I/O). The Fibre Channel specification supports high-speed system and network interconnects using a wide variety of popular protocols, including:

1-10

●

SCSI

●

Internet Protocol (IP)

●

Adaptation Layer for Computer Data (AAL5) (ATM)

●

Fibre Channel Link Encapsulation (FC-LE)

●

Institute of Electrical and Electronics Engineers specification for data link layer transmission (IEEE 802.2)

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interfaces for Sun Storage Devices

Fibre Channel Compared to SCSI As shown in Table 1-3, Fibre Channel technology has many advantages compared to SCSI. Table 1-3 Fibre Channel Compared to SCSI Fibre Channel

SCSI

Full duplex operation

Half-duplex operation

100 Mbytes/sec

40 Mbytes/sec

Performance not affected by disk position.

Performance can be negatively affected by the physical disk position.

Multipath disk access is supported.

A disk can connect to only one SCSI channel.

Up to four hosts can connect to an appropriate array.

A maximum of two hosts can connect to a single SCSI channel.

Serial data transmission

Parallel data transmission

Suitable for use in a SAN. It allows the use of switches and hubs.

Unsuitable for use in a SAN. It allows no use of switches and hubs.

Can connect 30 km (25 MHz)/ 10 km (100 MHz) maximum distances.

Maximum distance is 25 meters with a differential SCSI.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-11

RAID Technology

RAID Technology RAID virtual data structures can be created and managed by software applications, or they can be a resident-hardware function of some storage devices.

Host-Based RAID (Software RAID Technology) VxVM is a good example of software RAID technology. All Sun storage arrays can be used with VxVM. As shown in Figure 1-3, user applications access a virtual structure through a single path that is composed of three separate disk drives.

User or Application Access

VM Software

3-Gbyte virtual volume

Controller c4 1-Gbyte physical disks T1

T2

T3

Storage Array

Figure 1-3

Host-Based RAID Technology

A typical virtual volume path name would be similar to the following: /dev/vx/dsk/dga/volume-01 Although the physical paths to the three disk drives in Figure 1-3 still exist, they are not accessed directly by users or applications. Only the virtual volume paths are referenced by users. Software that runs on the host system creates and manages the virtual software.

1-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Technology

Controller-Based RAID (Hardware RAID Technology) Controller-based RAID solutions use firmware that runs internally on storage array logic boards to maintain virtual structures that are composed of one or more physical disk drives. Sun hardware RAID models include: Sun StorEdge™ A1000, Sun StorEdge A3000, Sun StorEdge A3500, Sun StorEdge A3500FC, and Sun StorEdge T3 arrays. As shown in Figure 1-4, RAID configuration software (Sun StorEdge RAID Manager) running on the host system configures virtual structures in the array controller board. After initial configuration, the controller board firmware manages the virtual structures. Storage Array

Host System

RAID Firmware

Host Bus Adapter

User Access RAID Configuration Software

Figure 1-4

Disk Disk Disk Disk

Disk Disk Disk Disk

Controller-Based RAID Technology

Note – The Sun StorEdge T3 array RAID structures are configured using either the Sun StorEdge Component Manager software or resident storage array operating system commands. A typical hardware RAID device appears to be the same as any logical device path, such as /dev/dsk/c0t5d0s0. Applications do not recognize the underlying RAID structures. Hardware RAID solutions can have better performance than host-based RAID solutions for some types of RAID structures. Hardware RAID overhead calculations are performed at very high speeds by the controller-resident hardware, instead of on the host system as in host-based RAID. Hardware-based RAID-5 write performance can be much better than host-based RAID 5.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-13

Disk Storage Concepts

Disk Storage Concepts This section describes the following disk storage concepts that are common to many storage installations: ●

Hot swapping

●

Storage area networking

●

Multihost access

●

Multipath access

Hot Swapping Most Sun storage arrays are engineered so that a failed disk drive can be replaced without interrupting customer applications. The disk-replacement process includes one or more software operations that can vary with each disk storage platform.

General VxVM Disk-Replacement Procedure In its basic form, the process to replace a failed disk drive that is under VxVM control is as follows: 1.

Use the VxVM vxdiskadm utility to logically remove the disk.

2.

Use the VxVM vxdiskadm utility to logically install the new disk.

Caution – Never hot-swap a disk that is under control of VxVM. Always use the vxdiskadm utility or command-line equivalents.

1-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Alternative Disk-Replacement Procedure The VxVM disk replacement process is more complex for some storage arrays, such as the Sun StorEdge A5x00 array. The Sun StorEdge A5x00 array procedure is as follows:

.

1.

Use the VxVM vxdiskadm utility options 4 and 11 to logically remove the disk and place it offline.

2.

Use the luxadm utility’s remove_device command.

3.

Use the luxadm utility’s insert_device command.

4.

Run the vxdctl enable command to read in the new configuration.

5.

Use the VxVM vxdiskadm utility option 5 to logically install the new disk.

Note – You must be familiar with the disk-replacement process for your particular disk storage devices.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-15

Disk Storage Concepts

Storage Area Networking The current Sun storage area network (SAN) implementation is composed of the following hardware and software: ●

Supported arrays are Sun StorEdge A3500, Sun StorEdge, A5200, Sun StorEdge T3, and Sun StorEdge T3+ arrays

●

Sun Fibre Channel switches (8 and 16 ports)

●

Management software (fabric and switch management)

SANs are usually based on the Fibre Channel data communications standard.

Basic SAN Configuration Traditionally, data is made available through Network File System (NFS) mounts over Ethernet user networks. This configuration is referred to as network-attached storage (NAS). User networks can be overloaded by heavy data traffic. A SAN is a Fibre Channel network that off-loads data traffic from the user network. As shown in Figure 1-5, a SAN is a highperformance network composed of servers, storage devices, and interconnect devices, such as switches, hubs, cables, and converters. Switches Sun StorEdge T3 Array Partner Pair TL

Host Adapter

Host Adapter TL

Fiber-optic cables

Figure 1-5

SAN Components

A SAN can enable gigabit-speed data transfer with high-availability (no single-point-of-failure). Redundant SAN switches can be configured to have automatic failover capability. Data is automatically re-routed through a redundant switch.

1-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

SAN Definitions The SAN fabric. is composed of several elements. Without a basic understanding, the terminology can be confusing or meaningless.

SAN Fabric Networks that use Fibre Channel switches are referred to as fabrics. The term fabric characterizes a network of multiple switches as opposed to a network with a single switch. Each connection in a fabric can use the full 100-megabytes per second (Mbyte/sec) Fibre Channel bandwidth. Switches in a fabric use a routing technique known as cut-through switching. Cut-through switching refers to the process by which the Fibre Channel switch can route the incoming data frame almost immediately by reading its link-level destination ID (D_ID). It does not need to read the entire frame prior to transmitting it to its target.

Fibre Channel Frames The data elements referred to as data packets in Transmission Control Protocol/Internet Protocol (TCP/IP) are referred to as frames in Fibre Channel. The basic Fibre Channel frame structure contains a 24-byte header followed by up to 2112 bytes of data. A key portion of the 24-byte header is a 3-byte link-level D_ID that defines the port address to which the data frame must be transmitted.

Fibre Channel Devices There are two types of Fibre Channel devices, public and private. A Fibre Channel private loop is a traditional storage configuration, such as one or more Sun StorEdge A5200 arrays, connected to a host system either singly, in a daisy-chain configuration, or through a Fibre Channel hub. The devices are available on the loop only to the physically connected host. Private devices do not have full Fibre Channel addressing capability. They have only the Arbitrated Loop Physical Address (ALPA) portion of the Fibre Channel physical address. These devices exist only on loops, and, unless the Switch offers extra support, these devices cannot communicate outside their own loop.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-17

Disk Storage Concepts Public devices have full Fibre Channel addressing capability and, therefore, can communicate with any other public device on the fabric. They can be connected directly to the switch (one device per port) or arranged in an arbitrated loop with up to 127 nodes in the loop and the loop connected to the switch.

Switch Port Functions You can configure the Fibre Channel switch ports to function in several ways using switch management software. The primary reason for different port functionality is to allow selective access between Fibre Channel devices and host systems. You should only use the following port configurations: ●

Fabric port (F_Port) – A fabric port connects a Fibre Channel switch to a fabric-aware node port (or N_Port) on an end-device.

●

Segmented loop port (SL_Port) – Segmented loop ports provide support for private arbitrated loops on a Fibre Channel switch. All segmented loop ports in the same SL zone behave as one private arbitrated loop (and so they share the same ALPA space).

●

Translated loop port (TL_Port) – Translated loop ports provide support for public and private loop devices on a Fibre Channel switch. Translated loop ports translate between private and public addresses, allowing public devices and private devices to communicate with one another.

●

Trunk port (T_Port) – A trunk port connects a Fibre Channel switch to another Fibre Channel switch (this is known as cascading).

Zones Zoning refers to the deliberate segregation of SAN resources from other SAN resources. Zones essentially create sub-networks that provide different levels of connectivity or addressability between different hosts and devices on the network. Routing tables control the access of hosts to devices. You can individually configure device ports to be accessible or inaccessible to other specific ports.

1-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Sample SAN Configuration The example in Figure 1-6 shows how a multihost SAN configuration might be arranged to support both private and public use of storage resources. Sun StorEdgeTM L180 or Sun StorEdgeTM L700 FC Tape Library

Sun EnterpriseTM 420 Server

Switch 0

1 Host Adapter

2 Zone

3

4

5 Zone 6 Host Adapter

Sun StorEdge T3 Array Partner Pairs

7 Zone 8

Sun EnterpriseTM 3500 Server Switch 1 Host Adapter

1 3

Host Adapter

2 Zone

4

5 Zone 6 7 Zone 8 Sun StorEdge A5200 Array

Figure 1-6

Sample SAN Multihost Configuration

Note – To take advantage of the multipath connections from each host system, you must use a multipath product, such as VERITAS DMP or the Sun StorEdge Traffic Manager software.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-19

Disk Storage Concepts

SAN Device Addressing Traditionally, the Solaris OS has named storage devices based on the controller, target ID, and logical unit number of the device. When large SANs with multiple paths to large storage arrays are constructed, this naming convention could lead to thousands of targets per controller and storage identified more than once by different controllers and targets. The new method incorporates the World Wide Name (WWN) of the device into the device name used by the host. The new names provide the benefit of uniquely identifying storage devices to the host. When using Sun StorEdge Traffic Manager software, a device with multiple connections to a host is known to that host by one name. Old device path: /devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@3,0 New device path: /devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f2000 00225,0 Old symbolic device name: /dev/dsk/c4t3d0s2 New symbolic designation: /dev/dsk/c4t50020f200000225d0s2 The number of storage devices that can be attached to a host can grow to the thousands with the advent of SANs with native fabric connectivity. Probing all these devices at boot time and creating device nodes can increase the boot time greatly. In addition, a host might not need access to all of the storage devices it can access. The Sun StorEdge Network FC Switch-16, Version 3.0, no longer creates device nodes for every storage device attached. Instead, the administrator creates device nodes on demand by using the cfgadm utility. The device nodes, once created, are persistent across reboots.

1-20

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Multihost Storage Access With the advent of technology, such as the Sun StorEdge A5200 array, as many as four different hosts can be connected to the same storage device. Multihost connections are also possible on some versions of the SCSI.

Multi-Initiated SCSI Sun StorEdge MultiPack storage devices support physical SCSI interface connections from two different host systems. The SCSI interface on each of the systems must have a different initiator identifier (ID) setting, which is a system firmware configuration known as the scsi-initiator-id. As shown in Figure 1-7, you must change the scsi-initiator-id on one of the host systems to eliminate the addressing conflict between the two host systems. Host System B

Host System A Internal SCSI Bus

Internal SCSI Bus

scsi-initiator-id=7

scsi-initiator-id=7

Internal SCSI Bus

Internal SCSI Bus

scsi-initiator-id=6

scsi-initiator-id=7

SCSI Card

SCSI Card

In

t9

Out

t12

t10 t11

Figure 1-7

t13 t14

Multi-Initiated SCSI Configuration

The SCSI initiator values are changed using complex system firmware commands. The process of changing these values varies with system hardware platforms. Do not change the external SCSI bus, scsi-initiator-id, globally, change it at the interface card level. Read the documentation carefully. The procedures are hardware platform-specific.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-21

Disk Storage Concepts

Multihost Fiber-Optic Interface Two different fiber-optic interface storage arrays support multiple host connections. The SPARCstorage® Array 100 unit allows up to two host systems to connect to a single storage array. The Sun StorEdge A5000, Sun StorEdge 5100, and Sun StorEdge 5200 (Sun StorEdge A5x00) arrays allow up to four host system connections. The serial optical channel (SOC) connections are shown in Figure 1-8. SOC+ host adapter

Host 0

Host 1

Sun StorEdge A5x00 Storage Array

Interface Board A

Host 2 Interface Board B Host 3

SOC host adapter

SPARCstorage Array 100

Host 1 Port A Host 2 Port B

Figure 1-8

1-22

Multihost Fiber-Optic Configurations

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interface board

Disk Storage Concepts

Multipath Storage Access Multipathing is defined as dual connections to a storage array from a single host system. Multipathing can be used to provide redundant access paths in case of hardware failures and, in some cases, to perform load balancing between the two access paths. There are several different hardware and software schemes that can be used to implement multipath configurations.

Redundant Dual Active Controller Driver Some Sun storage devices allow dual connections to a storage array from a single host system. As shown in Figure 1-9, one host adapter can be configured as a backup if the primary access path fails. Storage Array Drive

Drive

Drive

Drive

Drive

Drive

Controller

Drive

Ultra SCSI Card

Drive

Host System

C1

Ultra SCSI Card C2

Controller

RDAC Driver

RAID Configuration Software

Figure 1-9

Redundant Dual Active Controller Driver

The Redundant Dual Active Controller (RDAC) driver is a special purpose driver that manages dual-interface connections. This driver is available with some of the Sun storage arrays, which include the Sun StorEdge A3500 and Sun StorEdge A3500FC array models. Applications directly interface with the RDAC driver and are unaware of interface failure. If one of the dual-controller paths fails, the RDAC driver automatically directs I/O to the functioning path.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-23

Disk Storage Concepts

Note – The Sun StorEdge A3500FC array uses a Fibre Channel interface instead of the SCSI interface used on the other RDAC-controlled storage arrays.

Alternate Path Driver for the Solaris™ OS The Alternate Path (AP) software for the Solaris OS works with Dynamic Reconfiguration (DR) to provide the ability to move all I/O off a system board before removal for upgrade or repair. AP is not applicable to all architectures. As shown in Figure 1-10, the AP software package contains user-level applications and kernel device drivers to allow the construction of meta-I/O devices from multiple controllers to a single physical device (disk or network). This function allows I/O on active disk and network I/O adapters to be seamlessly redirected to a previously specified and configured I/O device. Failed I/O operations to a device are automatically retried on the alternate path. System Interconnect Board #1

Board #2

Dual-Ported Storage Device

Figure 1-10 Alternate Path Hardware Configuration

1-24

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Dynamic Multipathing Driver The Dynamic Multipathing (DMP) driver is unique to the VxVM product. It is used only with fiber-optic interface storage arrays. As shown in Figure 1-11, the DMP driver can access the same storage array through more than one path. The DMP driver automatically manages multiple paths to the storage array. Depending on the storage array model, the paths are either used for load-balancing in a primary mode of operation or in a backup mode of operation in the event of a failure. Storage Array Drive

Drive

Drive

Drive

Drive

Drive

HBA Card

Interface

Drive

HBA Card

Drive

Host System

C1

Interface

C2

DMP Driver

HBA=Host Bus Adapter

Figure 1-11 Dynamic Multipathing Driver Specific paths can be enabled and disabled with the VxVM vxdmpadm command. Note – During a VxVM installation, you must take special steps to ensure that the DMP feature is compatible with AP, SAN, and the Sun StorEdge Traffic Manager software.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-25

Disk Storage Concepts

Sun StorEdge™ Traffic Manager Software The Sun StorEdge Traffic Manager software is a new architecture for representing and managing Fibre Channel devices that are accessible through multiple host controller interfaces (HCI). Sun StorEdge Traffic Manager software provides automatic load balancing and automatic failover to an alternate controller. A typical Sun StorEdge Traffic Manager software configuration is shown in Figure 1-12. System Interconnect Board #1

Board #2

Dual-Ported Storage Device

Figure 1-12 Sun StorEdge Traffic Manager Software Configuration Sun StorEdge Traffic Manager is the official name for the MPxIO product.

The minimum software and hardware requirements for Sun StorEdge Traffic Manager software are:

1-26

●

Solaris 8 OE Update 4

●

VxVM 3.2

●

Solstice DiskSuite™ 4.2.1 software

●

Sun StorEdge A5200 and Sun StorEdge T3 arrays

●

Peripheral Component Interconnect (PCI) single- and dual-Fibre Channel network adapters

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts

Multipath Software Summary The RDAC, AP, DMP, and Sun StorEdge Traffic Manager software can coexist in some configurations, but you might need to selectively configure each interface for use by only one of the applications. Table 1-4 compares some of the multipathing software features. Table 1-4 Multipath Software Comparison Functionality

Sun StorEdge Traffic Manager

Alternate Pathing

Vendor

Sun

Maximum paths

DMP

RDAC

Sun

VERITAS Software Corporation

LSI Logic/ Symbios

Two

Two

Four

Two

Active/active

Yes

No

Yes

No

Active/passive

Yes

Yes

Yes

Yes

Automatic failover

Yes

Yes

Yes

Yes

Supported devices

Sun StorEdge A5200 array, Sun StorEdge T3 array

SPARCstorage Array 100, Sun StorEdge A5x00 array, Sun StorEdge T3 array

Any VxVM supported disk

LSI Logic devices only

Host bus adapter type

Fibre Channel only

Fibre Channel only

SCSI and SCSI and Fibre Channel Fibre Channel

Comments

Cannot run with DMP at this time

Cannot run with DMP, and not supported on Sun Fire™ servers

VxVMmanaged disks only

LSI Logic/ Symbios controllers only

Naming

Special names Special names /dev/scsi_vhci /dev/ap

Special names /dev/vx/dmp

Native names

Environment

Solaris 8 OE, Update 4 or higher

Multiple operating systems and levels

Multiple operating systems and levels

Solaris 2.5.1 or higher

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-27

Storage Configuration Identification

Storage Configuration Identification To be an effective administrator, you must be able to identify storage configurations and decode logical device addresses.

Conducting Physical Inventory The first step in determining your storage configuration is to examine and record the number and types of storage arrays. You can trace cables from a system to attached storage devices. It is helpful to create a configuration diagram showing all attached storage devices and any unique identification information associated with them, such as world wide numbers, IP addresses, or names.

Displaying Storage Configurations The format utility and the luxadm utility are commonly used to display all available storage devices.

Using the luxadm Command to Display Storage Use the luxadm probe option as follows to locate several types of Sun storage arrays including Sun StorEdge T3 array logical unit numbers (LUNs). In the following example, two SENA type arrays were found along with two single-LUN T3 storage arrays. The luxadm command does not identify the exact model of storage. # luxadm probe Found Enclosure(s): SENA Name:AA Node WWN:5080020000034ed8 Logical Path:/dev/es/ses0 Logical Path:/dev/es/ses1 Node WWN:50020f200000c193 Device Type:Disk device Logical Path:/dev/rdsk/c2t1d0s2 Node WWN:50020f200000c367 Device Type:Disk device Logical Path:/dev/rdsk/c3t1d0s2

1-28

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Using the format Command to Display Storage The Solaris OS format utility is a reliable program for gathering basic storage configuration information. This utility reports all storage devices, regardless of type or model. The following sample output shows a configuration consisting of two internal disk drives and a single Sun StorEdge T3B array that is divided into six LUNs. # format Searching for disks...done

AVAILABLE DISK SELECTIONS: 0. c0t0d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135> /pci@1f,4000/scsi@3/sd@0,0 1. c0t1d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135> /pci@1f,4000/scsi@3/sd@1,0 2. c2t1d0 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0 3. c2t1d1 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,1 4. c2t1d2 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,2 5. c2t1d3 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,3 6. c2t1d4 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,4 7. c2t1d5 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,5 Specify disk (enter its number):

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-29

Storage Configuration Identification

Identifying Controller Addressing When the Solaris OS software is first configured, all disk storage interfaces are assigned a unique controller number. As shown in Figure 1-13, controller numbers are a single point of attachment to a single storage device or hub. Host System Internal SCSI c0

Sun StorEdge D1000 Array UDWS c1 FC-AL c2

Sun StorEdge A5x00 Array

FC-AL c3 0 1 2 3 4 5 6 FC-AL Hub Sun StorEdge A5x00 Array

Sun StorEdge A5x00 Array

Figure 1-13 Controller Address Numbering

Using VxVM Commands to Display Controller Addresses After the VxVM software is installed and licensed, you use the vxdmpadm command to display the basic controller configuration. The following is an example. # vxdmpadm listctlr all CTLR-NAME ENCLR-TYPE STATE ENCLR-NAME ===================================================== c0 Disk ENABLED Disk c2 SENA ENABLED SENA0 c3 SENA ENABLED SENA1 c4 T3 ENABLED T30 c5 T3 ENABLED T31

1-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Identifying Device Path Components To configure and maintain storage devices, you must understand the terminology that describes and locates storage interface boards, storage arrays, and disk drives within a storage array. Each disk drive in a non-hardware RAID Sun storage array is identified by two unique paths: ●

Physical device path (under the /devices directory)

●

Logical device path (under the /dev/dsk and /dev/rdsk directories)

System drivers and applications use the device paths to access specific disk drives.

Physical Device Path Components Physical paths describe the addresses of physical hardware components that connect to a particular device. A typical physical device path is: # ls -l /dev/dsk/c0t0d0s2 lrwxrwxrwx 1 root root 41 Oct 21 21:01 /dev/dsk/c0t0d0s2 ->../.. /devices/pci@1f,4000/scsi@3/sd@0,0:c # ls -l /dev/dsk/c2t1d0s2 lrwxrwxrwx 1 root root 74 Oct 21 21:01 /dev/dsk/c2t1d0s2 ->../.. /devices/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0:c # ls -l /dev/dsk/c3t1d0s2 lrwxrwxrwx 1 root root 74 Sep 24 22:46 /dev/dsk/c2t1d0s2 ->../.. /devices/pci@6,4000/pci@3/SUNW,qlc@4/fp@0,0/ssd@w21000020370c055a,0:c

Physical device paths describe internal hardware paths in a manner that varies depending on system type, interface type, and storage device type. The paths show the location and type of interface cards and also the type of software drivers used. Use the man pages to research physical path components as follows: # man pci # man qlc # man ssd

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-31

Storage Configuration Identification

Logical Device Path Components Logical device paths to disk drives are found under the /dev/dsk directory for block devices and under the /dev/rdsk directory for raw devices. A logical device path is a more basic representation of a physical path. A typical logical device path is: /dev/dsk/c2t3d0s7 Each logical device is linked to a physical device path: # ls -l /dev/dsk/c1t3d0s2 lrwxrwxrwx 1 root root 90 Dec 2 1998 /dev/dsk/c1t3d0s2 -> ../../devices/iounit@f,e1200000/sbi@0,0/SUNW,soc@3,0/ SUNW,pln@a0000800,201cad7e/ssd@3,0:c The following path, /dev/dsk/c0t3d0s2, is an example of a typical logical device path. Standard logical device paths are composed of four components:

1-32

●

Controller address (c0)

●

Target address (t3)

●

Device address (d0)

●

Slice address (s2)

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Configuration Identification

Typical Storage Device Addresses The number of devices associated with each target depends on the type of storage device. The relationship between target and device numbers for software RAID Sun storage is as follows: ●

●

●

●

●

●

SPARCstorage Array 100: ●

Thirty disks

●

Six targets, t0–t5

●

Five devices (d0–d4) associated with each target

SPARCstorage® RSM™ array: ●

Two selectable target ranges

●

Seven disks

●

Seven targets, t0–t6 or t8–t14

●

A single device (d0) associated with each target

Sun StorEdge D1000 array: ●

Two selectable target ranges

●

Eight disks, t0–t3 and t8–t11

●

Twelve disks, t0–t5 and t8–t13

●

A single device (d0) associated with each target

Sun StorEdge A5x00 array: ●

Four selectable target ranges

●

Fourteen disks, targets, t0–t6 and t16–t22

●

Twenty-two disks, targets t0–t10 and t16–t26

●

A single device (d0) associated with each target

Sun StorEdge MultiPack array: ●

Two selectable target ranges for a six-disk model

●

Six disks, targets, t1–t6 or t9–t14

●

Twelve disks, targets t2–t5 and t8–t15

●

A single device (d0) associated with each target

Sun StorEdge MultiPack-FC array: ●

Fifteen selectable target ranges

●

Six disks, targets, t8–t13

●

A single device (d0) associated with each target

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-33

Storage Configuration Identification

Identifying DMP Devices DMP connections can be identified using the format utility as follows: AVAILABLE DISK SELECTIONS: 0. c0t0d0 <SUN2.1G cyl 2733 alt 2 hd 19 sec 80> /sbus@3,0/SUNW,fas@3,8800000/sd@0,0 1. c2t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133> /sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w22000020370c0de8,0 2. c3t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133> /sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w21000020370c0de8,0 Notice that the device paths for devices 1 and 2 have the same disk drive identifier, 20370c0de8. Because the controller numbers are different, devices 1 and 2 are connected to two different controller interfaces in the same system.

1-34

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Storage Array Firmware Several Fibre Channel hardware components have resident firmware that can be at various revision levels. You must verify the minimum levels for this firmware before starting an installation. If necessary, you need to upgrade the firmware to an acceptable revision level. The related Fibre Channel components are: ●

Fibre Channel HBA cards

●

Storage array controller/interface boards

●

Storage array disk drives

Fibre Channel HBA Cards There are several generations of Fibre Channel HBA cards including: ●

FC/S (also known as FC25/S or SOC cards) The FC/S HBAs are SBus-based cards that were used with the older SPARCstorage arrays and SBus systems.

●

FC100/S (also known as SOC+ cards) The FC100/S HBAs were faster versions of the FC/S cards and were also used with older SBus-based systems.

●

FC100/P and FC100/2P (ISP 2100 or ISP 2200 cards) The FC100/P HBAs are PCI bus cards. The FC100/P with ISP 2100 chips are single channel only. The FC100 cards with ISP 2200 chips can be single channel (/P) or dual channel (/2P). There are different firmware files for the ISP 2100 and ISP 2200 versions.

●

Compact PCI (cPCI) Dual Fibre Channel Adapter

Note – The numbers ISP 2100 and ISP 2200 are model numbers of integrated circuit chips on the interface cards.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-35

Storage Array Firmware

Verifying Fibre Channel HBA Firmware Fibre Channel HBA card firmware revisions can be checked with the luxadm utility, which is a standard Solaris OS command that resides in the /usr/sbin directory. There are different luxadm command options for each generation of Fibre Channel HBA cards. However, the most current version of luxadm has a single option (fcode_download) that can be used to verify and upgrade firmware on most Fibre Channel HBA cards. An example of the command output follows. # /usr/sbin/luxadm fcode_download -p Found Path to 0 FC/S Cards Complete Found Path to 0 FC100/S Cards Complete Found Path to 2 FC100/P, ISP2200 Devices Opening Device: /devices/pci@6,4000/SUNW,ifp@2:devctl Detected FCode Version: FC100/P FC-AL Host Adapter Driver: 1.9 00/03/10 Opening Device: /devices/pci@6,4000/SUNW,ifp@3:devctl Detected FCode Version: FC100/P FC-AL Host Adapter Driver: 1.9 00/03/10 Complete The firmware revision in the preceding example is 1.9. Caution – The same luxadm command option also downloads new firmware into the HBA cards.

1-36

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Verifying SPARCstorage Array 100 Firmware You use the luxadm display command to determine SPARCstorage Array 100 controller board firmware revisions. You use the controller address to identify the array you want to examine. An example follows. # /usr/sbin/luxadm display c3 SPARCstorage Array 110 Configuration (luxadm version: 1.42 SunOS 5.6) Controller path: /devices/sbus@6,0/SUNW,soc@d,10000/SUNW,pln@a0000000,78ccf9 :ctlr DEVICE STATUS TRAY 1 TRAY 2 TRAY 3 slot 1 RESERVED Drive: 2,0 Drive: 4,0 2 Drive: 0,1 Drive: 2,1 Drive: 4,1 3 Drive: 0,2 Drive: 2,2 Drive: 4,2 4 Drive: 0,3 Drive: 2,3 Drive: 4,3 5 Drive: 0,4 Drive: 2,4 Drive: 4,4 6 Drive: 1,0 Drive: 3,0 Drive: 5,0 7 Drive: 1,1 Drive: 3,1 Drive: 5,1 8 Drive: 1,2 Drive: 3,2 Drive: 5,2 9 Drive: 1,3 Drive: 3,3 Drive: 5,3 10 Drive: 1,4 Drive: 3,4 Drive: 5,4 CONTROLLER STATUS Vendor: SUN Product ID: SSA110 Product Rev: 1.0 Firmware Rev: 3.6 Serial Num: 00000078CCF9 Accumulate Performance Statistics: Enabled

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-37

Storage Array Firmware

Verifying Sun StorEdge A5x00 Array Firmware You can use the luxadm command to display the firmware revision of Sun StorEdge A5x00 array interface boards. An example follows. # luxadm display AA SENA DISK STATUS SLOT FRONT DISKS (Node WWN) REAR DISKS (Node WWN) 0 Not Installed On (O.K.) 200000203704f9fc 1 On (O.K.) 20000020370c055a Not Installed 2 Not Installed Not Installed 3 On (O.K.) 20000020370d5c28 On (O.K.) 20000020370d54a9 4 Not Installed On (O.K.) 20000020370d528a 5 Not Installed Not Installed 6 On (O.K.) 20000020370d5676 On (O.K.) 20000020370c0b8a SUBSYSTEM STATUS FW Revision:1.09 Box ID:0 Node WWN:5080020000034ed8 Enclosure Name:AA Power Supplies (0,2 in front, 1 in rear) 0 O.K.(rev.-02) 1 O.K.(rev.-02) 2 Not Installed Fans (0 in front, 1 in rear) 0 O.K.(rev.-05) 1 O.K.(rev.-00) ESI Interface board(IB) (A top, B bottom) A: O.K.(rev.-04) GBIC module (1 on left, 0 on right in IB) 0 O.K.(mod.-05) 1 Not Installed B: O.K.(rev.-04) GBIC module (1 on left, 0 on right in IB) 0 Not Installed 1 Not Installed Disk backplane (0 in front, 1 in rear) Front Backplane: O.K.(rev.-04) Temperature sensors (on front backplane) 0:33ºC 1:34ºC 2:36ºC 3:36ºC 4:34ºC 5:33ºC 6:36ºC (All temperatures are NORMAL.) Rear Backplane: O.K.(rev.-04) Temperature sensors (on rear backplane) 0:36ºC 1:36ºC 2:37ºC 3:34ºC 4:33ºC 5:36ºC 6:37ºC (All temperatures are NORMAL.) Interconnect assembly O.K.(rev.-02) Loop configuration Loop A is configured as a single loop. Loop B is configured as a single loop.

1-38

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Verifying Sun StorEdge T3 Array Firmware The Sun StorEdge T3 arrays have internal configuration and control software (pSOS) that you use to verify firmware revisions. You must perform a telnet command to a particular array, log in as user root, and use the ver (version) command to verify the current firmware revision. An example of a typical session follows. $ telnet t3 Trying 129.150.47.115... Connected to purple15. Escape character is ’^]’. pSOSystem (129.150.47.115) Login: root Password: T300 Release 1.00 1999/12/15 16:55:46 (129.150.47.115) t3:/:<1> ver T300 Release 1.14 1999/12/15 16:55:46 (129.150.47.115)

Note – In the preceding example, the firmware version is 1.14.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-39

Storage Array Firmware

Verifying Sun StorEdge A5x00 Disk Drive Firmware You can use the luxadm command to verify the firmware revision of the disk drives, as shown in the following example. The firmware is upgraded using a special program that is contained in the related disk firmware patch. # luxadm disp AA,f1 DEVICE PROPERTIES for disk: AA,f1 Status(Port A): O.K. Vendor: SEAGATE Product ID: ST19171FCSUN9.0G WWN(Node): 20000020370d336f WWN(Port_A): 21000020370d336f Revision: 117E Serial Num: 9812R69750 Unformatted Capacity: 8637.338 MByte Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0xffff Location: In slot 1 in the Front of the enclosure named: AAA Path(s): /dev/rdsk/c2t1d0s2 /devices/pci@6,4000/SUNW,ifp@3/ssd@w21000020370d336f,0:c,raw In the preceding example, the disk drive firmware revision is 117E. Note – You can also use the format utility’s inquiry command option to verify firmware revisions in selected disk drives.

1-40

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Storage Array Firmware

Firmware Upgrade Best Practices Updating storage array firmware can be a complex task. There are strict dependency rules and procedures that must be followed or permanent damage to interface cards can result. Check with your Sun field representative before attempting any firmware updates. The most current versions of firmware are available through software patches. You should read all firmware-related patch README notes carefully. The README notes frequently have specific warnings and procedure requirements that can help prevent extended system downtime. In some cases, permanent HBA damage can result if you try to upgrade firmware from an old version to a new version. Review the patch README notes for instructions informing you to first perform an upgrade to an intermediate revision level. You should always idle your system when upgrading storage-related firmware. Usually, the firmware patch README notes will instruct you to place all systems attached to the storage in single-user mode. Ask for help if you are not sure how to proceed.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-41

Exercise: Recording Your Storage Configuration

Exercise: Recording Your Storage Configuration In this exercise, you complete the following tasks: ●

Review Sun storage features

●

Identify host adapter configurations

●

Identify storage array configurations

●

Verify storage interface firmware revisions

●

Verify array disk drive firmware revisions

Preparation Ask your instructor to identify the system and storage that is assigned for your use during this exercise. If you want to simplify the task of documenting the training system configuration, you can precede this lab with a short lecture describing your particular system configuration.

Task 1 – Reviewing Sun Storage Features Answer the following questions concerning the general features of Sun storage devices and tools. 1.

List at least four major administration tasks associated with VxVM. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

The answers are installation, initialization, volume design, volume creation, and volume administration.

2.

Which of the following are not related to multihost storage access? a.

Multi-initiated SCSI

b.

Dynamic Multipathing

c.

Multihost fiber-optic

d.

Multi-level sequencing

The answers are b and d.

1-42

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration 3.

List four software applications that can be used to configure multipath storage. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

The answers are Redundant Dual Active Controller Driver (RDAC), Solaris Alternate Path Driver (AP), VERITAS Dynamic Multipathing driver (DMP), and Sun StorEdge Traffic Manager software.

4.

List two applications that are used to administer controller-based RAID storage. _____________________________________________________________ _____________________________________________________________

The answers are Sun StorEdge RAID Manager, pSOS, or Sun StorEdge Component Manager.

5.

Which of the following tools assist with swapping out a failed disk drive? a.

vxdiskunsetup

b.

vxdiskadm

c.

vxdisk

d.

vxdctl

e.

luxadm

The answers are b, d, and e.

6.

Which of the following apply to traditional SCSI interfaces? a.

They are SAN compatible.

b.

They transfer data in parallel.

c.

You can attach up to four hosts to a single storage unit.

d.

Maximum data transfer speed is currently 40-Mbytes/sec.

The answers are b and d.

7.

What is the main advantage of differential SCSI interfaces? _____________________________________________________________

The answer is longer maximum cable lengths.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-43

Exercise: Recording Your Storage Configuration 8.

What is the primary reason for using SAN? _____________________________________________________________

The answer is to off-load data traffic from user networks.

9.

What SAN feature isolates data resource from general access? a.

Access control lists (ACL)

b.

Partitioning

c.

Zoning

d.

Segmentation

e.

Isolation protocol

The correct answer is c.

10. What is the primary SAN data transfer mechanism? a.

Packets

b.

Bitmaps

c.

Groups

d.

Frames

The correct answer is d.

Task 2 – Identifying Host Adapter Configurations Perform the following steps to determine the controller numbers of the HBA cards in your classroom system. 1.

Record the model number of your classroom system. Determine this by visual inspection and the uname -a command. _____________________________________________________________

2.

Record the controller numbers of each HBA on your classroom system. Either inspect the /dev/dsk directory contents or use the format utility to examine your system. _____________________________________________________________ _____________________________________________________________

1-44

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration

Task 3 – Identifying Storage Array Configurations Each of the following sections is dedicated to a particular storage array model and describes the most useful methods of gathering more specific storage array information. Before proceeding with this task, record the type of storage arrays attached to your classroom system and how many of each type there are. Determine this by visual inspection. Type of Storage Arrays

Number of Each Type

_______________

_____

_______________

_____

_______________

_____

SPARCstorage Array 100 and Sun StorEdge A5x00 Array Use the luxadm utility to determine very specific information about both of these storage devices. The most commonly used commands are: # # # # # #

luxadm probe luxadm display controller_number luxadm display enclosure_name luxadm display logical_path luxadm display enclosure_name, disk_location luxadm fcode_download -p

For each storage array, record the WWN, the enclosure name (Sun StorEdge A5x00 array only), the number of disks present, and the controller number. WWN (12 or 16 digits)

Enclosure Name

Number of Disks Installed

Controller Number

_______________

_____

_____

_____

_______________

_____

_____

_____

_______________

_____

_____

_____

Note – For the Sun StorEdge A5x00 array, the WWN of the enclosure is not used in the physical path. You must use luxadm display command to determine the WWN of the Sun StorEdge A5x00 units.

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-45

Exercise: Recording Your Storage Configuration

Sun StorEdge D1000 Array The luxadm utility does not recognize the Sun StorEdge D1000 array. Specific array information must be gathered using other tools such as the format utility and visual identification. Use the format utility to determine the controller number of each Sun StorEdge D1000 HBA card and the number of disks in each storage unit. Controller Number

Array Type

Number of Disks

_______

__________

_________

_______

__________

_________

_______

__________

_________

_______

__________

_________

Task 4 – Verifying Storage Interface Firmware Revisions Use the luxadm utility to determine the firmware revision levels of your classroom system storage interface components. 1.

2.

1-46

Record the firmware revision of each HBA card. Controller Number

Host Adapter Firmware Revision

_______

______________________

_______

______________________

_______

______________________

_______

______________________

Record the firmware revision of each SPARCstorage Array 100 controller board or Sun StorEdge A5x00 interface board. Controller Number Revision

Array Controller/IB Firmware

_______

______________________

_______

______________________

_______

______________________

_______

______________________

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Recording Your Storage Configuration

Task 5 – Verifying Array Disk Drive Firmware Revisions Use the luxadm utility or the format utility to determine the firmware revisions of your classroom storage array disk drives. Record the firmware revision of at least one Sun StorEdge A5x00 disk drive if you have this array model. Controller Number

Disk Drive Firmware Revision

_______

______________________

_______

______________________

_______

______________________

_______

______________________

Sun Storage Concepts Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

1-47

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

1-48

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Module 2

Managing Data Objectives Upon completion of this module, you should be able to: ●

List the advantages of using virtual disk management

●

Describe standard RAID terminology

●

List the common features of each supported RAID level including:

●

●

Concatenation – RAID 0

●

Striping – RAID 0

●

Mirroring – RAID 1

●

Mirrored Stripe – RAID 0+1

●

Mirrored Concatenation – RAID 0+1

●

Striped Mirror – RAID 1+0

●

Concatenated Mirror – RAID 1+0

●

Striping with distributed parity – RAID 5

Describe the optimum hardware configuration for each supported RAID level

2-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Virtual Disk Management

Virtual Disk Management Servers today must meet high standards in the areas of availability, performance, scalability, and maintainability.

Availability VxVM provides availability improvements in this area in the following ways: ●

Preventing failed disk drives from making data unavailable The probability of a single disk drive failure increases with the number of disk drives on a system. Data redundancy techniques prevent failed disk drives from making data unavailable.

●

Allowing file systems to grow while they are in use Allowing file systems to grow while they are in use reduces the system downtime and eases the system administration burden.

●

Allowing multiple-host configurations In a dual-host configuration, one host can take over disk drive management for another failed host. This configuration prevents a failed host from making data unavailable.

Performance Many applications today require high data throughput levels. The VxVM products can assist in this area by more efficiently balancing the I/O load across disk drives.

2-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Virtual Disk Management

Scalability Traditionally, file system size has been limited to the size of a single disk drive. Using VxVM techniques, you can create file systems that consist of many disk drives. The fact that there are multiple disk drives is transparent to all applications. The size limit of file systems is increased to the UNIX limit of 1 terabyte (Tbyte). Note – The size of a single file is limited to approximately 800 gigabytes (Gbytes) because of the space required for file system overhead.

Maintainability Administering large installations can be much easier with the assistance of well-designed tools. VxVM has both graphical and command-line tools to assist administrators. VxVM’s graphical administration tool is called VERITAS Enterprise Administrator (VEA). VEA offers complete management assistance for all functions. VxVM also has a number of command-line programs and utilities that are useful and are preferred by many administrators. They can also be used in shell programs to perform virtually all administration tasks.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-3

RAID Technology Introduction

RAID Technology Introduction The RAID concept was introduced at the University of California at Berkeley in 1987 by David Patterson, Garth Gibson, and Randy Katz. Their goal was to show that RAID could be made to achieve performance comparable to or higher than the available performance on single, large expensive disk drives of the day. During the development phase of the project, it was determined that it was necessary to provide redundancy to avoid data loss due to frequent disk drive failures. This aspect of the project became of great importance to the future of RAID.

Supported RAID Standards Many RAID levels are technologically possible but are not commonly used. The complete list of RAID levels includes: ●

RAID 0 – Striping or concatenation

●

RAID 1 – Mirroring

●

RAID 0+1 – Mirroring plus concatenation or striping

●

RAID 1+0 – Concatenation or striping plus mirroring

●

RAID 2 – Hamming code correction

●

RAID 3 – Striping with dedicated parity

●

RAID 4 – Independent reads and writes

●

RAID 5 – Striping with distributed parity

●

RAID 6 – RAID 5 with a second parity calculation

Note – RAID levels 2, 3, 4, and 6 are not available with VxVM. They are not commonly implemented in commercial applications. RAID 0+1 and RAID 1+0 are not true RAID levels but are abstractions composed of more than one RAID level. RAID 6 is a proprietary controller-based RAID scheme owned by a single manufacturer. It is similar to RAID 5, but has additional parity that is written to disk. It has a higher tolerance to disk drive failures than RAID 5 structures.

2-4

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Technology Introduction

RAID Terminology In the explanation of RAID levels in the following pages, a number of technical terms are used to describe RAID structure components. The following are some of the definitions: ●

Stripe unit refers to a sequential group of data blocks on a single disk drive. The stripe unit size is configurable.

●

The terms disk drive and column are synonymous in RAID discussions.

●

Stripe width is the stripe unit size times the number of columns.

●

Transfer rate and I/O per second (IOPS) are performance metrics: ●

Transfer rate is the speed (measured in Mbytes per second) at which a system can move data through its controller. In RAID systems, read and write transfer rates can vary considerably. High transfer rates are particularly valuable for applications that must move large amounts of data quickly, such as document imaging, data mining, or digital video applications.

●

IOPS is a measure of the ability of a storage system to handle multiple, independent I/O requests in a certain period of time. RAID systems with high transfer rates do not always have good IOPS performance. Database and transaction processing systems are examples of applications that typically require high I/O rate performance.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-5

RAID Level Common Features

RAID Level Common Features The section reviews the common features of the following supported RAID levels: ●

Concatenation – RAID 0

●

Striping – RAID 0

●

Mirroring – RAID 1

●

Mirrored Stripe – RAID 0+1

●

Mirrored Concatenation – RAID 0+1

●

Striped Mirror – RAID 1+0

●

Concatenated Mirror – RAID 1+0

●

Striping with distributed parity – RAID 5

Concatenation – RAID 0 The primary reason for using concatenation is to create a virtual disk drive that is larger than one physical disk drive. Concatenation obtains more storage capacity by logically combining portions of two or more physical disk drives. Concatenation also enables you to grow a virtual disk drive by concatenating additional physical disk drive space to it. This technique does not restrict the mix of different size drives and member drives can be of any size. Therefore, no storage space is lost. The example in Figure 2-1 on page 2-7 shows the concatenation of three physical disk drives. Each portion of the concatenation is managed by VxVM and is called a subdisk. A subdisk is the basic unit that VxVM uses to assemble and control all data storage areas.

2-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features The array management software is responsible for taking the three physical disk drives and combining them into one virtual disk drive. This disk drive is presented to the application as a contiguous storage area. Block 1

Physical Disk 1

Block 1000 Block 1 Physical Disk 2

Block 1001

Array Management Software

Block 2000 Block 3000 Physical Disk 3

Block 2001

Virtual Disk

Block 3000

Figure 2-1

RAID-0 Concatenated Structure

The term block represents a disk drive block or sector (512 bytes) of data. Concatenation is the default vxassist configuration.

Advantages The following advantages can be gained by using concatenated structures: ●

Concatenation can improve performance for random I/O because the data is spread over multiple disk drives.

Ask students to discuss the impact of concatenation on performance and cost. ●

One hundred percent of the disk drive capacity is available for user data.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-7

RAID Level Common Features

Limitations The limitations of concatenated structures include: ●

If you use only concatenation, there is no redundancy. Concatenated volumes can be mirrored to achieve redundancy.

●

Concatenation is less reliable because the loss of one disk drive results in the loss of data on all disk drives.

●

When the disk drives are full, the data is spread throughout all the members. However, when the disk drives are not full, the last disk drives are unused, thereby lowering the use of all the drives.

Ask students to discuss the impact of concatenation on failure and recovery.

Striping – RAID 0 The primary reason for using striping is to improve IOPS performance. The performance increase comes from accessing the data in parallel. Parallel access increases I/O throughput because all disk drives in the virtual device are busy most of the time servicing I/O requests. The array management software is responsible for making the array look like a single virtual disk drive. Striping takes portions of multiple physical disk drives and combines them into one virtual disk drive that is presented to the application. As shown in Figure 2-2 on page 2-9, the I/O stream is divided into segments called stripe units (SUs), which are mapped across two or more physical disk drives, forming one logical storage unit. The stripe units are interleaved so that the combined space is made alternately from each slice, and is, in effect, shuffled like a deck of cards. The stripe units are analogous to the lanes of a freeway.

2-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features There is no data protection in this scheme and, because of the way that striping is implemented, the loss of one disk drive results in loss of data on all striped disk drives. Therefore, while this implementation improves performance, it degrades reliability.

Physical Disk 1

Physical Disk 2

Physical Disk 3

SU 1

SU 4

SU 2

SU 5

SU 3

SU 6

Array Management Software

SU SU SU SU SU SU

1 2 3 4 5 6

Virtual Disk SU = Stripe Unit

Figure 2-2

RAID 0 Striped Structure

The layout specification to use for vxassist is layout=stripe.

Advantages The following advantages can be gained by using a striped structure: ●

Performance is improved for large sequential I/O requests and for random I/O. You can optimize stripe unit size for sequential or random access.

●

One hundred percent of the disk drive capacity is available for data.

Ask students to discuss the impact of striping on performance and cost.

Limitations Some of the limitations of striped structures are: ●

There is no redundancy.

●

Striping is less reliable, because the loss of one disk drive results in the loss of data on all striped disk drives.

Ask students to discuss the impact of striping on failure and recovery.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-9

RAID Level Common Features

Mirroring – RAID 1 The primary reason for using mirroring is to provide a high level of availability or reliability. Mirroring provides data redundancy by recording data multiple times on independent spindles. The mirrored disk drives appear as one virtual disk drive to the application. In the event of a physical disk drive failure, the mirror on the failed disk drive becomes unavailable, but the system continues to operate using the unaffected mirrors. The array management software takes duplicate copies of the data located on multiple physical disk drives and presents one virtual disk drive to the application, as shown in Figure 2-3. Block 1 Block 2 Block 3 Block 1

Block 4 Array Management Software Block 1

Block 2 Block 3 Block 4

Block 2

Virtual Disk

Block 3 Block 4

Figure 2-3

RAID-1 Mirror Structure

In VxVM, the mirror is seen as a single logical address, block 0 to n blocks in length. Because of this, VxVM does not concern itself with the format of each individual mirror. One side of the mirror can be striped, and the other side of the mirror can be concatenated. This implementation can be used if there is a lack of enough physical disk drives or for performance reasons. VxVM can guarantee consistent data across both sides of the mirror, no matter what the format, because it writes to a given logical block address. The layout specification to use for vxassist is layout=mirror.

2-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Advantages The following advantages can be gained by using a RAID-1 mirrored structure: ●

There is a fully redundant copy of the data on one or more disk drives. If the mirror resides in a storage array that is attached to a different interface board, a high level of availability can be achieved.

●

All drives can be used for reads to improve performance. Mirroring improves read performance only in a multiuser or multitasking situation where more than one disk drive member can satisfy read requests. Conversely, if there is only a single thread reading from the volume, performance is not improved.

●

You can set up three-way mirroring, but there is a performance penalty. Write performance can suffer up to 44 percent with a three-way mirror.

Ask students to discuss the impact of mirroring on performance and cost. Ask students to discuss the impact of mirroring on failure and recovery.

Limitations Some of the limitations of using a RAID-1 mirrored structure are: ●

Mirroring uses twice as many disk drives, which essentially doubles the cost per Mbyte of storage space.

●

Mirroring degrades write performance by about 15 percent. This is substantially less than the typical RAID-5 write penalty (which can degrade as much as 70 percent).

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-11

RAID Level Common Features

Mirrored Stripe – RAID 0+1 The primary reason for using mirrored stripe volumes is to gain the performance offered by striping and the availability offered by mirroring. Mirrored stripe configurations are a relatively high-cost installation, but many customers consider it a worthwhile investment. As shown in Figure 2-4, two drives are first striped and then mirrored. The reliability is as high as with mirroring. Because the technique of striping is also used, the performance is much better than when using just mirroring.

SU SU SU SU

1 3 5 7

SU SU SU SU

2 4 6 8

Array Management Software Striping

Physical Disk SU SU SU SU

1 3 5 7

SU SU SU SU

2 4 6 8

SU SU SU SU SU SU SU SU

1 2 3 4 5 6 7 8

Virtual Disks

Array Management Software Mirroring

Array Management Software Striping

Figure 2-4

SU SU SU SU SU SU SU SU

1 2 3 4 5 6 7 8

Mirrored Stripe RAID Structure

The layout specification to use for vxassist is layout=mirror-stripe.

2-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

SU SU SU SU SU SU SU SU

1 2 3 4 5 6 7 8

Virtual Disk

RAID Level Common Features

Advantages The primary advantage of mirrored stripe volumes is spreading data across multiple disk drives (improved I/O per second) while providing added redundancy of the data.

Limitations Some of the limitations of using a Mirrored Stripe – RAID 0+1 are: ●

Mirrored stripe volumes suffer the high cost of mirroring, requiring twice the disk drive space of non-redundant volumes.

●

The loss of a single disk drive disables a mirror and results in the loss of redundancy.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-13

RAID Level Common Features

Mirrored Concatenation – RAID 0+1 The primary reason for using mirrored concatenation volumes is to gain the availability offered by mirroring while maximizing storage utilization. Mirrored concatenation volumes can be a relatively high-cost installation, but many customers consider it a worthwhile investment. As shown in Figure 2-5, two drives are first concatenated and then mirrored for increased reliability. Because the technique of concatenation is used, varied storage segments of dissimilar size can be combined to maximize storage utilization.

Blocks 0-499

Blocks 500-999

Array Management Software Concatenation Block 0 Array Management Software Mirroring

Volume

Blocks 0-499

Blocks 500-999

Figure 2-5

Block 999

Array Management Software Concatenation

Mirrored Concatenation RAID Structure

The layout specification to use for vxassist is layout=mirror-concat.

Advantages The primary advantage of mirrored concatenations is increased reliability combined with maximized storage utilization.

Limitations Mirrored concatenated volumes suffer the high cost of mirroring, requiring twice the disk drive space of non-redundant volumes.

2-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Striped Mirror – RAID 1+0 Striped mirror volumes have the performance and reliability advantages of mirrored stripe volumes, but can tolerate a higher percentage of disk drive failures without data loss. Striped mirror volumes also have a quicker recovery time after a disk drive failure because only a single stripe must be resynchronized instead of an entire mirror. As a best practice, use striped mirror volumes for large volumes where failure recovery time and performance are issues. As shown in Figure 2-6, the concept of striped mirrors is fundamentally different from mirrored stripes. In a striped mirror configuration, each stripe is mirrored separately. Additional internal structures called sub-volumes (SVs) are used to implement RAID 1+0 volumes.

SV 1 Array Management Software

SU 1

Mirroring SV 2 Physical Disk Sub-Volumes

Virtual Disks

Array Management Software

SU 1 SU 2

Striping

Virtual Disk

SV 2 Array Management Software

SU 2

Mirroring SV 2

Figure 2-6

Striped Mirror RAID Structure

Note – VxVM refers to the striped mirror configuration as a layered volume or as a Pro volume structure.

The layout specification to use for vxassist is layout=stripe-mirror.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-15

RAID Level Common Features

Advantages The following advantages can be gained by using a Striped Mirror – RAID 1+0 structure: ●

Because each stripe is mirrored separately, a larger number of disk drive failures can be tolerated without disabling the volume. Availability increases exponentially with disk drive (stripe component) count.

●

Striped mirror configurations have the same performance benefits of mirrored stripe volumes.

Limitations Some of the limitations of using a Striped Mirror – RAID 1+0 are: ●

Striped mirror volumes suffer the high cost of mirroring, requiring twice the disk drive space of non-redundant volumes.

●

A mirror cannot be detached and used to perform an online backup. With a RAID 0+1 volume, you can detach a mirror and back it up to tape.

Note – Do not detach a volume mirror to perform online data backups. The active data is not protected by redundancy during the backup operation.

2-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Concatenated Mirror – RAID 1+0 Concatenated mirror volumes generally have a much quicker recovery time after a disk drive failure because only a portion of the concatenation must be resynchronized (instead of an entire mirror). As a best practice, use concatenated mirror volumes for large volumes where failure recovery time is an issue. As shown in Figure 2-6, the concept of a concatenated mirror volume is fundamentally different from a mirrored concatenation volume. In a concatenated mirror configuration, each part of a concatenation is mirrored separately.

Block 1

Block 1

Mirror

Block 1000

Block 1000 Block 1

Block 1001

Block 2000

Block 2001

Block 3000

Block 1001

Mirror

Array Management Software

Block 2000

Block 2001

Mirror

Block 3000 Virtual Disk

Block 3000

Figure 2-7

Concatenated Mirror RAID Structure

Note – VxVM refers to the concatenated mirror configuration as a layered volume and as a Pro volume structure.

The layout specification to use for vxassist is layout=concat-mirror.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-17

RAID Level Common Features

Advantages Because each portion of the concatenation is mirrored separately, recovery time is much faster.

Limitations Some of the limitations of using a Concatenated Mirror – RAID 1+0 are: ●

Concatenated mirror volumes suffer the high cost of mirroring, requiring twice the disk drive space of non-redundant volumes.

●

A mirror cannot be detached and used to perform an online backup. With a mirrored concatenated volume, you can detach a mirror and back it up to tape.

Note – Do not detach a volume mirror when performing online data backups. The active data is not protected by redundancy during the backup operation.

2-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Level Common Features

Striping With Distributed Parity – RAID 5 RAID-5 configurations can be an attractive choice for read-intensive applications that require increased data protection. Three of the RAID levels introduced by the Berkeley Group have been referred to as parity RAID because they use a common data protection mechanism. RAID 3, 4, and 5 all use the concept of bit-by-bit parity to protect against data loss. RAID 3, 4, and 5 all implement the Boolean Exclusive OR (XOR) function to compute parity. Parity is implemented bit-by-bit to corresponding stripe units of member drives, and the result is written to a corresponding parity disk drive found in RAID 3 and 4. As shown in Figure 2-8, the parity (P) is distributed throughout all the member drives in RAID 5.

Disk 1 SU 1 SU 5

SU 9 P(10-12)

Disk 2 SU 2 SU 6 P(7-9) SU 10 Array Management Software

Disk 3 SU 3 P(4-6) SU 7

SU 11

P(1-3)SU 4

SU 12

Disk 4 SU 8

SU = Stripe Unit

SU 1 SU 2 SU 3 SU 4 SU 5 SU 6 SU 7 SU 8 SU 9 SU 10 SU 11 SU 12 Virtual Disk

Figure 2-8

RAID-5 Structure

Additional features include: ●

Independent access is available to individual drives.

●

Data and parity are both striped across spindles.

●

Reads per second can reach the disk drive rate multiplied by the number of disk drives (minus the parity disk) in a column.

Ask students to discuss the impact of RAID 5 on performance, cost, failure, and recovery. The layout specifications to use for vxassist is layout=raid5 (or raid5nolog), logging is the default.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-19

RAID Level Common Features

Advantages Some advantages of RAID-5 volumes are: ●

Parity protects against single disk drive failure.

●

RAID 5 requires only one additional drive beyond those used for data.

●

Read performance can be similar to a RAID 0 striped volume.

Limitations Some limitations of RAID-5 volumes are: ●

A minimum of three disk drives are required to implement RAID 5 in VxVM.

●

RAID 5 cannot be mirrored using VxVM. Redundancy is provided through the parity information.

●

Write-intensive performance is poor. Overall random I/O performance is dependent on the percentage of writes. If more than 20 percent of the I/O operations are writes, consider an alternative RAID option, such as RAID 1+0.

●

There can be severe performance degradation with a failed disk drive in a write-intensive environment.

Point out that this is the reason for hardware-based RAID, such as that used in the Sun StorEdge A3500 and Sun StorEdge A1000 arrays.

2-20

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Exercise: Optimizing System Configurations In this exercise, you complete the following tasks: ●

Review software RAID features

●

Identify availability and performance cabling

●

Optimize RAID-0 volumes

●

Optimize RAID-1 volumes

●

Optimize RAID-5 volumes

●

Optimize RAID-0+1 volumes

●

Optimize RAID-1+0 volumes

●

Identify effective storage utilization

●

Select disk drives for use

Preparation The first several tasks in this exercise are group discussions about optimizing hardware configurations to suit particular volume structures. The last task involves identifying and recording your VxVM server configuration and selecting six disk drives for use during the remainder of this course. You should work on VxVM servers in small groups of two or three. Each group has six disk drives with which to work. Ideally, three of the disk drives are on one array, and three are on a different array. You must examine your classroom setup and determine which disk drives you are going to use. Caution – More than one group might be working on the same server. It is essential that each group is using their own disk drives and does not accidentally reconfigure disk drives that are being used by another group.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-21

Exercise: Optimizing System Configurations

Task 1 – Reviewing Software RAID Features Answer the following questions concerning the general features of software RAID administration. 1.

List four major areas of concern when using software RAID products. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

The answers are availability, performance, scalability, and maintainability.

2.

Which of the following RAID levels are not supported by VxVM? a.

RAID 0

b.

RAID 1

c.

RAID 2

d.

RAID 3

e.

RAID 4

f.

RAID 5

g.

RAID 6

The answers are c, d, e, and g.

3.

Which of the following are characteristics of a RAID-0 concatenated structure? a.

High tolerance to disk drive failures

b.

Maximized disk space utilization

c.

Enhanced write performance

d.

Good random read performance

e.

Data storage space is contiguous

The answers are b, d, and e.

2-22

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations 4.

Which of the following are characteristics of a RAID-0 striped structure? a.

High tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Good random read performance

e.

Data storage space is contiguous

The answers are b, c, and d.

5.

Which of the following are characteristics of a RAID-1 mirrored structure? a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced read performance

d.

Enhanced write performance

e.

Data storage space is contiguous

The answers are a and c.

6.

Which of the following are characteristics of a RAID-0+1 mirrored stripe structure? a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Poor write performance

e.

Data storage space is contiguous

The answers are a and c.

7.

What is the size of disk drive block? _____________________________________________________________

The answer is 512 bytes

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-23

Exercise: Optimizing System Configurations 8.

Which of the following are characteristics of a RAID-1+0 striped mirror structure? a.

High tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Poor write performance

e.

Data storage space is contiguous

The answers are a and c.

9.

Which of the following are characteristics of a RAID-5 structure? a.

Enhanced tolerance to disk drive failures

b.

Maximized disk space availability

c.

Enhanced I/O performance

d.

Potentially poor write performance

e.

Data storage space is contiguous

The answers are a and d.

10. What is another term for striping? _____________________________________________________________ The answer is interleaving.

11. Which of the following RAID structures has the highest tolerance to disk drive failures? a.

RAID 0+1

b.

RAID 5

c.

RAID 1

d.

RAID 1+0

e.

RAID 0

The answer is d.

2-24

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 2 – Identifying Availability and Performance Cabling For increased performance, availability, or both, you can distribute virtual volume components across system boards, host bus adapters (HBAs), and storage array targets, as shown in Figure 2-9. Host System System Board

HBA

c3

System Board

HBA

c4

Array t1

Array t1

t2

t2

t3

Figure 2-9

t3

Availability and Performance Cabling

Discussion – Consider the following:

! ?

What are the general advantages of the physical configuration shown in Figure 2-9?

Multiple boards, HBAs, and arrays reduce the possibility of a catastrophic failure that disables an entire site. Volume mirrors can be distributed for higher availability. Volume stripes can be distributed for higher performance. Distribution across array targets can increase general performance.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-25

Exercise: Optimizing System Configurations

Task 3 – Optimizing RAID-0 Volumes Each of the following volume structures represent some advantage. The advantage might be cost reduction, simplicity, availability, or performance. Availability, performance, or both, can be improved for some RAID structures by optimizing the physical system configuration. For some RAID structures, such as the basic concatenation or stripe shown in Figure 2-10, you can improve performance but not availability. Volume

Volume

Subdisk 1

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Subdisk 2

RAID 0 (Striped)

Subdisk 3 Subdisk 4 RAID 0 (Concatenated)

Figure 2-10 RAID-0 Volume Structures

Discussion – Consider the following:

! ?

1.

What hardware configurations can improve the availability or performance of either structure shown in Figure 2-10?

2.

What is the ideal hardware configuration assuming there are no hardware budget limitations?

Hint – Assume each subdisk or stripe is a different disk drive: losing one disk drive disables the volume. Good – One system board, one HBA, and one array is as good as the concatenation gets. The stripe performance can be improved a lot. Better – One system board, four HBAs, and four arrays provide the best stripe performance. Best – Four system boards, four HBAs, and four arrays provide a slight performance gain.

2-26

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 4 – Optimizing RAID-1 Volumes RAID-1 volumes consist of two mirrors, as shown in Figure 2-11. Each of the mirrors can be a concatenation, but that does not affect the optimized hardware configuration.

Volume

RAID 1 (Mirrored)

Subdisk 1

Subdisk 1

Subdisk 2

Subdisk 2

Subdisk 3

Subdisk 3

Subdisk 4

Subdisk 4

Mirror

Mirror

Figure 2-11 RAID-1 Volume Structures

Discussion – Consider the following:

! ?

1.

What hardware configurations can improve the availability or performance of the structure shown in Figure 2-11?

2.

What is the ideal hardware configuration assuming there are no hardware budget limitations?

Hint – Assume each subdisk is a different disk drive: losing two disk drives can disable the volume. Good – One system board, two HBAs, and two arrays provide near maximum availability, but not much possibility of performance increase. Better – Two system boards, two HBAs, and two arrays provide a slight availability increase, but no performance gains. Best – There is not much else to do except use a SunPlex™ platform configuration.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-27

Exercise: Optimizing System Configurations

Task 5 – Optimizing RAID-5 Volumes RAID-5 volumes are constructed in a fashion similar to a RAID-0 striped volume, but they have the added complication of distributed parity. A typical RAID-5 volume structure is shown in Figure 2-12. RAID 5 Volume Data or Parity

Data or Parity

Data or Parity

Data or Parity

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Figure 2-12 RAID-5 Volume Structures

Discussion – Consider the following:

! ?

1.

What hardware configurations can improve the availability or performance of the structure shown in Figure 2-12?

2.

What is the ideal hardware configuration assuming there are no hardware budget limitations?

Hint – Assume each stripe or log is a different disk drive: losing two disk drives can disable the volume. Good – One system board, one HBA, and one array: a single HBA failure disables the volume. Better – Four system boards, four HBAs, and four arrays provide some availability increase and better performance. Best – There is no best. It is generally all or nothing with RAID-5 structures.

2-28

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 6 – Optimizing RAID 0+1 Volumes As shown in Figure 2-13, RAID 0+1 volumes combine mirroring and striping with the mirror components above the stripes. The primary shortcoming is that the loss of any stripe disables a mirror. Volume

Mirror

Mirror

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Figure 2-13 RAID 0+1 Volume Structure

Discussion – Consider the following:

! ?

1.

What hardware configurations can improve the availability or performance of the structure shown in Figure 2-13?

2.

What is the ideal hardware configuration assuming there are no hardware budget limitations?

Hint – Assume each stripe is a different disk drive: losing two disk drives can disable the volume. Good – Two system boards, two HBAs, and two arrays provide some availability and a performance gain from striping. Better – Two system boards, two HBAs, two hubs, and eight arrays increase availability and improve performance. Best – Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-29

Exercise: Optimizing System Configurations

Task 7 – Optimizing RAID 1+0 Volumes As shown in Figure 2-14, RAID 1+0 volumes combine striping and mirroring with the stripe components above the mirrors. If the hardware is properly configured, a RAID 1+0 volume can tolerate a higher percentage of hardware failures without disabling the volume.

Volume

Stripe Unit

Stripe Unit

Mirror Subdisk

Mirror Subdisk

Mirror Subdisk

Stripe Unit

Mirror Subdisk

Mirror Subdisk

Stripe Unit

Mirror Subdisk

Mirror Subdisk

Mirror Subdisk

Figure 2-14 RAID 1+0 Volume Structure

Discussion – Consider the following:

! ?

What hardware configurations can improve the availability or performance of the structure shown in Figure 2-14?

Hint – Assume each mirror is a different disk drive: you can lose up to four disk drives without disabling the volume. Good – Two system boards, two HBAs, and two arrays provide availability and some performance gains from striping. Primary mirrors are on one array, secondaries are on the other. Better – Two system boards, two HBAs, two hubs, and eight arrays increase availability and improve performance. Best – Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.

2-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Optimizing System Configurations

Task 8 – Identifying Effective Storage Utilization The storage configuration shown in Figure 2-15 is sufficient to simultaneously configure an example of every supported RAID structure. HBA HBA HBA HBA

c6

c5

c4

c3

t1

t32

t64

t96

t2

t33

t65

t97

t3

t34

t66

t98

t4

t35

t67

t99

t5

t36

t68

t100

Array

Array

Array

Array

Figure 2-15 Storage Utilization Study 1.

2.

Circle the disk drives in Figure 2-15 that you would use to build the following RAID structures: ●

A three-disk RAID-0 striped volume

●

A two-disk RAID-0 concatenated volume

●

A two-disk RAID-1 mirrored volume

●

A four-disk RAID-5 volume (no log)

●

A four-disk RAID-0+1 mirror-stripe volume

●

A four-disk RAID-1+0 stripe-mirror volume

If all the disk drives in Figure 2-15 are 9 Gbytes in size, what is the approximate data storage available for each of the structures?

Assume you are using entire disk drives. There is one disk drive left over for a spare.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-31

Exercise: Optimizing System Configurations

Task 9 – Selecting Disk Drives for Use Each workgroup must select six disk drives for use during the rest of this course. The ideal configuration of disk drives is: ●

Three disk drives on one storage array

●

Three more disk drives on a different storage array

You create and destroy disk groups and several different volume structures, so it is important that the disk drives you select are not being used by another group. Use the following commands to select and record the logical addresses of the six disk drives that your group chooses: ●

format

●

luxadm probe

●

luxadm display

Record the logical paths to the six disk drives for your workgroup in the form c2t3d4. Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Caution – If there are other workgroups using the same VxVM server, you must check with them to ensure that you are not using some of their disk drives.

2-32

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

Managing Data Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

2-33

Module 3

VERITAS Volume Manager Installation Objectives Upon completion of this module, you should be able to: ●

List the key elements of pre-installation planning

●

Research VxVM software patch requirements

●

Install the VxVM software

●

Initialize the VxVM software

●

Verify the post-installation environment

●

Prepare for virtual disk drive management

●

Install the VEA client software

●

Use the basic VEA features

●

Use the VxVM error numbering system

3-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installation Planning

Installation Planning VxVM installations vary in size from small desktop systems to large servers with Tbytes of data storage. Regardless of the system size, the installation should be carefully planned in advance.

System Downtime During a new installation or an upgrade, some system downtime is always required. Usually you should schedule system downtime so that it occurs during off-peak system-usage time. Thorough pre-installation planning usually minimizes the system downtime.

Storage Configuration Assessment You must decide which disks should be managed by VxVM. As you further configure VxVM, you must determine the logical organization or grouping of the disk drives. For example, 10 of the disk drives should be allocated for accounting applications, and 20 disk drives should be allocated for marketing applications. You have the option of not placing certain disk drives under VxVM control. This option is useful if you have applications that are currently using file systems or partitions, and you do not want to update the applications’ references to these file systems or partitions. In contrast, you might want to put your system disk under VxVM control so that it can be mirrored. Existing data on disk drives can be encapsulated. When data is encapsulated, the partition is added to VxVM’s control, leaving the partition intact and maintaining the integrity of the data. This action can be done at installation or at a later time. You might also need to plan for new disk storage devices. In addition, you might need to add more memory and larger backup tape systems to compensate for the increased storage load.

3-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installation Planning

Upgrade Resources Some of the most frustrating installation issues can be discovering that you are missing a CD-ROM, discovering that you do not have the needed patches, or discovering that you have misplaced the installation documents. Having all the required CD-ROMs, patches on the appropriate media, and documentation minimizes your frustration. Not only should you have the documentation (for example, release notes and installation procedures), but you should read it. Reading the installation documentation is the only way to ensure that you have all of the required patches.

Licensing VxVM uses license keys to control access. If you have a SPARCstorage Array 100 or a Sun StorEdge A5x00 array attached to your system, VxVM automatically configures a basic-use license. You can also configure non-array drives connected to the same host. Other storage arrays might require manual license installations.

Current System Checkpoint When installing a new and complex application, such as VxVM, you must always be prepared to return your system to its original state.

Backups Not only must you have backups, but you must verify them. If there is a hardware failure or not enough space to facilitate the upgrade, you must be able to recover or back out the software. Perform a complete backup immediately prior to the installation process.

Testing the New Configuration If this is a new installation, test it prior to going into production mode. It would be ideal to test all the components, including the storage subsystem. During this time, any issues related to patches and firmware can be resolved.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-3

Researching Software Patches

Researching Software Patches Before installing the VxVM software, you should research any patches that might be required. Your Sun field representative can assist you. Ensure that you read all of the README notes in all of the patches. Storage-related patches can require careful attention and knowledge because they frequently must be installed in groups and in an exact sequence. Array firmware patches usually install new software drivers that are sometimes designed to work only with a small range of array firmware revisions. Caution – If the mismatch between the system software drivers and the array-resident firmware is too great, the storage arrays can become unavailable. Correcting the problem can be difficult and might require Sun support and hours of downtime.

Researching Current Patch Information Current patch information can be obtained from the external Sun Web site at http://patchpro.sun.com. From the Web site, you can use the PatchPro Interactive tool to display a customized patch listing that reflects your system configuration. You can configure a PatchPro query to include the following information:

3-4

●

Solaris OS release

●

Platform type (system model)

●

Disk arrays and disk drives

●

Tape libraries and tape drives

●

Switches and HBA models

●

SAN products

●

Software versions (common applications, such as VxVM)

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Researching Software Patches Currently, the information is organized and retrieved in separate categories, such as Enterprise Systems Products and Network Storage Products. Figure 3-1 shows a typical SunSolveSM PatchPro Network Storage Products listing for the following configuration: ●

Solaris 9 12/02 Operating System

●

Sun Fire™ V480 server

●

Sun StorEdge A5200 arrays

●

Sun StorEdge T3B arrays

●

X6799A (FC100/2P) Fibre Channel interface cards

Figure 3-1

SunSolve Program PatchPro Information

The configuration used to produce the preceding output purposely uses some out-of-date components. A configuration using all of the most recent components produces little or no output. At the time of writing, PatchPro has still not been updated to reflect the current VxVM version.

Additional patches might be required or recommended by the application vendor.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-5

Researching Software Patches

Installing Patches The following is a typical patch installation scenario for the configuration shown in Figure 3-1 on page 3-5: 1.

Pay close attention to the PatchPro listing Legend section and the Patch Fixes column. The order of the patches can be critical. Firmware patches must be installed with care. You must carefully study all firmware patch README notes before taking any action, especially in the following areas:

2.

●

Keyword and Synopsis section

●

Patches Required With This Patch section

●

WARNING and Patch Installation Instructions sections

Examine the /var/sadm/patch directory to check for patches that were installed after the operating system installation. You can also use the patchadd -p command, but it displays many screens of patches that are incorporated into the currently installed operating system.

3.

Verify all firmware levels before attempting to install firmware patches. Verifying firmware levels varies according to system and storage types. Older products are checked using the luxadm command. Newer products, such as the Sun StorEdge T3 array, require you to use array-resident firmware programs to verify revision levels.

Vendor Software Patches Patches that are distributed on vendor CD-ROMs or listed in vendor documentation are usually out of date. Do your research before starting patch installation.

3-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

Installing VxVM Software With every release of VxVM, there is an installation guide that accompanies the software. This guide is a comprehensive document that defines the various scenarios for installing and upgrading current VERITAS software products, as well as the Solaris OS.

Software Distribution Overview The VxVM software distribution CD-ROM contains the following top-level directories: ●

pkgs The pkgs directory contains all the software files and documentation.

●

scripts The scripts directory contains script files to assist you in performing VxVM upgrades. These scripts (upgrade_start and upgrade_finish) are referenced in the upgrade procedures in the VxVM documentation. There are other scripts (fixsetup, fixstartup, fixmountroot, and fixunroot) that can be used to attempt recovery from difficult boot failure situations. The use of these scripts is not documented, and they should not be used without express instructions from technical support personnel.

●

win32 The win32 directory contains a single disk operating system (DOS) executable, SETUP.EXE, for installing the VEA client software on Microsoft Windows systems.

●

patches The patches directory contains any patches that were critical at the time of the product release.

●

release_notes The release_notes directory contains several small Portable Document Format (PDF) and Adobe® PostScript release documents.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-7

Installing VxVM Software ●

support The support directory contains a group of support tools used to gather configuration information. Use these tools only under the direction of technical support personnel. Sun technical support personnel use a different information gathering tool called Explorer.

●

veritas_enabled The veritas_enabled directory contains many library files to support a wide range of Sun and third-party storage arrays.

Software Package Summary The VxVM software distribution CD-ROM contains the software packages listed in Table 3-1. Installation should be performed in the order shown. Table 3-1 VxVM Software Packages Package

Title

VRTSvlic

VERITAS licensing utilities

VRTSvxvm

VxVM software

VRTSvmdoc

VxVM user documentation

VRTSvmman

VxVM manual pages

VRTSob

VEA Service

VRTSobgui

VEA

VRTSalloc

Intelligent Storage Provisioning

VRTSddlpr

Device Discovery Layer

VRTSvmpro

VERITAS Volume Manager Management Services Provider

VRTSfspro

VERITAS File System Management Services Provider

The VERITAS File System (VxFS) package, VRTSfspro, is required even if you are not going to install the VxFS software. There are several other VxFS packages on the distribution CD-ROM that are not discussed in this module.

3-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

Software Package Installation You should use the pkgadd command to install the VxVM software packages. To avoid package dependency messages, install the packages in the order shown in the following example. 1.

Install the license package and the basic VxVM packages. # pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman

2.

Install the VEA software packages and reference the custom administration file, VRTSobadmin. # pkgadd -a ../scripts/VRTSobadmin -d . VRTSob VRTSobgui

3.

Install the remaining VxVM software packages. # pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note – There is an additional localization package, VRTSmuob, that adds some localized VEA information in French, Japanese, or Chinese. The VRTSmuob package can be added at any time if needed. The example does not show the installation of the VxFS software packages.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-9

Installing VxVM Software

Software Installation User Interaction Throughout the VxVM software installation, you are asked questions, and some warning messages appear. The following sections summarize required interactions by software package.

The VRTSvlic Package Installation Questions This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

The VRTSvxvm Package Installation Questions Install for which version of Solaris? [9, 8, 7] (default: 9): 9 This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

The VRTSvmdoc Package Installation Questions (None) The VRTSvmdoc package does not prompt for any user-input during its installation.

The VRTSvmman Package Installation Questions The following files are already installed on the system and are being used by another package: /opt/VRTS /opt/VRTS/man Do you want to install these conflicting files [y,n,?,q] y

3-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing VxVM Software

The VRTSob Package Installation Questions The selected base directory must exist before installation is attempted. Do you want this directory created now [y,n,?,q] y

The VRTSobgui Package Installation Questions The VRTSobgui package does not prompt for any user-input during its installation.

The VRTSalloc Package Installation Questions This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

The VRTSddlpr Package Installation Questions This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

The VRTSvmpro Package Installation Questions This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

The VRTSfspro Package Installation Questions The following files are already installed on the system and are being used by another package: * /opt/VRTSob * - conflict with a file which does not belong to any package.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-11

Installing VxVM Software Do you want to install these conflicting files [y,n,?,q] y This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y At this point, the VxVM software is installed but not operational. If you reboot the system, you see at least two VxVM error messages similar to the following: VxVM NOTICE V-5-2-3347 Volume Manager not started VxVM NOTICE V-5-2-3365 VxVM not started VxVM Provider initialization warning: Configuration daemon is not accessible Only the vxsvc daemon is running. VxVM must be initialized using the vxinstall utility before it can start successfully.

3-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Initializing VxVM Using the vxinstall Utility

Initializing VxVM Using the vxinstall Utility The VxVM software cannot start correctly until it has been initialized using the vxinstall utility. In previous VxVM versions, a disk group named rootdg containing at least one disk drive was a mandatory requirement for VxVM operation. The VERITAS Volume Manager 4.0 release has eliminated that requirement.

The vxinstall Utility Dialogue As of this release, the vxinstall utility no longer calls the vxdiskadm utility to configure an initial disk group. As a result of the vxinstall changes and the removal of the rootdg disk group requirement, the VxVM initialization process is now greatly simplified. The following is an example of the entire dialogue. # vxinstall VxVM uses license keys to control access. If you have a SPARCstorage Array (SSA) controller or a Sun Enterprise Network Array (SENA) controller attached to your system, then VxVM will grant you a limited use license automatically. The SSA and/or SENA license grants you unrestricted use of disks attached to an SSA or SENA controller, but disallows striping, RAID-5, and DMP on non-SSA and non-SENA disks. If you are not running an SSA or SENA controller, then you must obtain a license key to operate. Licensing information: System host ID: 80960386 Host type: SUNW,Ultra-4 SPARCstorage Array or Sun Enterprise Network Array: found Do you want to use enclosure based names for all disks ? [y,n,q,?] (default: n) n Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk array DISKS, datype = Disk Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk array 5080020000034ed8, datype = SENA Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk array 5080020000029e70, datype = SENA Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk array 60020f200000c3670000000000000000, datype = T3

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-13

Initializing VxVM Using the vxinstall Utility Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added disk array 60020f200000c1930000000000000000, datype = T3 Sep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336 Unlicensed array S/N 60020f200000c3670000000000000000 installed Sep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336 Unlicensed array S/N 60020f200000c1930000000000000000 installed Do you want to setup a system wide default disk group? [y,n,q,?] (default: y) n The installation is successfully completed.

Licensing Requirements The configuration used in the following example features two Sun StorEdge T3B arrays. According to the restrictions outlined in the vxinstall output, an additional license must be installed. If there are no automatically licensed storage arrays attached, the vxinstall utility prompts you for a license key. SPARCstorage Array or Sun Enterprise Network Array: No arrays found. No valid licenses found. VM lite/full license(s) is needed to proceed with vxinstall. See VERTIAS Volume Manager Installation Guide for more details on obtaining and installing the license keys. Are you prepared to enter a license key [y,n,q] (default: y) y Enter your license key : 8ZYE-XJ4R-UPQ3-DOBL-PPO3-PNT6-PRVP

Note – You can also use the vxlicinst utility to manually install a license key at any time.

3-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Initializing VxVM Using the vxinstall Utility

Verifying Licensed Features After the system is initialized using the vxinstall command, you verify the licensed features using the vxdctl license command. Following are examples of several licensed configurations. ●

Example 1 – The following is a system with only Sun StorEdge A5200 arrays attached and an automatically generated license.

# vxdctl license Licensed features: Mirroring Root Mirroring Concatenation Disk-spanning DMP (multipath enabled) Striping (restricted to Sun Enterprise Network Array) RAID-5 (restricted to Sun Enterprise Network Array) ●

Example 2 – The following is a system with only Sun StorEdge A5200 arrays attached and with an additional manually installed license.

# vxdctl license All features are available: Mirroring Root Mirroring Concatenation Disk-spanning Striping RAID-5 VxSmartSync DMP (multipath enabled) ●

Example 3 – The following is a system with only Sun StorEdge T3B arrays attached and with a manually installed license.

# vxdctl license All features are available: Mirroring Root Mirroring Concatenation Disk-spanning Striping RAID-5 VxSmartSync DMP (multipath enabled)

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-15

VxVM Post-Installation Environment

VxVM Post-Installation Environment After the VxVM software is installed and initialized, you must be familiar with the general environment to be an effective administrator.

VxVM System Files During VxVM installation, the following changes are made to the /etc/system file, and VxVM startup files are added to several of the /etc run-level directories.

Changes to the /etc/system File Entries are appended to the end of the /etc/system file to force-load various VxVM drivers, as shown in the example: * vxvm_START (do not remove) forceload: drv/vxdmp forceload: drv/vxio forceload: drv/vxspec * vxvm_END (do not remove)

Run-Level File Additions A number of script files are added to the /etc run-level directories to control the VxVM software when the system is booted or shut down. ●

/etc/rc0.d/K50isisd This script file stops the ISIS service daemon (vxsvc) associated with the VEA graphical interface when transitioning to run-level 0 during a system shutdown.

●

/etc/rcS.d/K29vxvm-vsshutdown This script file sends a shutdown message to VEA clients

●

/etc/rcS.d/K50isisd This script file stops the ISIS service daemon (vxsvc) associated with the VEA graphical interface when transitioning to the single-user run-level.

3-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment ●

/etc/rcS.d/S25vxvm-sysboot This script file runs early in the boot sequence to configure the / and /usr volumes. This file also contains configurable debugging parameters.

●

/etc/rcS.d/S35vxvm-startup1 This script file runs after the / and /usr volumes are available. It also makes other volumes available that are needed by the Solaris OS early in the Solaris OS boot sequence, such as swap and /var.

●

/etc/rcS.d/S85vxvm-startup2 This script file starts I/O daemons, rebuilds the /dev/vx/dsk and /dev/vx/rdsk directories, imports all disk groups, and starts all volumes that were not started earlier in the boot sequence.

●

/etc/rcS.d/S86vxvm-reconfig This script file contains commands to execute the fsck utility on the root partition before anything else on the system executes.

●

/etc/rc2.d/S50isisd This script file starts the ISIS service daemon (vxsvc) associated with the VEA graphical interface during the system boot sequence.

●

/rc2.d/S94vxnm-vxnetd This script file starts the vxnetd daemon if the VVR software option is installed and licensed.

●

/etc/rc2.d/S95vxvm-recover This script file attaches and resynchronizes plexes and starts several VxVM watch daemons, including: vxrelocd, vxcached, and vxconfigbackupd. You can also modify this file to change the default VxVM disk drive failure response from hot relocation to hot sparing.

●

/etc/rc2.d/S96vradmind This script file starts the vradmind daemon if the VERITAS Volume Replicator (VVR) software option is installed and licensed.

●

/etc/rc2.d/S96vxrsyncd This script file starts the vxrsyncd daemon if VVR is installed and licensed.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-17

VxVM Post-Installation Environment

System Startup Messages When the Solaris OS is booted and the VxVM startup files execute, several important boot messages appear. In the following example, the key messages are in bold type. Rebooting with command: boot Boot device: disk:a File and args: SunOS Release 5.9 Version Generic_112233-03 32-bit Copyright 1983-2002 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. VxVM sysboot INFO V-5-2-3245 Starting restore daemon... VxVM sysboot INFO V-5-2-3242 starting in boot mode... NOTICE: VxVM vxdmp V-5-0-34 added disk array DISKS, datype = Disk NOTICE: VxVM vxdmp V-5-0-34 added disk array 5080020000034ed8, datype = SENA NOTICE: VxVM vxdmp V-5-0-34 added disk array 5080020000029e70, datype = SENA NOTICE: VxVM vxdmp V-5-0-34 added disk array 60020f200000c3670000000000000000, datype = T3 NOTICE: VxVM vxdmp V-5-0-34 added disk array 60020f200000c1930000000000000000, datype = T3 Hostname: ns-east-104 VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup... The system is coming up. Please wait. NIS domain name is Ecd.East.Sun.COM starting rpc services: rpcbind keyserv ypbind done. Setting netmask of lo0 to 255.0.0.0 Setting netmask of hme0 to 255.255.255.0 Setting default IPv4 interface for multicast: add net 224.0/4: gateway ns-east-104 syslog service starting. volume management starting. The system is ready. ns-east-104 console login:

3-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

System Startup Processes The following is an example of typical VxVM processes to help you understand how they are started during a system boot operation. # ps -e |grep vx PID TTY 13 ? 296 ?

TIME CMD 0:04 vxconfig 0:05 vxsvc

473 ? 507 ?

0:00 vxrelocd 0:00 vxrelocd

495 ? 509 ? 513 ?

0:00 vxnotify 0:00 vxnotify 0:00 vxnotify

474 ? 512 ?

0:00 vxcached 0:00 vxcached

475 ? 494 ?

0:00 vxconfig 0:00 vxconfig

(vxconfigd)

(vxconfigbackupd) (vxconfigbackupd)

The vxconfigd Daemon The volume configuration daemon (vxconfigd) is started by the /etc/rcS.d/S25vxvm-sysboot script early in the boot process. This daemon must be running for the VxVM software to function.

The vxsvc Daemon The vxsvc daemon is started by the /etc/rc2.d/S50isisd script. The vxsvc daemon services requests from the VEA graphical interface.

The vxrelocd or vxsparecheck Daemons One of these daemons is started by the S95vxvm-recover script during the boot process. The vxrelocd daemons monitor for VxVM failure events and relocate failed subdisks. By default, the /etc/rc2.d/S95vxvm-recover script file starts the newer vxrelocd processes, which provide hot relocation. You can edit the file so that it starts the older vxsparecheck process that provides only hot sparing.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-19

VxVM Post-Installation Environment

The vxnotify Daemons These daemons are started by either the vxrelocd or the vxsparecheck daemons. The vxnotify daemons display events related to disk and configuration changes, as managed by the VxVM configuration daemon, vxconfigd.

The vxcached Daemons The vxcached daemons manage cache volumes associated with space-optimized snapshots. Cache space is added when needed.

The vxconfigbackupd Daemons The vxconfigbackupd daemons are started by the etc/rc2.d/S95vxvm-recover script. These daemons record any VxVM disk group configuration changes in the /etc/vx/cbr/bk directory.

The vxrecover Daemon This daemon can be run by the /etc/rcS.d/S35vxvm-startup1, /etc/rcS.d/S85vxvm-startup2, or the /etc/rc2.d/S95vxvm-recover script files during a system boot, depending on the need for volume repair. The daemon terminates when the repair process is completed. Volume-repair needs differ depending on several variables, such as a system crash. The repair typically involves attaching plexes and resynchronizing mirrors.

3-20

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

System and User Executable Files VxVM uses many different script and binary program files to perform management and administration functions. Many programs call other programs. Many of the script files and programs should not be directly run by a user.

VxVM Software in the /opt Directory The following directories are present in the /opt directory after a standard VxVM software installation: ●

/opt/VRTS This directory contains the VxVM man pages.

●

/opt/VRTSalloc This directory contains the VxVM storage allocation software known as Intelligent Storage Provisioning.

●

/opt/VRTSddlpr This directory contains software to enable and disable the device discovery layer feature associated with the VEA application.

●

/opt/VRTSfspro This directory contains VxFS registry information used by the vxsvc daemon.

●

/opt/VRTSob This directory contains the VEA client and server software.

●

/opt/VRTSlic This directory contains VxVM license installation and verification programs

●

/opt/VRTSvmpro This directory contains VxVM registry information used by the vxsvc daemon.

●

/opt/VRTSvxms This directory contains library files for a DMP plug-in.

●

/opt/VRTSvxvm This directory contains the VERITAS technical manuals.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-21

VxVM Post-Installation Environment

VxVM Software in the /usr/sbin Directory Although there are many VxVM program files in the /usr/sbin directory, only a few are commonly used. They include vxassist, vxstat, vxinfo, vxprint, vxtask, vxinstall, vxdg, vxdisk, and vxdiskadm. # ls /usr/sbin/vx* vxadm vxdiskpr vxassist vxdmpadm vxcache vxedit vxclust vxexport vxcmdlog vxibc vxconfigd vxinfo vxdco vxinstall vxdctl vxiod vxddladm vxmake vxdg vxmemstat vxdisk vxmend vxdiskadd vxnetd vxdiskadm vxnotify vxdiskconfig vxplex

vxpool vxprint vxrecover vxrecover.wrap vxrelayout vxrlink vxrsync vxrvg vxscriptlog vxsd vxsnap vxsp vxspcshow vxstart_vvr

vxstat vxtask vxtemplate vxtrace vxtranslog vxtune vxusertemplate vxvol vxvoladm vxvoladmtask vxvset

VxVM Software in the /etc/vx/bin Directory The script and program files in the /etc/vx/bin directory are called by higher-level user commands and are not commonly used directly. # ls /etc/vx/bin egettxt vxckdiskrm strtovoff vxclustadm ugettxt vxclustipc vsshutdown vxcntrllist vxa5kchk vxconfigbackup vxapslice vxconfigbackupd vxbadcxcld vxconfigrestore vxbaddxcld vxconvarrayinfo vxbootsetup vxcxcld vxcached vxdarestore vxcap-part vxdevlist vxcap-vol vxdevpromnm vxcdsconvert vxdisksetup vxcheckda vxdiskunsetup vxchksundev vxdxcld

3-22

vxedvtoc vxeeprom vxencap vxevac vxldiskcmd vxmirror vxmksdpart vxnewdmname vxparms vxpartadd vxpartinfo vxpartrm vxpartrmall vxprtvtoc vxr5check

vxr5vrfy vxreattach vxrelocd vxresize vxroot vxrootmir vxslicer vxspare vxsparecheck vxsplitlines vxswapctl vxtaginfo vxunreloc vxunroot

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Post-Installation Environment

Verifying Initial Disk Drive Status After a fresh VxVM installation, you should immediately assess the current disk storage status. You should carefully research any existing file systems before initiating encapsulation or initialization operations. The vxdisk utility shows the current VxVM status of all disks drives attached to the system. # vxdisk list DEVICE TYPE c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:none c2t3d0s2 auto:none c2t5d0s2 auto:none c2t16d0s2 auto:none c2t18d0s2 auto:none c2t20d0s2 auto:none c2t22d0s2 auto:none c3t32d0s2 auto:none c3t33d0s2 auto:sliced c3t35d0s2 auto:none c3t37d0s2 auto:none c3t50d0s2 auto:none c3t52d0s2 auto:none c4t1d0s2 auto:none c5t1d0s2 auto:none

DISK testdg01 -

GROUP testdg -

STATUS online online online online online online online online online online online online online online online online online

invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid

Disks that show a status of online invalid are not under VxVM control. Disks that show a status of online have been initialized, but are not assigned to a disk group. When online disks are added to a disk group, they are assigned a name which appears in the DISK column. By default, the disk name is derived from the name of the disk group. What is not evident is that slice 7 of the disk c2t16d0 is mounted with a file system. You must plan for all existing data before proceeding with disk drive encapsulation or initialization.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-23

Preparing for VxVM Disk Drive Management

Preparing for VxVM Disk Drive Management Disk drives can be placed under VxVM software control in two fundamentally different ways. If a disk drive has existing file systems, VxVM will prompt you to encapsulate the disk, thereby preserving any existing data. An alternate method of placing disks under VxVM control is to initialize the disk drive, which destroys existing data. You configure special files to protect specific storage devices from inadvertent use by VxVM command-line programs.

Disk Drive Initialization Process When any disk drive is initialized by VxVM, the disk drive is repartitioned into a standard VxVM configuration. The initialized disk drive has only two slices, slices 3 and 4. One slice is very small and stores VxVM configuration records. The other slice is the rest of the disk drive. When a disk drive is initialized by VxVM, all existing data is lost. Note – When you use VxVM software such as the vxdiskadm utility to manage disk drives, the software takes extensive steps to detect any existing data structures.

VxVM Initialized Disk Format As shown in Figure 3-2 on page 3-25, a physical disk drive that has been initialized by VxVM is divided into two sections called the private region and the public region:

3-24

●

The private region is used for configuration information.

●

The public region is used for data storage.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management By default, VxVM uses partitions 3 and 4 for the private and public regions.

VxVM configuration and management information Private Region

VxVM User Data Storage Public Region

Figure 3-2

VxVM Physical Disk Drive Layout

VxVM requires a single cylinder for the private region. On larger drives, one cylinder can store more than an Mbyte. The public region is configured to be the rest of the physical disk drive. The volume table of contents (VTOC) listing for a freshly initialized VxVM disk drive is shown in the following example. Some output is omitted for clarity. # prtvtoc /dev/dsk/c2t22d0s2 ... ... * First * Partition Tag Flags Sector 2 3 4

5 15 14

01 01 01

0 0 3591

Sector Count 17682084 3591 17678493

Last Sector 17682083 3590 17682083

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-25

Preparing for VxVM Disk Drive Management

Private Region Disk Header The disk header is a block stored in the private region of a disk drive that defines the following import properties of the disk drive: ●

Current host ownership of the disk drive When a disk drive is part of a disk group that is in active use by a particular host, the disk drive is stamped with that host’s host ID (host name). If another VxVM system attempts to access the disk drive, its VxVM daemons detect that the disk drive has a nonmatching host ID (host name) and disallows access until the first system releases the disk drive.

●

Disk identifier A 64-byte unique identifier is assigned to a physical disk drive when its private region is initialized.

Private Region Configuration Database The configuration database (sometimes called configdb) contains information about the configuration of a particular disk group. By default, VxVM keeps four copies of the configuration database per disk group to avoid any possibility of losing the disk group information. Each copy of the configuration database contains the following information: ●

dgname – The disk group name is assigned by the administrator.

●

dgid – The disk group ID is a 64-byte universally unique identifier that is assigned by VxVM to a disk group when the disk group is created. This identifier is in addition to the disk group name.

●

Records – VxVM virtual structure information for the disk group.

Kernel Log The kernel log is kept in the private region on the disk drive and is written by the VxVM kernel. The kernel log contains records describing certain types of actions, such as transaction commits, plex detaches resulting from I/O failures, dirty-region log failures, the first write to a volume, and volume close information. The kernel log is used after a crash or clean reboot to recover the state of the disk group just prior to the crash or reboot.

3-26

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Disk Drive Encapsulation Process When any disk drive is encapsulated by VxVM, existing file systems are preserved and a small portion of the disk drive is used for VxVM overhead. If the disk drive is the system boot disk, VxVM is aware of this distinction and takes special care to preserve all data. Generally, you do not encapsulate disk drives with existing data unless you want to increase availability or performance of the data through the use of software RAID structures. When a disk with existing data structures (such as a mounted file system) is encapsulated, VxVM analyzes the disk structure and takes measures to preserve all existing data and the disk partition map found on block zero.

Disk Drive Before Encapsulation The following shows the partition map and mount information of a Solaris OS disk drive with a mounted file system. For clarity, some output is omitted. # prtvtoc /dev/rdsk/c2t16d0s2 First Partition Tag Flags Sector 2 7

5 0

01 00

0 0

Sector Count

Last Sector

Mount Directory

17682084 17682083 2100735 2100734 /Test

# more /etc/vfstab |grep Test /dev/dsk/c2t16d0s7/dev/rdsk/c2t16d0s7 /Test ufs 2 yes -

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-27

Preparing for VxVM Disk Drive Management

Disk Drive After Encapsulation After encapsulation, using a VxVM utility such as vxdiskadm, the disk’s partition map and mount information are modified as shown in the following example. # prtvtoc /dev/rdsk/c2t16d0s2 First Partition Tag Flags Sector 2 3 4

5 14 15

01 01 01

Sector Count

Last Sector

Mount Directory

0 17682084 17682083 0 17682084 17682083 17674902 7182 17682083

# more /etc/vfstab |grep Test /dev/vx/dsk/testdg/Test /dev/vx/rdsk/testdg/Test /Test ufs 2 yes #NOTE: volume Test (/Test) encapsulated partition c2t16d0s7

3-28

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Protecting Storage Devices From Usage Before initializing or encapsulating storage devices, you can automatically exclude storage devices from VxVM use using two different methods. Depending on which method you use, the effects range from limited exclusion to total exclusion. The exclusion files prevent the vxdiskadm utility and the VEA GUI from using the excluded devices.

Determining the VERITAS Storage Types Although the format utility displays all known storage devices, you must use the vxdmpadm command as shown to display the storage types and enclosure names for use in device exclusion. # vxdmpadm listctlr all CTLR-NAME ENCLR-TYPE STATE ENCLR-NAME ===================================================== c0 Disk ENABLED Disk c2 SENA ENABLED SENA0 c3 SENA ENABLED SENA1 c4 T3 ENABLED T30 c5 T3 ENABLED T31 # vxdmpadm listenclosure all ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE ========================================================================= Disk Disk DISKS CONNECTED Disk SENA0 SENA 5080020000034ed8 CONNECTED A/A SENA1 SENA 5080020000029e70 CONNECTED A/A T30 T3 60020f200000c3670000 CONNECTED T31 T3 60020f200000c1930000 CONNECTED The controller names (CTLR-NAME) and enclosure names (ENCLR-NAME) are used in the appropriate exclude files. The disk array type field (datype) displayed during the system boot process equates to the ENCLR_TYPE field displayed in the output of the vxdmpadm command.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-29

Preparing for VxVM Disk Drive Management

Limited Exclusion You can exclude enclosure, controllers, or devices from use by the vxinstall and vxdiskadm utilities by manually modifying three special files in the /etc/vx directory. The following describes the three files and an explanation of their intent. ●

enclr.exclude Each line of the enclr.exclude file specifies an enclosure to be excluded, for example, sena1.

●

cntrls.exclude Each line of the cntrls.exclude file specifies the address of a controller to exclude, for example, c2.

●

disks.exclude Each line of the disks.exclude file specifies a disk drive to exclude, for example, c0t3d0.

Caution – The three manual exclusion files do not prevent other VxVM commands from seeing and operating on the storage devices. You can still see and perform operations on all the devices using VxVM commands, such as vxdg, vxdisk, vxdisksetup, and vxassist. The manual exclusion files are used to protect specific storage devices from being initialized or encapsulated after an initial software installation. The exclusion files are also useful to protect specific SAN storage devices. You can remove or rename the manual exclusion files after you complete the initialization or encapsulation process.

3-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preparing for VxVM Disk Drive Management

Manual Exclusion File Formats The following example shows the format using each of the different manual exclusion files. # more /etc/vx/enclr.exclude SENA0 # more /etc/vx/cntrls.exclude c4 # more /etc/vx/disks.exclude c0t0d0 c0t1d0 In the previous example, a Sun StorEdge A5000 array (SENA0) is excluded, a Sun StorEdge T3 array connected to controller c4 is excluded, and two internal system disk drives are excluded. If you try to initialize all attached storage using the vxdiskadm utility, you see exclusion messages similar to the following: Select disk devices to add:[<pattern-list>,all,list,q,?] all VxVM INFO V-5-2-428 This disk that you specified has been excluded by the /etc/vx/cntrls.exclude file: /dev/vx/rdmp/c4t1d0s2 These disks that you specified have been excluded by the /etc/vx/disks.exclude file: /dev/vx/rdmp/c0t0d0s2 /dev/vx/rdmp/c0t1d0s2 These disks that you specified have been excluded by the /etc/vx/enclr.exclude file: /dev/vx/rdmp/SENA0_0s2 /dev/vx/rdmp/SENA0_2s2 /dev/vx/rdmp/SENA0_4s2 /dev/vx/rdmp/SENA0_6s2

/dev/vx/rdmp/SENA0_1s2 /dev/vx/rdmp/SENA0_3s2 /dev/vx/rdmp/SENA0_5s2

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-31

Preparing for VxVM Disk Drive Management

Global Exclusion There are two additional files, /etc/vx/vxvm.exclude and /etc/vx/vxdmp.exclude, that should not be manually edited. They are modified indirectly using the vxdiskadm utility option 17, Prevent multipathing/Suppress devices from VxVM’s view. Storage that is suppressed using the vxdiskadm utility, is removed from the view of all VxVM software. This feature is intended for complex configurations that require the coexistence of AP, DMP, and SAN devices. The following is an abbreviated example of excluding a controller from VxVM’s view using the vxdiskadm utility. # vxdisk list DEVICE TYPE DISK c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:none c2t3d0s2 auto:none c2t5d0s2 auto:none c2t16d0s2 auto:none c2t18d0s2 auto:none c2t20d0s2 auto:none c2t22d0s2 auto:none c3t32d0s2 auto:none c3t33d0s2 auto:none c3t35d0s2 auto:none c3t37d0s2 auto:none c3t50d0s2 auto:none c3t52d0s2 auto:none c4t1d0s2 auto:none c5t1d0s2 auto:none -

GROUP -

STATUS online online online online online online online online online online online online online online online online online

# vxdiskadm .... .... Select an operation to perform: 17 Exclude Devices Menu: VolumeManager/Disk/ExcludeDevices VxVM INFO V-5-2-1239

3-32

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid

Preparing for VxVM Disk Drive Management This operation might lead to some devices being suppressed from VxVM’s view or prevent them from being multipathed by vxdmp (This operation can be reversed using the vxdiskadm command). Do you want to continue ? [y,n,q,?] (default: y) y Volume Manager Device Operations Menu: VolumeManager/Disk/ExcludeDevices 1 Suppress all paths through a controller from VxVM’s view 2 Suppress a path from VxVM’s view 3 Suppress disks from VxVM’s view by specifying a VID:PID 4 Suppress all but one paths to a disk 5 Prevent multipathing of all disks on a controller by VxVM 6 Prevent multipathing of a disk by VxVM 7 Prevent multipathing of disks by specifying a VID:PID 8 List currently suppressed/non-multipathed devices .... .... Select an operation to perform: 1 Exclude controllers from VxVM Menu: VolumeManager/Disk/ExcludeDevices/CTLR-VXVM Use this operation to exclude all paths through a controller from VxVM. Enter a controller name [,all,list,listexclude,q,?] c4 All paths to the following enclosures through controller c4 will be hidden from VxVM as a result of this operation: T30 Continue operation? [y,n,q,?] (default: y) y Sep 11 14:21:57 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-50-110 disabled controller /pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0 connected to disk array 60020f200000c36700000000 00000000 VxVM NOTICE V-5-2-1323 The controller c4 will be disabled. The entries for paths on controller c4 will still be visible through VxVM utilities in a disabled state till a reboot. After a reboot these entries will not be seen. ....

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-33

Preparing for VxVM Disk Drive Management .... # reboot .... .... # vxdisk list DEVICE TYPE DISK c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:none c2t3d0s2 auto:none c2t5d0s2 auto:none c2t16d0s2 auto:none c2t18d0s2 auto:none c2t20d0s2 auto:none c2t22d0s2 auto:none c3t32d0s2 auto:none c3t33d0s2 auto:none c3t35d0s2 auto:none c3t37d0s2 auto:none c3t50d0s2 auto:none c3t52d0s2 auto:none c5t1d0s2 auto:none -

GROUP -

STATUS online online online online online online online online online online online online online online online online

invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid

# more /etc/vx/vxvm.exclude exclude_all 0 paths # controllers c4 /pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0 # product # pathgroups #

Note – The format utility still sees the c4 controller and can use it normally.

3-34

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

Installing the VEA The VEA software is composed of server and client software. The server software must reside on the VxVM server. The client software can be installed on the VxVM server, but it can also be installed on one or more remote administration systems. The VEA software is a graphical VxVM administrative interface. As shown in Figure 3-3, you can use the VEA client software to: ●

Run the client software remotely on an administration system

●

Run the client software on the VxVM server and display the VEA GUI locally

●

Run the client software on the VxVM server and display the VEA GUI remotely Remote System VEA Client Software

Network VM Server

Local Display Monitor

VEA Client Software

VEA Server Software

Disk Disk Groups Volumes

Figure 3-3

VEA Server/Client Relationship

To remotely display an application, you set the local DISPLAY variable to point to a remote system: setenv DISPLAY 129.148.152.93:0.0.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-35

Installing the VEA

VEA Software Initialization The VEA GUI is a Java™ technology application that can be run locally on the VxVM server or remotely on any networked system. The VEA client software can run in any Java runtime environment.

VEA Software Package Installation The VRTSob and VRTSobgui packages are both installed on the VxVM server, which includes the VEA client interface and the VEA server software. You can start the client software on the server. However, the VEA client software package, VRTSobgui, is more commonly installed on a remote administration workstation.

VEA Server Software Startup The VEA server software daemon, vxsvc, is automatically started at boot time by the /etc/rc2.d/S50isisd script. You can manually stop and start the VEA server software on the VxVM server using the /etc/init.d/isisd stop (or start) command options. Table 3-2 shows the options you can use to control the /opt/VRTSob/bin/vxsvc program directly: Table 3-2 The /opt/VRTSob/bin/vxsvc Program Options

3-36

Option

Function

-k

Shuts down the vxsvc daemon.

-m

Returns the current VEA service state.

-v

Prints the version of the VEA service.

-r

Specifies the registry file to be used.

-n

Disables client connection authentication.

-f

Starts the vxsvc daemon in the foreground for debugging.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

VEA Client Software Startup You start the VEA client software by manually running the /opt/VRTSvmsa/bin/vmsa script file. The VEA client software can be started and displayed on the server, started on the server and remotely displayed on another system, or loaded and started on a remote system. Start the VEA client software by running the /opt/VRTSob/bin/vea script file: # /opt/VRTSob/bin/vea & You can also use the startup options shown in Table 3-3 as needed. Table 3-3 Startup Options Option

Function

-v

Prints the version of the VEA client software.

-d

Starts the client software in debug mode.

-s

Specifies the fully qualified class name for the skin.

-c

Specifies the absolute path of the configuration file for the current skin.

-p

Specifies the frequency (in minutes) for the VEA GUI to check the status of the VEA server processes.

-cp

Specifies the user’s class path (for libraries) to be appended to the application’s class path.

-DVAR

Specifies the environment variable to be set pointing to the Java software directory. This option works only on Microsoft Windows.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-37

Installing the VEA

Host Connection Window When the client software starts, a Connection window, as shown in Figure 3-4, is displayed. Enter the name of the VxVM server and the appropriate authentication information.

Figure 3-4

VEA Host Connection Window

If you enable the Remember password feature, the next time you connect you select the hostname from the pull-down menu. The Username and Password fields are automatically configured.

3-38

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Installing the VEA

Resolving Low-Bandwidth Access Problems If you need to access a VxVM server using a low-bandwidth connection, such as a dial-up modem, you must take special measures to prevent fatal connection errors.

Low-Bandwidth Download Failures During the initial VEA client-to-server connection cycle, approximately 7 Mbytes of VEA plug-ins are downloaded from the VxVM server into the client $HOME/.VRTSob directory. In a low-bandwidth environment, the download process commonly fails approximately 15 minutes into the process with cryptic and misleading errors such as File System Full and Java exception errors.

Low-Bandwidth Corrective Measures Perform the following steps to permanently eliminate low-bandwidth download failures: 1.

The first time you start the VEA client software, increase the host discovery frequency from the default value of 3 minutes to approximately 45 minutes. % vea -p 45 &

2.

In the VEA, on the Tools menu, perform the following steps: a.

Select Preferences, and then select General.

b.

Disable the Delete temporary files on exit feature. After the first-time download, the plug-in download files remain in the $HOME/.VRTSob directory and do not need to be downloaded again.

3.

Initiate the connection to the VxVM server.

The first connection might take up to 20 minutes to download the plug-in files. Subsequent connections take less than 2 minutes. Administrative operations response times are acceptable even through a 56-Kbyte modem. If the connection fails while performing administrative tasks, reconnect using the toolbar Connect icon.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-39

Using Basic VEA Features

Using Basic VEA Features Before using the VEA GUI for administrative duties, you must be familiar with the VEA components and options.

Main Window Functional Areas The VEA GUI has distinct functional areas, as shown in Figure 3-5. Menu bar

Object tree

Figure 3-5

Grid area

Tool bar

Message area

VEA GUI

Note – The Actions menu entries change according to the type of objects being displayed in the grid area. Some of the toolbar icons’ functions also change as different objects are displayed in the grid area.

3-40

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Menu Bar Functions The menu bar in the VEA GUI has the functions shown in Figure 3-6. The Actions menu entries change according to the type of objects currently displayed in the grid area. The tear off menu opens a separate window relating to the tabs in the current grid area display. The tear off feature is useful when analyzing multiple aspects of a grid area display.

Figure 3-6

Menu Bar Functions

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-41

Using Basic VEA Features

Toolbar Buttons The toolbar, shown in Figure 3-7, provides direct access to general VEA functions. Some of the toolbar selections change according to the type of objects being displayed in the grid area.

Figure 3-7

Toolbar Icons

All the toolbar functions are available elsewhere in menus, but the toolbar offers a convenient way to access commonly used functions. The toolbar buttons perform the following functions:

3-42

●

Connect to a VEA server

●

Disconnect from the current VEA server

●

Create a new volume

●

Create a new disk group

●

Search for virtual objects by criteria

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Object Tree Pane The object tree window, shown in Figure 3-8, has an icon for every type of VEA object that is referenced during VxVM administration. The objects are arranged in a hierarchy starting with VxVM servers at the top. You can expand small nodes on the object tree branches to display detailed information about the node’s subject.

Figure 3-8

Object Tree Expansion

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-43

Using Basic VEA Features

Grid Area Pane When you select an object tree icon with the first mouse button, expanded configuration information about that object appears in the grid area. The grid area display, shown in Figure 3-9, results from selecting Enclosure in the object tree.

Figure 3-9

3-44

Grid Area

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Resizing Display Panes The object tree and message areas, shown in Figure 3-10, have two types of pane resizing controls: arrows for fixed size changes and a slider for variable size changes.

Pane resizing arrows

Pane resizing bar

Figure 3-10 Pane Resizing Controls In the previous example, the object tree pane has been widened using its resizing bar, and the message area has been fully collapsed using its resizing arrow so that it is no longer visible.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-45

Using Basic VEA Features

Modifying Preferences The Preferences window, accessed by selecting Preferences from the Tools menu, contains three tabs. ●

Appearance tab (shown in Figure 3-11) – Used to modify the general look and feel of the VEA GUI.

Figure 3-11 Preferences Window – Appearance Tab

3-46

●

General tab – Used to modify settings so that any downloaded plug-in files are retained locally when you exit VEA. This can save 45 minutes of download time if you are running the VEA client software from a dial-up modem.

●

Volume Manager General tab – Present only when you are connected to a VxVM server and modifies several minor features.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Customizing the Grid Display Using drag-and-drop and the display tabs, you can modify the type of data displayed in the grid area, and you can customize the location of the data status columns.

Rearranging Grid Display Status Columns You can temporarily rearrange display columns in the grid using drag-and-drop on the column headers. The effect is temporary.

Using the Grid Display Data Tabs Each type of grid display has data tabs associated with it. You can use the tabs to display different information related to the current grid display. By default, when a disk group is displayed in the grid area, the Disks tab is active. In the example shown in Figure 3-12, the Volumes tab is selected so that all volumes associated with the sdgA disk group are displayed.

Figure 3-12 Grid Display Data Tabs

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-47

Using Basic VEA Features

Examining VEA Command Logs The VxVM server logs commands resulting from VEA client software operations in the /var/vx/isis/command.log file. The log file is a useful tool for learning VxVM command-line program syntax. Examples of typical command log entries: Description: Add Disk Date: Tue Dec 11 20:09:15 2003 Command: /usr/sbin/vxdg init sdgA sdgA00=c3t32d0s2 sdgA01=c3t33d0s2 sdgA02=c3t35d0s2 sdgA03=c3t37d0s2 Output: Exit Code: 0 Description: Date: Command: sdisk01 Output: Exit Code:

Rename Disk Tue Dec 11 20:26:15 2003 /usr/sbin/vxedit -g sdgA -v rename sdgA01

0

Description: Create Volume Date: Wed Dec 12 11:51:41 2003 Command: /usr/sbin/vxassist -g sdgA -b make newvol_01 1638400s layout=nostripe Output: Exit Code: 0

Note – An exit code of 0 means the command executed successfully. See the vxintro man page for a list of standard exit codes.

3-48

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Basic VEA Features

Using the VEA Search Tool You can use the VEA Search form, as shown in Figure 3-13, to compile configuration summaries for supported VxVM objecting including controllers, enclosures, disks, disk groups, volumes, and file systems. In the example shown in Figure 3-13, a search was made for all volumes at least 1 Gbyte in size.

Figure 3-13 VEA Search Form To use the search tool, perform the following steps: 1.

Use the pull-down lists and type-in fields to define the criteria to use in your search.

2.

Click Add to add the criteria to the criteria list.

3.

Repeat the previous two steps for each criteria to use.

4.

Click the Search Now button to perform the search and display the results.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-49

Decoding VxVM Error Messages

Decoding VxVM Error Messages The VxVM 4.0 software error message format is organized differently than in previous VxVM releases. All VxVM 4.0 error messages contain unique error numbers that are listed in the VERITAS Volume Manager 4.0 Troubleshooting Guide. The new error message numbers are grouped into two sections as follows: ●

Errors V-5-0-2 through error V-5-0-386

●

Errors V-5-1-90 through error V-5-1-5929

Error message are composed of the following 5 fields: ●

Product name

●

Product component

●

Severity level

●

Error message number

●

Message text

An example of a typical error message follows. # vxdg deport dgSP VxVM vxdg ERROR V-5-1-584 Disk group dgSP: Some volumes in the disk group are in use Most of the error listings in the VERITAS Volume Manager 4.0 Troubleshooting Guide contain a description of the error and, if appropriate, suggested administrative actions.

3-50

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Exercise: Configuring VxVM In this exercise, you complete the following tasks: ●

Review key lecture points

●

Install the VxVM software

●

Verify the VxVM system files

●

Evaluate the storage configuration

●

Install the VEA client software

●

Start the VEA client software

●

Customize the VEA client appearance

●

Navigate the VxVM technical manuals

●

Use the VxVM error numbering system

Preparation If you are installing the VxVM software on a central server, your instructor must perform the installation as a demonstration. 1.

Ask your instructor for the location of the VxVM software. VxVM location: ______________________________

2.

If the lab system does not have certain Sun storage array models attached, you are asked for a license string during the VxVM VRTSvlic package installation. Ask your instructor for a demonstration license string. Demo license: _____________________________

The Adobe Acrobat Reader program must be available to the students to examine the PDF versions of the VERITAS documents. Discuss the classroom configuration and the process students should follow until everyone is clear about how to proceed. Divide students into groups depending on how many systems you have available for VxVM installation. You can install and initialize the VxVM software yourself while students watch. It is up to you to determine which method works best in your classroom configuration.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-51

Exercise: Configuring VxVM

Task 1 – Reviewing Key Lecture Points Answer the following questions about the major areas of concern during the VxVM installation and initialization process. 1.

Which of the following are not part of planning a VxVM installation? a.

Estimating system downtime

b.

Backing up your system

c.

Running a system performance check

d.

Verifying correct boot disk configuration

e.

Determining licensing requirements

The answer is c.

2.

Which of the following are useful for researching patch requirements? a.

The patch README notes

b.

The prtvtoc command

c.

The patchadd command

d.

The PatchPro tool

e.

The /var/adm directory contents

The answers are a, c, and d.

3.

Which of the following VxVM packages should be installed first? a.

VRTSvmman

b.

VRTSob

c.

VRTSvxvm

d.

VRTSvlic

e.

VRTSvmdoc

The answer is d.

3-52

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM 4.

What is the correct order for the following installation steps? a.

Install storage firmware patches.

b.

Install VxVM software patches.

c.

Encapsulate the system boot disk.

d.

Install required operating system patches.

e.

Install VxVM software packages.

The correct order is d, a, e, b, and c.

5.

Which slices does VxVM use when it initializes a disk drive? a.

Slices 6 and 7

b.

Slices 2 and 4

c.

Slices 3 and 4

d.

Slices 0 and 1

e.

Slices 4 and 7

The answer is c.

6.

What is the default amount of space that VxVM requires for a disk drive’s private region? a.

4800 sectors

b.

2048 sectors

c.

1648 sectors

d.

4096 sectors

e.

1024 sectors

The answer is b.

7.

When should you use VxVM disk drive encapsulation instead of disk drive initialization? _____________________________________________________________ _____________________________________________________________

The answer is when you want to preserve existing data on the disk drive.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-53

Exercise: Configuring VxVM 8.

What is the primary purpose of the enlr.exclude, cntrls.exclude, and disks.exclude files? a.

To make storage devices invisible to all VxVM software

b.

To make storage devices invisible to the AP software

c.

To make storage devices invisible to the vxinstall and vxdiskadm utilities

d.

To make storage devices invisible to the DMP software

The answer is c.

9.

Which of the following are not VxVM system processes? a.

vxnotify

b.

vxdiskadm

c.

vxconfigd

d.

vxrecover

e.

vxdctl

The answers are b and e.

10. Which directory contains the VxVM technical manuals? a.

/opt/VRTSdocs

b.

/etc/vx/docs

c.

/opt/VRTSvxvm/docs

d.

/usr/vx/docs

The answer is c.

3-54

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 2 – Installing the VxVM Software To install the VxVM software, complete the following steps: 1.

Log in as user root on the system attached to the storage arrays.

2.

Change to the VxVM installation package location.

3.

Install the license package and the basic VxVM packages.

# pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman 4.

Install the VEA software packages and reference the custom administration file, VRTSobadmin.

# pkgadd -a ../scripts/VRTSobadmin -d . VRTSob VRTSobgui 5.

Install the remaining VxVM software packages.

# pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note – There is an additional localization package, VRTSmuob, that adds some localized VEA information in French, Japanese, or German. The VRTSmuob package can be added at any time if needed.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-55

Exercise: Configuring VxVM 6.

During the installation, answer yes to all questions unless you feel there are serious problems. If there are problems, ask your instructor for advice.

7.

After the package installation is completed, log out and log back in again. The installation process usually alters search-path values.

8.

Use the vxinstall utility to initialize VxVM and answer the following key questions: Do you want to use enclosure based names for all disks? [y,n,q,?] (default: n) n Do you want to setup a system wide default disk group? [y,n,q,?] (default: y) n

Note – If your system does not have Sun SPARCstorage 100 or Sun StorEdge A5x00 arrays attached, you are prompted for a license key. 9.

Verify that the vxprint status command completes without errors. # vxprint

10. Verify that the following VxVM command and man page locations are configured in your environment. Edit the .profile or .cshrc files as necessary. # echo $PATH /usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bin:/op t/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin # echo $MANPATH /usr/man:/opt/VRTS/man:/opt/VRTSvlic/man The correct syntax for .profile file entries in the Bourne shell environment is: PATH=$PATH:/opt/VRTS/bin:/opt/VRTSalloc/bin:/opt/VRTSvlic/b in:/opt/VRTSvmpro/bin /etc/vx/bin MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/man export PATH MANPATH The correct syntax for .cshrc file entries in the C shell environment is: set path = ($path /opt/VRTS/bin /opt/VRTSalloc/bin /opt/VRTSvlic/bin /opt/VRTSvmpro/bin /etc/vx/bin) setenv MANPATH ’/usr/man:/opt/VRTS/man:/opt/VRTSlic/man’

3-56

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 3 – Verifying the VxVM System Files Perform the following steps to verify the post-installation files: 1.

Verify that the following directories are present in the /opt directory.

# ls /opt SUNWits VRTS VRTSddlpr SUNWrtvc VRTSalloc VRTSfspro 2.

VRTSvxvm

Verify that the following software is present in the /usr/sbin directory.

# cd /usr/sbin # ls vx* vxadm vxassist vxcache vxclust vxcmdlog vxconfigd vxdco vxdctl vxddladm vxdg vxdisk vxdiskadd vxdiskadm vxdiskconfig 3.

VRTSob VRTSvmpro VRTSvlic VRTSvxms

vxdiskpr vxdmpadm vxedit vxexport vxibc vxinfo vxinstall vxiod vxmake vxmemstat vxmend vxnetd vxnotify vxplex

vxpool vxprint vxrecover vxrecover.wrap vxrelayout vxrlink vxrsync vxrvg vxscriptlog vxsd vxsnap vxsp vxspcshow vxstart_vvr

vxstat vxtask vxtemplate vxtrace vxtranslog vxtune vxusertemplate vxvol vxvoladm vxvoladmtask vxvset

Verify that the following software is present in the /etc/vx/bin directory.

# ls /etc/vx/bin egettxt vxckdiskrm strtovoff vxclustadm ugettxt vxclustipc vsshutdown vxcntrllist vxa5kchk vxconfigbackup vxapslice vxconfigbackupd vxbadcxcld vxconfigrestore vxbaddxcld vxconvarrayinfo vxbootsetup vxcxcld vxcached vxdarestore vxcap-part vxdevlist vxcap-vol vxdevpromnm vxcdsconvert vxdisksetup vxcheckda vxdiskunsetup vxchksundev vxdxcld

vxedvtoc vxeeprom vxencap vxevac vxldiskcmd vxmirror vxmksdpart vxnewdmname vxparms vxpartadd vxpartinfo vxpartrm vxpartrmall vxprtvtoc vxr5check

vxr5vrfy vxreattach vxrelocd vxresize vxroot vxrootmir vxslicer vxspare vxsparecheck vxsplitlines vxswapctl vxtaginfo vxunreloc vxunroot

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-57

Exercise: Configuring VxVM

Task 4 – Evaluating the Storage Configuration Perform the following steps to evaluate your storage configuration: 1.

Type the vxdmpadm listctlr all command and the format command to evaluate your storage configuration. Use the Control-D keyboard sequence to exit from the format utility.

2.

Record the controller name, enclosure name, and number of devices associated with each enclosure. Controller Number

3-58

Enclosure Name

Number of Devices

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 5 – Installing the VEA Client Software Use the following steps to install the VEA client software on your administration system: 1.

On your administration workstation, change to the location of the VRTSobgui software package. Your instructor should have given you the location of the VxVM software in a previous exercise.

2.

Install the VEA client software on your administration system:

# pkgadd -d . VRTSobgui Processing package instance from VERITAS Enterprise Administrator The selected base directory must exist before installation is attempted. Do you want this directory created now [y,n,?,q] y Using as the package base directory. ## Processing package information. ## Processing system information. ## Verifying disk space requirements. ## Checking for conflicts with packages already installed. ## Checking for setuid/setgid programs. This package contains scripts which will be executed with super-user permission during the process of installing this package. Do you want to continue with the installation of [y,n,?] y

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-59

Exercise: Configuring VxVM

Task 6 – Starting the VEA Client Software To start the VEA client software, complete the following steps: 1.

Start the VEA client software on your administration system: # vea &

2.

Perform the following steps in the Session Initialization window, as shown in Figure 3-14: a.

Type the VxVM server host name.

b.

Type the user name root.

c.

Type the password.

d.

Ensure that the Remember password box is checked.

Figure 3-14 VEA Client Initialization The VEA GUI initializes after a short delay.

3-60

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 7 – Customizing the VEA GUI Appearance Perform the following steps to customize the appearance of the VEA GUI: 1.

Click the left mouse button on the Tools menu, and then select Preferences from the pop-up menu.

2.

In the Preferences window, click the Appearance tab.

3.

Click Items and select Toolbar.

4.

Click Icons and Text.

5.

Click OK.

Task 8 – Navigating the VxVM Technical Manuals There are several technical manuals installed in the /opt/VRTSvxvm/docs directory. Perform the following steps to examine the content of the manuals: 1.

Change to the VxVM manual directory. # cd /opt/VRTSvxvm/docs # ls pitc_ag.pdf vxvm_hwnotes.pdf vxvm_ag.pdf vxvm_ig.pdf vxvm_cdsag.pdf vxvm_ispag.pdf

2.

vxvm_tshoot.pdf vxvm_ug.pdf

View the contents of the PDF manuals using the Adobe Acrobat Reader (acroread) program The hypertext links and search features of the Adobe Acrobat Reader are useful when you are looking for specific information in the manuals. Use the Control-F sequence to enable the Adobe Acrobat Find window.

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-61

Exercise: Configuring VxVM 3.

Use the Adobe Acrobat Find feature to answer the following questions for each of the indicated VxVM technical manuals. a.

Answer the following questions for the vxvm_hwnotes.pdf VxVM manual: 1.

Which Sun storage array models are supported for boot disk mirroring? _____________________________________________________ _____________________________________________________ _____________________________________________________

2.

What is the mirroring configuration requirement when you upgrade T3B firmware? _____________________________________________________ _____________________________________________________ _____________________________________________________

b.

Answer the following questions for the vxvm_ig.pdf VxVM manual: 1.

Is the Solaris OS 2.6 release supported by the current VxVM software? _____________________________________

2.

When is the installvm script used? _____________________________________________________ _____________________________________________________ _____________________________________________________

3-62

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxVM

Task 9 – Using the VxVM Error Numbering System Perform the following steps to research the use of the VERITAS unique error numbering system. Use the Adobe Acrobat Reader (acroread) program to review Chapter 5, Error Messages, of the VERITAS Volume Manager Troubleshooting Guide. Answer the following questions: 1.

What are the five fields in the error message format? _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

2.

What are the six severity levels? _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

3.

What is the suggested action for error number V-5-0-145? _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

VERITAS Volume Manager Installation Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

3-63

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

3-64

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Module 4

VERITAS Volume Manager Basic Operations Objectives Upon completion of this module, you should be able to: ●

Describe the function of VxVM disk groups

●

List disk group administrative operations including: ●

Initialize disk drives for VxVM use

●

Create disk groups

●

Add and remove disk drives for a disk group

●

Import and deport disk groups

●

Destroy a disk group

●

Rename VxVM disk drives

●

Administer disk groups using the vxdiskadm utility

●

Administer disk groups using command-line programs

●

Administer disk groups using the VEA GUI

4-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

VxVM Disk Group Functions A disk group is a collection of VxVM disk drives that share a common configuration. Volume structures are created within a disk group and are confined to the disk drives that are associated with that disk group.

Primary Functions of a Disk Group Disk groups have two primary functions: ●

Assist administrative management

●

Provide increased data availability

Easier Administration Disk groups enable you to group disk drives into logical collections for administrative convenience. You can group them according to department or application. For example, you can create separate disk groups for sales, finance, and development.

Increased Data Availability You can move a disk group and its components as a unit from one host machine to another. This feature provides higher availability of the data in the following ways:

4-2

●

If one system fails, another system running VxVM can import the failed system’s disk group and provide access to it.

●

The first system deports the disk group.

●

Deporting a disk group disables access to that disk group by the first system. Another host can then import the disk group and start accessing all the disk drives in the disk group.

●

The second system imports the disk group and starts accessing it.

●

A host can only import disk groups with unique names. Therefore, all disk groups on all systems should be given unique names, with the exception of the rootdg disk group.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

VxVM Disk Drives There are two phases to bringing a physical disk drive under VxVM control. Sometimes both operations are done in one step and you are unaware that the process is more complex. When you bring a disk drive under VxVM control, you can: ●

Add it to a new or existing disk group

●

Add it to the free-disk pool

The easiest operation is to add a disk drive to the free-disk pool. The vxdisksetup command repartitions the disk drive into VxVM format, and then a blank header is written to the disk drive. If you add a disk drive to a disk group, the disk drive is assigned a unique media name and it is associated with a disk group object. This information is then written into the blank header on the disk drive.

Disk Drive Media Names Unless you intervene, the default media names that are assigned to disk drives are based on either the disk group name or the logical device path to the disk drive. Default disk group-based media names are similar to dgX01 or DGa04. The default device path-based media names assigned by some command-line programs are similar to c3t0d16s2 or c5t4d0s2. Device path-based media names can lead to confusing status and configuration listings.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-3

VxVM Disk Group Functions

Standard VxVM Disk Groups A disk group is a collection of VxVM disk drives that share a common configuration. Typically, the disk group contains volumes that are all related in some way, such as file system volumes that belong to a particular department or database volumes that are all tables for a single database. Each disk group is owned by a single host system. The current ownership is written into all configuration records. Many of the disk drives in the disk group have a copy of the configuration record. A disk group and all its components can be moved as a unit from one host system to another. Usually, both host systems are connected to the same dual-ported storage arrays. As shown in Figure 4-1, even though a second host is attached to the same storage array, access is allowed only to the current owner of the disk group. A disk group can be deported from one host and imported by a different host, but this is generally used as an emergency solution to a catastrophic host system failure. When a disk group is imported by a different host, the name of the new host is written into the disk-based VxVM configuration records. Host 1 The disk group is owned by Host 1

X A

Host 2

A

Disk Group

Volume

Volume

Storage Array

Figure 4-1

4-4

VxVM Disk Group Ownership

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Functions

Shared VxVM Disk Groups VxVM has an optional shared disk group feature that is licensed. The shared disk group feature allows two or more host systems simultaneous access to the same disk group. The host systems are referred to as nodes in the SunPlex system environment. Shared disk groups are used in the Sun Cluster software environment to support the ORACLE® 9i Real Application Clusters (RAC) database application. Multiple ORACLE 9i RAC hosts can access a single database image. As shown in Figure 4-2, shared disk groups are owned by a cluster_name and not by the name of a single host. When a shared disk group is imported by any of the attached nodes, the name of the Sun Cluster software cluster (cluster_name) is written into the disk-resident VxVM configuration records, and the disk group is automatically accessible by all of the attached nodes. Host 2

Host 1 The disk group is owned by a cluster_name

A0 A1

B0 B1

Disk Group

Volume

Volume

Sun StorEdge A5200 Array

Figure 4-2

Shared VxVM Disk Group Ownership

Note – To prevent data corruption, all write activity must be coordinated between the nodes. The ORACLE 9i RAC database uses a global lockmanagement scheme to accomplish this coordination.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-5

VxVM Disk Group Functions

Cross-Platform Data Sharing Disk Groups A new VxVM disk group format called Cross-Platform Data Sharing (CDS) allows data residing on multi-host storage arrays to be utilized by different operating systems. The CDS feature is not licensed by Sun but, by default, disk drives are initialized in the CDS format. The new cdsdisk partition map allocates all disk space to slice 7 as shown in the following example. Part Tag 0 unassigned 1 unassigned 2 backup 3 unassigned 4 unassigned 5 unassigned 6 unassigned 7 -

Flag wm wm wu wm wm wm wm wu

Cylinders 0 0 0 - 4923 0 0 0 0 0 - 4923

Size 0 0 8.43GB 0 0 0 0 8.43GB

Blocks (0/0/0) 0 (0/0/0) 0 (4924/0/0) 17682084 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (4924/0/0) 17682084

Overriding the CDS Format Technically, the cdsdisk format does not interfere with standard VxVM operation. However, if you are not comfortable using the cdsdisk format at your site, you can disable it so that the vxdiskadm utility uses the sliced format when initializing disk drives. Before initializing or encapsulating any disk drives for VxVM use, use the vxdiskadm utility option 22, Change/Display the default disk layouts, to modify the default disk format and private region size. The changes are stored in the /etc/default/vxdisk file. Use the following file format: # more /etc/default/vxdisk format=sliced privlen=2048

4-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Group Operations

VxVM Disk Group Operations Basic VxVM administrative functions are those that affect disk drives and disk groups. The VxVM disk operations are performed using a variety of programs including command-line and graphical. The following list briefly summarizes each type of disk-related operation. ●

Placing disk drives under VxVM control

●

Removing disk drives from VxVM control

●

Adding a disk drive to a disk group

●

Removing disk drives from a disk group

●

Renaming disk drives

●

Creating disk groups

●

Destroying disk groups

●

Importing disk groups

●

Deporting disk groups

The most commonly used tools for basic operations include: ●

The vxdisksetup and vxdiskunsetup commands

●

The vxdg command

●

The vxedit command

●

The vxdiskadm utility

●

The VEA GUI

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-7

Verifying Disk Group Status

Verifying Disk Group Status The most commonly used disk group status commands are vxdisk and vxdg. Command-line status is usually more efficient than using the VEA GUI.

Using the vxdisk Command to Verify Disk Group Status Use the following vxdisk command to verify the status of the vxdiskadm changes. Disks that show a status of online invalid are not under VxVM control. Disks that show a status of online have been initialized but are not assigned to a disk group. When online disks are added to a disk group, they are assigned a name which appears in the DISK column. By default, the disk name is derived from the name of the disk group. # vxdisk list DEVICE TYPE c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:cdsdisk c2t3d0s2 auto:sliced c2t5d0s2 auto:simple c2t16d0s2 auto:sliced c2t18d0s2 auto:none c2t20d0s2 auto:none c2t22d0s2 auto:none c3t32d0s2 auto:cdsdisk c3t33d0s2 auto:cdsdisk c3t35d0s2 auto:cdsdisk c3t37d0s2 auto:sliced c3t50d0s2 auto:none c3t52d0s2 auto:sliced

DISK a5kdg01 a5kdg02 a5kdg03 newDG01 pdga01 pdga02 pdga03 newDG02 -

GROUP sdga sdga sdga newDG pdga pdga pdga newDG -

STATUS online online online online online online online online online online online online online online online

invalid invalid

invalid invalid invalid

invalid

Using the vxdg Command to Verify Disk Group Status Use the following vxdg command to verify the current status and unique ID of all active disk groups. # vxdg list NAME sdga pdga

4-8

STATE enabled enabled

ID 1064619733.28.ns-east-104 1065123027.40.ns-east-104

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the vxdiskadm Utility

Administering Disk Groups Using the vxdiskadm Utility The vxdiskadm utility is a Bourne shell program that provides menu-based user operations. The following example shows that when the vxdiskadm utility is started, the following menu operations are available: # vxdiskadm Volume Manager Support Operations Menu: VolumeManager/Disk 1 Add or initialize one or more disks 2 Encapsulate one or more disks 3 Remove a disk 4 Remove a disk for replacement 5 Replace a failed or removed disk 6 Mirror volumes on a disk 7 Move volumes from a disk 8 Enable access to (import) a disk group 9 Remove access to (deport) a disk group 10 Enable (online) a disk device 11 Disable (offline) a disk device 12 Mark a disk as a spare for a disk group 13 Turn off the spare flag on a disk 14 Unrelocate subdisks back to a disk 15 Exclude a disk from hot-relocation use 16 Make a disk available for hot-relocation use 17 Prevent multipathing/Suppress devices from VxVM’s view 18 Allow multipathing/Unsuppress devices from VxVM’s view 19 List currently suppressed/non-multipathed devices 20 Change the disk naming scheme 21 Get the newly connected/zoned disks in VxVM view 22 Change/Display the default disk layouts 23 Mark a disk as allocator-reserved for a disk group 24 Turn off the allocator-reserved flag on a disk list List disk information

? ?? q

Display help about menu Display help about the menuing system Exit from menus

Select an operation to perform: 1

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-9

Administering Disk Groups Using the vxdiskadm Utility

Functional Overview The title displayed when the vxdiskadm utility starts up is Volume Manager Support Operations. The vxdiskadm utility performs a wide range of support functions, but also offers assistance in performing a number of common administrative tasks. The basic disk-related vxdiskadm administrative functions presented in this module are: ●

Option 1 Add or initialize one or more disks You use this operation to add one or more disk drives to a disk group. You can add the selected disk drives to an existing disk group or to a new disk group that is created as a part of the operation. The disk drives are assigned a default media name based on the name of the intended disk group. You can override the default disk drive name if needed. The selected disk drives can also be initialized without adding them to a disk group by entering a disk group name of none.

●

Option 2 Encapsulate one or more disks You use this operation to bring disks under VxVM control while preserving existing data. You can also use this operation if you intend to mirror the data for increased reliability or stripe the data for increased performance.

●

Option 3 Remove a disk You use this operation to remove a disk drive from a disk group and return it to a pool for reuse. The disk remains in a VxVM format but its media name is cleared.

●

Option 8 Enable access to a disk group You use this option to place a disk group online (imports) that was previously deported (taken offline).

●

Option 9 Remove access to a disk group You use this option to take a disk group offline (deports). None of the disk group volume structures are available while it is deported.

4-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the vxdiskadm Utility

Creating a New Disk Group Before creating a new disk group, you must know the logical address of at least one disk drive you want to be in the disk group. You should also select media names for any disk drives you intend to add to the new disk group. You use vxdiskadm Option 1, shown in the following example, to create a new disk group and add one or more disk drives. Select an operation to perform: 1 Select disk devices to add: [<pattern-list>,all,list,q,?] c2t5d0 Which disk group [,none,list,q,?] (default: rootdg) newDG Create a new group named newDG? [y,n,q,?] (default: y) y Use a default disk name for the disk? [y,n,q,?] (default: y) y Add disk as a spare disk for newDG? [y,n,q,?] (default: n) y Enter the desired format [cdsdisk,sliced,simple,q,?] (default: sliced) sliced Enter desired private region length [<privlen>,q,?] (default: 2048) 2048 VxVM NOTICE V-5-2-120 Creating a new disk group named newDG containing the disk device c2t16d0 with the name newDG01. VxVM INFO V-5-2-305 Setting spare flag for disk newDG01 in disk group newDG. Add or initialize other disks? [y,n,q,?] (default: n) n

Note – For clarity, many informational messages are omitted from the previous example.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-11

Administering Disk Groups Using the vxdiskadm Utility

Removing a Disk Drive From a Disk Group You use vxdiskadm Option 3 to remove a disk drive from a disk group, as shown in the following example. The disk drive is placed in the free-disk pool and is still under VxVM control. Select an operation to perform: 3 Remove a disk Menu: VolumeManager/Disk/RemoveDisk Enter disk name [,list,q,?] newDG03 VxVM NOTICE V-5-2-284 Requested operation is to remove disk newDG03 from group newDG. Continue with operation? [y,n,q,?] (default: y) y VxVM INFO V-5-2-268 Removal of disk newDG03 is complete. Remove another disk? [y,n,q,?] (default: n) n

4-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Administering Disk Groups Using Command-Line Programs Although there are many VxVM command-line programs, only a few are necessary for the most commonly performed operations. This section describes the basic uses of the following command-line programs: ●

vxdiskunsetup

●

vxdisksetup

●

vxdg

Note – Only selected options for each command are described in this module. When appropriate, other options are described in later modules.

Using the vxdiskunsetup Command Caution – Be careful when using the vxdiskunsetup command. You can destroy existing data on disk drives. During the VxVM software installation and initialization, you might see error messages, such as: VxVM:vxconfigd: Disk device not VxVM:vxconfigd: Disk device not

WARNING: Disk c3t35d0 in group hanfs: found WARNING: Disk c2t18d0 in group hadbms: found

These errors can indicate that there are disk drives that still contain VxVM configuration records from a previous installation. You can clear these disk drives and return them to an uninitialized state by using the vxdiskunsetup command, as shown in the following example: # /etc/vx/bin/vxdiskunsetup -C c2t3d0 The vxdiskunsetup command will not clear a disk drive if the VxVM configuration records indicate it is imported by some other host. The -C option forces the de-partitioning of the disk drive in such a case. The disk drives are returned to standard Solaris OS partitioning.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-13

Administering Disk Groups Using Command-Line Programs

Initializing Disk Drives The vxdisksetup command initializes disk drives for VxVM use, but does not add them to a disk group. The disk drives remain in a free pool for future VxVM use. A typical vxdisksetup command is: # vxdisksetup -i c2t0d0 Without any options, the vxdisksetup command only repartitions the specified disk drive into the VxVM partition format. With the -i option, VxVM configuration records are written in the private region. Note – Usually, the vxdisksetup command is not used directly. It is called by other VxVM programs.

Using the vxdg Command Although the vxdg command has many options, only a few are presented in this module. You have already seen how the vxdiskadm utility creates new disk groups and adds disk drives to a disk group. You can also use the vxdg command to perform those functions. The vxdg command options related to basic disk-group administration are: ●

vxdg init

●

vxdg adddisk

●

vxdg rmdisk

●

vxdg import

●

vxdg deport

●

vxdg destroy

Note – You must initialize a disk drive before it can be added to a new or existing disk group using the vxdg command.

4-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Using the vxdg Command to Create a New Disk Group To create a new disk group using the vxdg command, you must furnish the disk drive logical access name (accessname) of at least one disk drive to be added to the disk group. The VxVM accessname is essentially the logical path to the disk drive in the form: c3t4d0. You should also specify a media name for the disk drive. If you do not specify a media name, it defaults to the accessname. The following shows a typical session to initialize a new disk drive and add it to a new disk group. # vxdisk list DEVICE c0t0d0s2 c0t1d0s2 c2t1d0s2 c2t3d0s2 c2t5d0s2 c2t16d0s2 c2t18d0s2 c2t20d0s2 c2t22d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2 c3t37d0s2 c3t50d0s2 c3t52d0s2

TYPE auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none auto:none

DISK -

GROUP -

STATUS online online online online online online online online online online online online online online online

invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid invalid

# vxdisksetup -i c2t1d0 # vxdisksetup -i c2t3d0 # vxdg init newDG ndg-01=c2t1d0 ndg-02=c2t3d0 # vxdisk -g newDG list DEVICE c2t1d0s2 c2t3d0s2

TYPE auto:sliced auto:sliced

DISK ndg-01 ndg-02

GROUP newDG newDG

STATUS online spare online spare

# vxdg list NAME newDG

STATE enabled

ID 1065465185.43.ns-east-104

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-15

Administering Disk Groups Using Command-Line Programs

Adding and Removing Disk Drives The following shows an example of initializing two disk drives, adding them to an existing disk group, and then removing one of them. # vxdisksetup -i c2t5d0 # vxdisksetup -i c2t16d0 # vxdg -g newDG adddisk ndg-03=c2t5d0 ndg-04=c2t16d0 # vxdisk list DEVICE TYPE c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:sliced c2t3d0s2 auto:sliced c2t5d0s2 auto:sliced c2t16d0s2 auto:sliced c2t18d0s2 auto:none c2t20d0s2 auto:none c2t22d0s2 auto:none c3t32d0s2 auto:none c3t33d0s2 auto:none c3t35d0s2 auto:none c3t37d0s2 auto:none c3t50d0s2 auto:none c3t52d0s2 auto:none

DISK ndg-01 ndg-02 ndg-03 ndg-04 -

GROUP newDG newDG newDG newDG -

STATUS online online online online online online online online online online online online online online online

invalid invalid

invalid invalid invalid invalid invalid invalid invalid invalid invalid

# vxdg -g newDG rmdisk ndg-04 The removed disk drive is still initialized and is available for future use. It is in the free-disk pool and shows a status of auto:sliced and online with no DISK or GROUP entry.

4-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Importing and Deporting Disk Groups At times it is useful to deport a disk group. This function makes the disk group unavailable and invisible to most commands. If the disk group resides on dual-ported storage arrays, a different host can import the disk group. This action is useful if a host system fails. A different host system that is running VxVM can import the disk group and make the data available again to users. The following example shows the process of deportng a disk group. # vxdg list NAME sdga newDG pdga

STATE enabled enabled enabled

ID 1064619733.28.ns-east-104 1065465185.43.ns-east-104 1065123027.40.ns-east-104

# vxdg deport newDG # vxdg list NAME sdga pdga

STATE enabled enabled

ID 1064619733.28.ns-east-104 1065123027.40.ns-east-104

When a disk group is deported, the host ID stored on all disk drives in the disk group is cleared (unless a new host ID is specified with -h). Therefore, the disk group is not reimported automatically when the system is rebooted. The disk group can be deported with the host ID unchanged or you can change the host ID to another system during the deport operation. Use the vxdg import command option to import a disk group again. For example: # vxdg import newDG

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-17

Administering Disk Groups Using Command-Line Programs

Identifying Deported Disk Groups If you forget the name of a deported disk group, you can use the vxdisk command to identify currently deported disk groups. The following example shows the names of currently deported disk groups enclosed in parenthesis. # vxdisk -o alldgs list DEVICE TYPE c0t0d0s2 auto:none c0t1d0s2 auto:none c2t1d0s2 auto:cdsdisk c2t3d0s2 auto:sliced c2t5d0s2 auto:simple c2t16d0s2 auto:sliced c2t18d0s2 auto:sliced c2t20d0s2 auto:sliced c2t22d0s2 auto:none c3t32d0s2 auto:cdsdisk c3t33d0s2 auto:cdsdisk c3t35d0s2 auto:cdsdisk c3t37d0s2 auto:sliced c3t50d0s2 auto:none c3t52d0s2 auto:sliced

DISK a5kdg01 a5kdg02 a5kdg03 pdga01 pdga02 pdga03 -

GROUP sdga sdga sdga (newDG) pdga pdga pdga (newDG) -

STATUS online online online online online online online online online online online online online online online

invalid invalid nohotuse nohotuse nohotuse

invalid

invalid

You can also use the vxdiskadm utility option 8, Enable access to (import) a disk group, to identify deported disk groups. The following is an excerpt of the resulting output. Select disk group to import [,list,q,?] (default: list) list GROUP newDG (id: 1065465185.43.ns-east-104) DEVICES: c2t16d0 c3t37d0

Note – The VEA interface also displays deported disk groups.

4-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-Line Programs

Destroying a Disk Group Before destroying a disk group, you must be absolutely sure that none of the data on the disk drives is important. A disk group cannot be destroyed if any volumes in the disk group are in use. You can destroy only imported disk groups. The following is an example of using the vxdg command to destroy a disk group. # vxdg list NAME sdga newDG pdga

STATE enabled enabled enabled

ID 1064619733.28.ns-east-104 1065465185.43.ns-east-104 1065123027.40.ns-east-104

# vxdg destroy newDG # vxdg list NAME sdga pdga

STATE enabled enabled

ID 1064619733.28.ns-east-104 1065123027.40.ns-east-104

Renaming VxVM Disk Drives When disk drives are added to a disk group, the disk drives are given standard VxVM media names, such as dga01, dga02, and dga03. You might want to rename the disk drives in a group so that there is no confusion about their intended use. You can use the following vxedit command to rename VxVM disk drives: # vxedit -g sdga rename a5kdg01 sdga01 # vxedit -g sdga rename a5kdg02 sdga02 # vxedit -g sdga rename a5kdg03 sdga03

Note – Volume components such as plexes and subdisks are named according to the VxVM disk media name. If you rename a disk, the plex and subdisk names do not change accordingly. It is best to rename disks before creating volume structures on them.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-19

Administering Disk Groups Using the VEA GUI

Administering Disk Groups Using the VEA GUI The following section describe a series of tasks using the VEA GUI. Only key points and input forms are presented.

Creating a New Disk Group You can use the VEA GUI to create a new disk group. Complete the following steps: 1.

If appropriate, you can pre-select disk drives in the grid area by simultaneously pressing the Control key and mouse button.

2.

Select the New Disk Group entry from the Actions menu.

3.

Complete the New Disk Group Form, as shown in Figure 4-3. Disk names default to variations of the disk group name, such as newDG01 or newDG02. Default disk names are usually acceptable.

Figure 4-3

4-20

New Disk Group Wizard

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI 4.

Click Next to continue. As shown in Figure 4-4, VxVM informs you of potentially dangerous situations it detects. Click on Yes to continue or No to abort.

Figure 4-4 5.

New Disk Group Warning

Complete the Organization window, as shown in Figure 4-5, by selecting a disk group organization principle of None.

Figure 4-5

New Disk Group Organization

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-21

Administering Disk Groups Using the VEA GUI

Adding and Removing Disk Drives To add disks to, or remove disk drives from, a disk group, complete the following steps: 1.

Select the disk group in the object tree, and then click the third mouse button.

2.

Select Add Disk to Disk Group from the pop-up menu, as shown in Figure 4-6.

Figure 4-6

4-22

Disk Groups Disk Add Menu Item

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI 3.

Complete the Add Disk wizard form, as shown in Figure 4-7.

Figure 4-7

Add Disk to Disk Group Wizard

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-23

Administering Disk Groups Using the VEA GUI

Deporting Disk Groups To deport a disk group, complete the following steps: 1.

Select the disk group in the object tree, and then click the third mouse button.

2.

Select Deport from the pop-up menu, as shown in Figure 4-8.

Figure 4-8 3.

Disk Groups Deport Menu Item

Complete the Deport Disk Group form, as shown in Figure 4-9. Usually, you do not modify the disk group name or host system ownership during a deport operation.

Figure 4-9

4-24

Deport Disk Group Form

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI

Importing Disk Groups To import a disk group, complete the following steps: 1.

Display Disk Groups in the grid area, and then select the deported disk group.

2.

Click the third mouse button on the deported disk group and select Import from the pop-up menu, as shown in Figure 4-10.

Figure 4-10 Import Disk Group Menu Item 3.

Complete the Import Disk Group form, as shown in Figure 4-11. Typically, the Force option is necessary only after a system crash.

Figure 4-11 Import Disk Group Form

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-25

Administering Disk Groups Using the VEA GUI

Destroying a Disk Group A disk group must be imported before it can be destroyed. To destroy a disk group, complete the following steps: 1.

Select the disk group in the object tree, and then click the third mouse button.

2.

Select Destroy Disk Group from the pop-up menu, as shown in Figure 4-12.

Figure 4-12 Disk Group Destroy Menu If there are mounted volumes associated with the disk group, the disk group destroy operation fails, as shown in Figure 4-13.

Figure 4-13 Destroy Disk Group Failure

4-26

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using the VEA GUI

Renaming VxVM Disk Drives To rename VxVM disk drives, complete the following steps: 1.

To change the media name of a VxVM disk drive, complete the following steps: a.

Display the disk group disk drives in the grid area.

b.

Select the disk drive to be renamed, as shown in Figure 4-14.

Figure 4-14 Disk Drive Media Name Listing 2.

Click the third mouse button on the highlighted disk, and then select Rename from the pop-up menu.

3.

Complete the Rename Disk form, as shown in Figure 4-15.

Figure 4-15 Rename Disk Form

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-27

Administering Disk Groups Using the VEA GUI

Displaying VEA Object Properties Most VxVM objects have associated properties that detail naming schemes, construction details, and current status. To display the properties of a VxVM object, complete the following steps: 1.

Click on the object and the third mouse button.

2.

Select Properties from the object’s pop-up menu. The Properties of a disk group is shown in Figure 4-16. Object Properties windows can have several tabs depending on the complexity of the object.

Figure 4-16 Disk Group Object Properties Form

4-28

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Exercise: Performing VxVM Basic Operations In this exercise, you complete the following tasks: ●

Review key lecture points

●

Verify the VxVM environment

●

Verify the initial disk drive status

●

Set the default disk drive format

●

Initialize disk drives

●

Create new disk groups

●

View command logs

●

Import and deport disk groups

●

Destroy a disk group

●

Rename disk drives

●

Use the vxdiskadm utility to perform basic operations (optional)

●

Verify ending lab status

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-29

Exercise: Performing VxVM Basic Operations

Preparation If the tasks in this exercise are performed by small groups using disk drives residing on a central VxVM server, each group must take care to not interfere with another group’s storage resources and structures. Ask your instructor to provide two unique code letters for your workgroup (A and B, C and D, E and F, and so on). Workgroup code letters:

dg ___

dg ___

Copy the logical paths to six disk drives for your work group from the information in Module 2, ‘‘Managing Data” in ‘‘Task 9 – Selecting Disk Drives for Use’’ on page 2-32. Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Disk: _______________

Assign each workgroup’s two code letters (A/B, C/D, E/F) so they can create two unique disk group names, such as dgA and dgB, or dgE and dgF. A workgroup consists of two or more students working with six disk drives from one keyboard on an administration workstation. An antiquated, but useful, management tool is to have each workgroup write the logical paths of their assigned disk drives on a 3 by 5 card and tape the card to their display monitor.

Many of the tasks are performed twice. The first time using the VEA GUI and the second time using command-line programs. For most tasks, you must destroy the structures before creating them again. Destroying and deleting structures is part of regular VxVM administrative tasks.

4-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 1 – Reviewing Key Lecture Points Answer the following questions about the major areas of concern during the VxVM installation and initialization process. 1.

What is the primary use for a shared disk group? a.

Support for load balancing

b.

Support for multi-host storage access

c.

Support for parallel databases

d.

Support for multipath storage access

The answer is c.

2.

Which two answers best describe the VxVM disk-drive initialization process? a.

Preserves existing file systems

b.

Places the disk drive in a free disk pool

c.

Adds the disk drive to a disk group

d.

Destroys existing file systems

The answers are b and d.

3.

Which of the following commands is not used to perform disk group administration? a.

vxdisk

b.

vxassist

c.

vxdg

d.

vxdiskadm

e.

vxdisksetup

The answer is b.

4.

Which of the following commands is used to rename disk drives? a.

vxdg

b.

vxassist

c.

vxdisk

d.

vxedit

The answer is d.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-31

Exercise: Performing VxVM Basic Operations

Task 2 – Verifying the VxVM Environment Complete the following steps to verify the environment on the VxVM administration system and the VxVM server: 1.

Log in to the VxVM server as user root and enter the env shell command.

2.

Verify that the following environment exists on the server: PATH=/usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bi n:/opt/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/man

4-32

3.

If you are working from a remote administration system, log out of the VxVM server.

4.

On the remote administration system, complete the following steps: a.

Type the env shell command.

b.

Verify that the /opt/VRTS/bin and /opt/VRTSob/bin directories are part of the PATH variable.

c.

Verify that the /opt/VRTS/man directory is part of the MANPATH variable.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 3 – Verifying the Initial Disk Drive Status To verify the initial status of your assigned disk drives, complete the following steps: 1.

Log in as user root to the VxVM server.

2.

Verify the status of your assigned disk drives. # vxdisk list

3.

Check the output carefully and verify that each disk drive assigned to your workgroup shows a status of online invalid. If any of your assigned disk drives show a status other than online invalid, check with your instructor. If appropriate, use the vxdiskunsetup command, as shown in the following example, to return all of your assigned disk drives to an online invalid status. # /etc/vx/bin/vxdiskunsetup -C c0t22d0

Note – You must substitute the VxVM accessname of your disk drives. Do not proceed until all your assigned disk drives are uninitialized.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-33

Exercise: Performing VxVM Basic Operations

Task 4 – Setting the Default Disk Drive Format Set the default VxVM disk initialization format on the VxVM server by completing the following steps: 1.

Log in to the VxVM server as user root.

2.

Complete the following steps: a.

Start the vxdiskadm utility.

b.

Select option 22, Change/Display the Default Disk Layouts.

# vxdiskadm 3.

Select option 1 from the Volume Manager Preferences menu.

4.

Select option 1 from the Disk Initialization Preferences menu and type sliced for the desired format.

5.

Select option 2 from the Disk Initialization Preferences menu and type 2048 for the desired private region length.

6.

Type q or quit at least twice to exit the vxdiskadm utility.

7.

Verify the default disk initialization format values are set correctly. # more /etc/default/vxdisk format=sliced privlen=2048

Caution – Values entered in the /etc/default/vxdisk file are honored only by command-line programs such as the vxdiskadm and vxassist utilities. The VEA GUI does not reference the /etc/default/vxdisk file.

4-34

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Task 5 – Initializing Disk Drives Perform the following procedures to initialize your assigned disk drives for use by VxVM. There are two procedures: one for using the VEA GUI and one for using command-line programs.

Using the VEA GUI to Initialize Disk Drives To initialize three of your assigned disk drives using the VEA GUI, complete the following steps: 1.

Expand the VxVM server node in the object tree and click Disks.

2.

Select three of your assigned disk drives in the grid area by simultaneously pressing the Control key while using the mouse button.

3.

Click the third mouse button on one of your highlighted disk drives, and select Initialize Disk in the pop-up menu.

4.

In the Initialize Disk form, click Yes To All. After a short delay, three of your assigned disk drives should appear in the grid area with a status of Free.

5.

Complete the following steps: a.

Use the vxdiskunsetup command to return each of your assigned disk drives to an uninitialized state.

b.

Replace the accessname variable in the following command with the address of your disk drives (for example, c4t3d0).

# vxdiskunsetup -C accessname 6.

Verify that the status of your assigned disk drives is once again online invalid. # vxdisk list

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-35

Exercise: Performing VxVM Basic Operations

Using Command-Line Programs to Initialize Disk Drives To initialize your remaining three disk drives using the vxdisksetup command-line program, complete the following steps: 1.

Use the vxdisksetup command to initialize each of your assigned disk drives, replacing the accessname with the address of your disk drives (for example, c4t3d0). # vxdisksetup -i accessname

2.

Verify the status of your assigned disk drives is online. # vxdisk list

Task 6 – Creating New Disk Groups In this section, you create a disk group containing three of your assigned disk drives. Name the disk group according to your first assigned workgroup letter. For example, if your assigned work group letters are A and B, then this first disk group should be named dgA.

Using the VEA GUI to Create a New Disk Group To create a new disk group using the VEA GUI, complete the following steps: 1.

Click New Group in the Toolbar. The initial New Disk Group Wizard form appears.

2.

Click the Do not show this page next time box, and then click Next. The disk selection form appears.

3.

Complete the disk selection form as follows: a.

Type the name of your new disk group.

b.

In the Available disk column, select three of your assigned disk drives, and then click Add.

c.

Do not enter disk names.

d.

Click Next. The organization principle form appears.

4-36

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations 4.

Complete the organization principle form as follows: a.

Select the None organization principle.

b.

Click Next. The summary form appears.

5.

Click Finish on the summary form to create the new disk group.

6.

Verify that your new disk group appears in the VEA object tree.

Using the vxdg Command to Create a Disk Group Use the vxdg command to create a second disk group that contains your three remaining assigned disk drives. Name the disk group according to your second workgroup letter. For example, if your work group letters are A and B, then this second disk group should be named dgB. Complete the following steps: 1.

Use the vxdg init command to create a second new disk group and add one of your remaining assigned disk drives. # vxdg init dgname medianame=accessname For example: vxdg init dgB dgB-01=c2t3d0

2.

Add the two remaining assigned disk drives to the new disk drive group. # vxdg -g dgname adddisk medianame=accessname For example: vxdg -g dgB adddisk dgB-02=c2t5d0 dgb-03=c2t6d0

3.

Use the vxprint command to verify the status of your new disk group.

4.

Use the vxdg rmdisk command to remove one of the disk drives from your new disk group. # vxdg -g dgname rmdisk medianame For example: vxdg -g dgB rmdisk dgB-03

5.

Use the vxdg adddisk command to add the same disk drive back into your disk group.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-37

Exercise: Performing VxVM Basic Operations

Task 7 – Viewing Command Logs The command log records the command-line programs used to accomplish VEA tasks. It is a good learning tool. To view the command log file, complete the following steps: 1.

Log in to the VxVM server as user root.

2.

View the command log file. # cd /var/vx/isis # ls alertlog state command.log swap savedqueries tasklog # more ./command.log

vea_portal vxisis.lock vxisis.log

Task 8 – Importing and Deporting Disk Groups In this task, you deport and import one or both of your disk groups.

Using the VEA GUI to Import and Deport Disk Groups To import and deport disk groups using the VEA GUI, complete the following steps: 1.

Display disk groups in the grid area.

2.

Click the name of one of your disk groups and select Deport Disk Group from its pop-up menu. The Deport Disk Group form appears.

Note – You can rename a disk group during a deport operation. You can also assign ownership to a different host. You might do this if you needed to take the current host down for maintenance and wanted a different host system to manage the disk group for a while. You might also rename the disk group if the second host already had a disk group with the same name. 3.

4-38

Click OK.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations 4.

Complete the following steps: a.

Click Browse and highlight the disk group you want to deport.

b.

Click OK. The status of your disk group should change to Deported in the grid area display.

5.

Click your deported disk group in the grid area and select Import from its pop-up menu.

Using Command-Line Programs to Import and Deport Disk Groups To import and deport disk groups using command-line programs, complete the following steps: 1.

Log in to the VxVM server and use the vxdg deport command to deport either of your disk groups. # vxdg deport dgname For example: vxdg deport dgA.

2.

Use the vxdisk command to identify the deported disk group. # vxdisk -o alldgs list

3.

Use the vxdg import command to import your disk group again. # vxdg import dgname For example: vxdg import dgA.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-39

Exercise: Performing VxVM Basic Operations

Task 9 – Destroying a Disk Group In this task, you destroy one of your disk groups.

Using the VEA GUI to Destroy a Disk Group To destroy a disk group using the VEA GUI, complete the following steps: 1.

Display Disk Groups in the grid area.

2.

Click one of your disk groups and select Destroy Disk Group from its pop-up menu.

3.

Click New Group in the Toolbar and recreate the disk group you destroyed.

Using Command-Line Programs to Destroy a Disk Group To destroy one of your disk groups using command line programs, complete the following steps: 1.

Use the vxdg command to destroy one of your disk groups. # vxdg destroy dgname For example: vxdg destroy dgA.

2.

Recreate the destroyed disk group using the tool of your choice.

Task 10 – Renaming Disk Drives In this task, you rename one or more of your assigned disk drives.

Using the VEA GUI to Rename Disk Drives To rename disk drives using the VEA GUI, complete the following steps:

4-40

1.

Display the disk drives from one of your disk groups in the grid area.

2.

Click one of the disk drives and select Rename Disk from its pop-up menu.

3.

Enter a unique media name for your disk drive.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing VxVM Basic Operations

Using Command-Line Programs to Rename Disk Drives To rename disk drives using command-line programs, complete the following step: Using the vxedit command, restore the original media name of the disk drive you modified in ‘‘Using the VEA GUI to Rename Disk Drives’’ on page 4-40. # vxedit rename oldname newname For example: vxedit rename zx12 dgB-03.

Task 11 – Using the vxdiskadm Utility to Perform Basic Operations (Optional) Ask your instructor if there is enough time to perform this optional task. To recreate your disk groups using the vxdiskadm utility features, complete the following steps: 1.

Use the vxdg command to destroy both of your disk groups. # vxdg destroy dgname

2.

Use the vxdiskadm utility option 1 to recreate both of your assigned disk groups.

3.

Use the vxdiskadm utility option 3 to remove a disk drive from each of your disk groups.

4.

Use the vxdiskadm utility option 1 to add the previously removed disk drives into your disk groups.

5.

Use the vxdiskadm utility option 9 to deport one or both of your disk groups.

6.

Use the vxdiskadm utility option 8 to import one or both of your assigned disk groups.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-41

Exercise: Performing VxVM Basic Operations

Task 12 – Verifying Ending Lab Status To verify your system configuration, complete the following steps: 1.

Use the vxdisk list command to ensure that both of your disk groups are still complete and meet the following guidelines: ●

You have two disk groups name dgX and dgY.

●

Each disk group contains three disk drives.

●

None of your assigned disk drives are designated as hot spares.

Note – Substitute your workgroup codes for the X and Y in dgX and dgY. 2.

4-42

Use the vxprint command to verify that there are no subdisk structures in either of your disk groups.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

4-43

Module 5

VERITAS Volume Manager Volume Operations Objectives Upon completion of this module, you should be able to: ●

Interpret volume structure listings

●

Describe volume planning activities

●

Create volumes using the VEA GUI

●

Create volumes using the vxassist command

●

Modify volume access attributes

●

Add file systems to existing volumes

●

Add and remove volume logs

●

Use the VEA GUI to analyze volume structures

5-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Interpreting Volume Structure Listings VERITAS VxVM volume structures are composed of three primary components: subdisks, plexes, and a volume name. The components are assigned names based on default conventions or can be manually chosen.

Subdisks A subdisk is a set of contiguous disk blocks. A subdisk must reside entirely on a single physical disk drive. The public region of a disk drive in a disk group can be divided into one or more subdisks. The subdisks cannot overlap or share the same portions of a public region. The smallest possible subdisk is a single sector (512 bytes), and the largest subdisk is the entire VxVM public region. By default, subdisks are named based on the VxVM media name of the disk drive on which they reside. This relationship is shown in Figure 5-1. Disk Group (DGa) VxVM Disk Physical Disk

Subdisks

disk01-01

c3t12d0 disk01-02 disk01 VxVM Disk disk02-01 Physical Disk c4t33d0

disk02-02 disk02-03 disk02

Figure 5-1

Subdisk Naming Conventions

Ask why spanning storage arrays might be a good idea. One answer is striping for performance or mirroring for availability.

5-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Plexes The VxVM application uses subdisks to build virtual objects called plexes. A plex consists of one or more subdisks located on one or more physical disk drives. Figure 5-2 shows the relationship of subdisks to plexes in a disk group named DGa. Disk Group (DGa)

Physical Disk

VxVM Disk

Plex

disk01-01

disk01-01

disk01-02

disk01-02

disk01

vol01-01

VxVM Disk

Plex

disk02-01

disk02-01

disk02-02

disk02-02

c3t12d0

Physical Disk c4t33d0

disk02-03

vol01-02

disk02

Figure 5-2

Plex Configurations

The data to be stored on the subdisks of a plex can be organized by using any of the following methods: ●

Concatenation

●

Striping

●

Striping with parity (RAID 5)

Note – A plex can contain a maximum of 4096 subdisks.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-3

Interpreting Volume Structure Listings

Volumes A volume consists of one or more plexes. By definition, a volume with two plexes is mirrored. Figure 5-3 shows the relationship of plexes in a mirrored volume in a disk group named DGa. Disk Group (DGa) Volume

Physical Disk

VxVM Disk

Plex

disk01-01

disk01-01

disk01-02

disk01-02

disk01

vol01-01

VxVM Disk

Plex

disk02-01

disk02-01

disk02-02

disk02-02

c3t12d0

Physical Disk c4t33d0

disk02-03

vol01-02

disk02

Figure 5-3

vol01

Mirrored Volume Structure

You should understand the following important points about volumes: ●

Volumes can have more than two mirrors.

Three mirrors is usually as many as most customers ever have in critical applications.

5-4

●

RAID-5 volumes cannot be mirrored.

●

A plex can also be a logging structure that is not used for data storage.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Interpreting Volume Structure Listings

Volume Structure Examples The following vxprint output shows an example of two volumes that belong to the same disk group. The vol_01 volume is a simple concatenation and the vol_02 volume is mirrored across two controllers. The listing indicates the following details: ●

Both volumes are 2 Gbytes in size (4194304 blocks/sectors)

●

vol_01 is composed of a single plex and associated subdisk

●

vol_02 is composed of two plexes each with an associated subdisk

# vxprint -g newDG TY NAME ASSOC dg newDG newDG

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm dm dm dm dm dm

newDG01 newDG02 newDG03 newDG04 newDG05 newDG06

c2t16d0s2 c2t18d0s2 c2t20d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2

-

17679776 17679776 17679776 17679776 17679776 17679776

-

-

v vol_01 pl vol_01-01 sd newDG01-01

fsgen vol_01 vol_01-01

ENABLED ENABLED ENABLED

4194304 4194304 4194304

0

ACTIVE ACTIVE -

v pl sd pl sd

fsgen vol_02 vol_02-01 vol_02 vol_02-02

ENABLED ENABLED ENABLED ENABLED ENABLED

4194304 4194304 4194304 4194304 4194304

0 0

SYNC ACTIVE ACTIVE -

vol_02 vol_02-01 newDG04-01 vol_02-02 newDG01-02

The following vxdg output shows the amount of available disk space. The OFFSET column represents the amount of space currently used and the LENGTH column represents the amount of free space. # vxdg -g newDG free DISK DEVICE newDG01 c2t16d0s2 newDG02 c2t18d0s2 newDG03 c2t20d0s2 newDG04 c3t32d0s2 newDG05 c3t33d0s2 newDG06 c3t35d0s2

TAG c2t16d0 c2t18d0 c2t20d0 c3t32d0 c3t33d0 c3t35d0

OFFSET 8388608 0 0 4194304 0 0

LENGTH 9291168 17679776 17679776 13485472 17679776 17679776

FLAGS -

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-5

Volume Planning

Volume Planning Creating volume structures is easy to do. It is also easy to make mistakes unless you understand each aspect of the volume creation process.

Volume Distribution A common mistake is to place all the disk drives in a single disk group. The configuration records for a disk group cannot contain information for more than 4096 objects. Each volume, plex, subdisk, and disk drive is considered to be an object and requires 256 bytes of space in the private region. The default private region length is 2048 blocks. Another reason for organizing disk drives into separate disk groups is that you might want to deport a disk group and import it to another connected host system. This action can be part of a disaster recovery plan or a load balancing measure. You can design a disk group so that it is better for particular tasks. Each disk group shown in Figure 5-4 has three disk drives, and each disk drive is in a different storage array. Host System

c0

c2

DGa

d1

d2

d3

DGb

d4

d5

d6

DGc

d7

d8

d9

Array

Array

Array

Figure 5-4

5-6

c1

Striped Volume Disk Groups

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning Disk groups organized in this manner are good for creating striped volume types (such as RAID 5) and for mirrored volumes. The most important feature is that each disk drive in the disk group is in a separate enclosure and on a different controller. Note – Exercise care with disk groups that span storage arrays. You must be sure that the loss of an entire array does not disrupt both mirrors in a volume or more than one column in a RAID-5 volume. Another disk group structure, such as the one shown in Figure 5-5, would be better used with simple concatenated volumes. Host System

c0

c1

c2

DGa

d1

d2

d3

DGb

d4

d5

d6

DGc

d7

d8

d9

Array

Array

Array

Figure 5-5

Concatenated Volume Disk Groups

If the volumes are large, static, read-only structures that need only a periodic backup to tape, they do not need any higher level of reliability or availability.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-7

Volume Planning

Volume Naming Conventions Within a disk group, the VxVM software requires a unique name for each new volume that is created. The volume names are usually systematic, such as vol01, vol02, vol03, and so on. Naming conventions can be used to reflect volume attributes, such as: ●

The volume structure

●

Which department uses them

●

With which database they are associated

●

Special purposes within a workgroup

Although naming conventions might not seem to be of much importance, they can help establish priorities during emergency situations, such as major power outages.

Space Allocation Planning Although choosing a general size for a volume is frequently dictated by the application, administrators frequently want to use as much space as is practical on a set of disk drives. There are several ways to allocate space for a volume. Among them are: ●

Let the VxVM software automatically find the space.

●

Limit the search for space to selected disk drives in a group.

●

Research available space with command-line programs.

Note – The VEA GUI New Volume Wizard also has limited space research capabilities during new volume creation.

Automatic Space Allocation If you do not specify disk resources when creating volumes, the VxVM software automatically finds portions of unused disk space and assembles them into a volume. This action can lead to a disorganized structure and create poor performance for some volume types.

5-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning

Restricted Space Allocation Rather than letting VxVM find space anywhere within a disk group, it is frequently better to direct VxVM to use a particular disk drive for a new volume. The disk group shown in Figure 5-6 can be used in several different ways depending on the type of volume structures you require: ●

A RAID-5 volume might use disk drives d1, d2, and d3

●

A concatenated volume might use disk drives d1, d4, and d7.

●

A mirrored and concatenated volume might use disk drives d1, d4, and d7 for one mirror and disk drives d3, d6, and d9 for a second mirror. Host System c0

DGa

Figure 5-6

c1

c2

d1

d2

d3

d4

d5

d6

d7

d8

d9

Array

Array

Array

Selecting Disk Drives for a Volume

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-9

Volume Planning

Researched Space Allocation Analyzing free disk space before creating a volume is sometimes beneficial. Look for patterns of free space that fit your needs. The following examples show how to research free space in a disk group. The vxdg command gathers a rough estimate of available disk space. The following is an example of using vxdg on a 9-Gbyte disk drive. # vxdg -g newDG free DISK DEVICE ndg-01 c2t1d0s2 ndg-02 c2t3d0s2 ndg-03 c3t32d0s2 ndg-04 c3t33d0s2

TAG c2t1d0 c2t3d0 c3t32d0 c3t33d0

OFFSET 0 0 0 0

LENGTH 17674902 17674902 17674902 17674902

FLAGS -

After creating a 6-Gbyte mirrored volume using the ndg-01 and ndg-03 disks, the following disk space is available: # vxdg -g newDG free DISK DEVICE ndg-01 c2t1d0s2 ndg-02 c2t3d0s2 ndg-03 c3t32d0s2 ndg-04 c3t33d0s2

TAG c2t1d0 c2t3d0 c3t32d0 c3t33d0

OFFSET 12586455 0 12586455 0

LENGTH 5088447 17674902 5088447 17674902

FLAGS -

After rough planning, you use the vxassist maxsize command to research more detailed information. The following examples show how to use the vxassist command: # vxassist -g newDG maxsize \ layout=nomirror,nostripe ndg-01 ndg-02 ndg-03 ndg-04 Maximum volume size: 45524992 (22229Mb) This is a basic concatenation that uses almost all of the available space, that is, 22 Gbytes.

# vxassist -g newDG maxsize \ layout=raid5,nolog ndg-01 ndg-02 ndg-03 ndg-04 Maximum volume size: 20353024 (9938Mb) RAID-5 volume column size is limited by the size of the smallest available column, which is 5,088,447 blocks in these examples. Additionally, approximately one disk drive’s worth of space is lost to parity.

The vxassist maxsize command is not usually needed unless you have especially limited disk drive space and need to maximize its use. It is a good practice to leave a small amount of space for log placement.

5-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Planning

Selecting Volume Types The VxVM application supports several types of volumes. Each volume type has a specific layout specifier. The following sections briefly describe each volume type and its associated layout specification.

Concatenated The advantages of concatenated volumes are efficient use of storage space and easier hardware requirements. Layout specification: concat

Striped The advantages of striped volumes are improved read and write performance. Layout specification: stripe

Mirrored Concatenation The advantage of a mirrored concatenation volume is improved availability through data redundancy. Layout specification: mirror-concat

Mirrored Stripe The advantages of a mirrored stripe volume are improved availability and increased read and write performance. Layout specification: mirror-stripe

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-11

Volume Planning

Concatenated Mirror (Layered Volume Type) The advantages of concatenated mirror volumes are improved availability and faster recovery time. This is because only a portion of a mirror must be recovered. Layout specification: concat-mirror

Striped Mirror (Layered Volume Type) The advantages of striped mirror volumes are improved performance, faster recovery times, and high disk drive failure tolerance. Layout specification: stripe-mirror

RAID 5 The advantage of RAID-5 volumes are somewhat improved availability with limited negative impact on disk space utilization. Layout specification: raid5 (or raid5nolog)

5-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI

Creating Volumes Using the VEA GUI A New Volume Wizard is initiated in the VEA GUI using either of the following methods: ●

The menu bar’s Actions menu New Volume item

●

The toolbar’s New Volume button

Disk Selection Method The initial New Volume Wizard form prompts you to select the disk selection method to be used. The default is to let VxVM decide which disks to use. However, when creating complex volumes, it is a best practice to enable manual disk selection, as shown in Figure 5-7.

Figure 5-7

New Volume Wizard Disk Selection Method

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-13

Creating Volumes Using the VEA GUI

Using the Disk Selection Form As shown in Figure 5-8, the New Volume Wizard disk selection form assists you in locating and selecting suitable disk drives.

Figure 5-8

New Volume Wizard Disk Selection

The Mirror Across and Stripe Across check boxes let you choose how you want stripes or mirrors distributed across your storage configuration. The Mirror Across Tray applies to specific storage array models that have separate disk drive trays in a single array. Unless you later specify a striped or mirrored volume structure, these features do not perform any useful function. The Ordered check box is an advanced function that uses the specified storage first to create concatenation, then form columns, and finally to create mirrors. Ordered allocation is an advanced subject presented later in this course.

5-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI

Using the Volume Attributes Form The New Volume Wizard volume attributes form, shown in Figure 5-9, assists you in defining the name, size, and type of a volume structure.

Figure 5-9

New Volume Wizard Attributes

Consider the following points when configuring new volume attributes: ●

Name volumes according to their purpose.

●

Use Maxsize after you have configured everything else. The Maxsize function calculates space based on the volume layout specification. The space available varies widely depending on the target volume configuration.

●

Use the Columns and Stripe unit size functions when you are creating volumes that have columns such as striped or RAID-5 volumes.

●

Note that the Mirrored check box is automatically enabled when Concatenated Mirrored or Striped Mirrored structures are selected. Manually selecting Concatenated, and then selecting Mirrored, creates a different structure than selecting Concatenated Mirrored.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-15

Creating Volumes Using the VEA GUI

Using the Create File System Form As shown in Figure 5-10, the New Volume Wizard File System form prompts for file system type (UNIX file system [UFS] or VxFS) and mount options. VEA automatically creates the mount point, modifies the /etc/vfstab file, and initializes and mounts the new file system. The New File System Details button enables newfs and mkfs option entry. The Mount File System Details button allows volume ownership and protection entry.

Figure 5-10 New Volume Wizard File System Form

Note – The VEA server logs the commands that perform all functions in the /var/vx/isis/command.log file. The log file is a useful learning tool.

5-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the vxassist Command

Creating Volumes Using the vxassist Command The vxassist program performs a wide range of volume-related tasks without the complexity of lower-level programs, such as vxvol. The vxassist program also protects against many mistakes that can be made when using lower-level command-line programs.

The vxassist Command The vxassist command has many options. Most of the options use default values if values are not explicitly entered. The most basic form of the vxassist command, which creates a volume, is: # vxassist make vol02 50m The problem with this format is that is assumes the following: ●

A default disk group, if one is configured

●

The default volume type is a basic concatenation

●

Any disk drives that have available space might be used

If you do not specify options, the vxassist command probably cannot give you what you need. It can also create a poorly performing volume or contribute to poor performance of other volumes.

Specifying Volume Size Volume sizes can be specified using the following units: ●

b (blocks)

●

g (gigabytes)

●

k (kilobytes)

●

m (megabytes)

●

s (sectors)

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-17

Creating Volumes Using the vxassist Command

Using vxassist Command Options If you furnish even a few options with the vxassist command, the outcome is more clearly defined. A typical command using limited options is: # vxassist -g dg2 make newvol 2g \ layout=raid5,nolog disk01 disk02 disk03 This form of the vxassist command is more explicit and guarantees that the following are true: ●

The disk group that is used is dg2.

●

The name of the volume is newvol.

●

The size of the volume is 2 Gbytes.

●

This is a RAID-5 volume without a log and with three columns.

●

All disk space comes from disk01, disk02, and disk03.

Although there are many vxassist command options, only a few are commonly used. Some of them require careful study. Always read the manual (man) pages and related documentation before attempting to use most of these options. The vxintro and vxassist man pages contain useful information that is difficult to find elsewhere.

Default Option Values Pay close attention to the default values of command options. When specifying volume layouts, ensure that you understand the default value or configuration for a particular volume type. The following examples show some layout specifications for RAID-5 volume types: layout=raid5 (creates a RAID-5 log by default) layout=raid5log layout=raid5,nolog layout=noraid5log The first two variations are equivalent and create the same volume structure. The last two are also equivalent.

5-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the vxassist Command

Examples of vxassist Command Options The following examples show each type of supported volume structure. The layered volumes require a larger minimum number of disk drives to implement. # vxassist -g dg3 make newvol 20m \ layout=concat disk01 # vxassist -g dg3 make newvol 20m \ layout=stripe disk01 disk02 # vxassist -g dg3 make newvol 20m \ layout=mirror-concat disk01 disk02 # vxassist -g dg3 make newvol 20m \ layout=mirror-strip disk01 disk02 disk03 disk04 # vxassist -g dg3 make newvol 20m \ layout=concat-mirror disk01 disk02 disk03 disk04 # vxassist -g dg3 make newvol 20m \ layout=stripe-mirror disk01 disk02 disk03 disk04 # vxassist -g dg3 make newvol 20m \ layout=raid5,nolog disk01 disk02 disk03 The vxassist command can frequently determine the best way to use the specified disk drives (media names) in a volume structure.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-19

Modifying Volume Access Attributes

Modifying Volume Access Attributes When new volumes are created in a disk group, they are given a set of default access attributes that include: ●

Owner

●

Group

●

Mode

The owner, group, and mode are usually those of the root user. For some volumes, especially raw volumes that are used by a database, the volume ownership must be modified. Caution – Do not use the chown, chgrp, or chmod command to set raw volume attributes. This is because the attributes revert to their original values after each system reboot. Change raw volume attributes using VxVM commands.

Verifying Volume Ownership The ownership and permissions of raw volumes can be checked like ordinary system files. You use the ls command to examine the raw volume files in the /dev/vx/rdsk/dg_name directories. # ls -l /dev/vx/rdsk/newDG/testvol crw------1 root root 199,73000 Feb3 20:37 /dev/vx/rdsk/newDG/testvol

Modifying Volume Ownership and Permissions To run applications, such as ORACLE 9i RAC, it might be necessary to change read/write permissions and ownership of the raw volumes. You use the vxedit command to change the raw volume’s permissions or ownership. # vxedit -g tpcs set user=oracle group=dba mode=660 acct06

Note – You can also use the VEA GUI New Volume Wizard to set volume ownership and permissions during initial volume creation. Or, later, you can use the Actions menu File System entry to create a file system and configure mount information for an existing volume.

5-20

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Adding a UFS File System to Existing Volumes

Adding a UFS File System to Existing Volumes Adding a UFS file system to a volume is easy to do. You can fill out a form using the VEA GUI, or you can create a file system from the command line using standard Solaris OS commands. Both methods are described in this section. Note – VxVM also supports the VxFS file system type, but the VxFS features are licensed separately.

Using the VEA GUI to Add a File System To add a new file system to an existing volume using the VEA GUI, highlight the target volume in the grid area and select File System New File System from the Actions menu. The resulting Create File System Form is identical to the New Volume Wizard Create File System Form.

New File System Details Form Using the New File System Details Form, shown in Figure 5-11, you can make fundamental file system changes.

Figure 5-11 New File System Details Form You enter a comma-separated list of mkfs file system options in the Extra options text field. Consult the mkfs and mkfs_ufs man pages for more detailed option information.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-21

Adding a UFS File System to Existing Volumes

Mount Details Form You can enter valid file system mount options, as shown in Figure 5-12.

Figure 5-12 Mount Details Form

5-22

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Adding a UFS File System to Existing Volumes

Adding a File System From the Command Line When a new file system is initialized from the command line, you can adjust certain file system parameters to make more efficient use of available space. They are: ●

Minimum file system free space

●

Number of bytes per inode

File System Free Space By default, the newfs utility calculates the minimum free space based on partition size (64 Mbytes ÷ partition size × 100), rounded down to the nearest integer. The default value is limited to between 1 percent and 10 percent. When the file system is full, the free space can only be accessed by user root. It can act as an emergency overflow. # newfs -m 10 /dev/vx/rdsk/newDG/vol_01 In very large file systems, you can safely set the minimum free space to a smaller percentage.

Number of Bytes per Inode The newfs utility calculates the number of bytes per inode based on file system size. By default, the newfs utility calculates the number of inodes as follows: ●

2048 bytes per inode for 0-1 Gbyte file system size

●

4096 bytes per inode for 1-2 Gbytes file system size

●

6144 bytes per inode for 2-3 Gbytes file system size

●

8192 bytes per inode for file system larger than 3 Gbytes

If you intend to create a large file system that will contain a small number of very large files, you might be able to decrease the total number of inodes, for example: # newfs -i 10240 /dev/vx/rdsk/newDG/vol01

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-23

Adding a UFS File System to Existing Volumes

Enabling the Solaris OS UFS Logging Feature UFS logging is a standard feature of the Solaris 8 OS. If the logging option is specified for a file system, then logging is enabled for the duration of the mounted file system. Logging is the process of storing transactions (changes that make up a complete UFS operation) in a log before the transactions are applied to the file system. After a transaction is stored, the transaction can be applied to the file system. This process prevents file systems from becoming inconsistent, therefore eliminating the need to run the fsck command. Because the fsck command can be bypassed, logging reduces the time required to reboot a system if it crashes or after an unclean halt. The default behavior is no logging. The log is allocated from free blocks on the file system, and it is sized at approximately 1 Mbyte per 1 Gbyte of file system, up to a maximum of 64 Mbytes. Logging can be enabled on any UFS, including root (/). The log created by UFS logging is continually flushed as it fills up. The log is totally flushed when the file system is unmounted or when the lockfs -f command is used.

Example of File System Commands The following commands summarize the types of operations that must be performed to manually create a UFS logging file system on an existing VxVM volume. # newfs /dev/vx/rdsk/newDG/testvol # mkdir /Test # vi /etc/vfstab /dev/vx/dsk/newDG/testvol ufs 1 yes logging

/dev/vx/rdsk/newDG/testvol /Test

# mount /Test

Note – The default file system behavior is nologging.

5-24

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Administering Volume Logs Both mirrored volumes and RAID-5 volumes can have logs. However, the logs for RAID 5 perform an entirely different function than the mirrored volume log, which is called a DRL.

Using DRLs A DRL is a VxVM log file that tracks data changes made to mirrored volumes. The DRL speeds recovery time when a failed mirror must be synchronized with a surviving mirror. A DRL is a small, special-purpose plex attached to a mirrored volume which has the following features: ●

It is a log that keeps track of the regions within volumes that have changed as a result of write operations to a plex. It does this by maintaining a bitmap and storing this information in a log subdisk.

●

After a system failure, only the regions marked as changed (dirty) in the DRL are recovered.

The following example shows a mirrored volume with a DRL. Notice that the log subdisk does not reside on either of the mirror disk drives. # vxprint -g newDG mirvol TY NAME ASSOC v mirvol fsgen pl mirvol-01 mirvol sd ndg-01-01 mirvol-01 pl mirvol-02 mirvol sd ndg-03-01 mirvol-02 pl mirvol-03 mirvol sd ndg-02-03 mirvol-03

KSTATE ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

LENGTH 4096 7182 7182 7182 7182 LOGONLY 33

PLOFFS 0 0 LOG

STATE ACTIVE ACTIVE ACTIVE ACTIVE -

DRLs are very small compared to the volume data storage.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-25

Administering Volume Logs

Using RAID-5 Logs RAID-5 logs help prevent data corruption in case a system crashes during a write operation. Without logging, if a system fails during a write operation, there is no way to tell if the data and parity were both written to the disk drives. This could result in corrupted data. When RAID-5 logging is used, a copy of the data and parity are written to the RAID-5 log before being written to the disk drives. RAID-5 logging is optional, but RAID-5 logs are created by default. You should always run a system with RAID-5 logs to ensure data integrity. The following example shows a RAID-5 volume with a log. Notice that the log subdisk does not reside on either of the stripe disk drives. # vxprint -g newDG raidvol TY NAME ASSOC v raidvol raid5 pl raidvol-01 raidvol sd ndg-02-02 raidvol-01 sd ndg-01-03 raidvol-01 sd ndg-03-03 raidvol-01 pl raidvol-02 raidvol sd ndg-04-02 raidvol-02

KSTATE ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

LENGTH 4096 7168 3591 3591 3591 3591 3591

PLOFFS 0 0 0 0

STATE ACTIVE ACTIVE LOG -

The size of RAID-5 logs is automatically set by VxVM. It is dependent on the stripe width of the volume. The larger the stripe width (not volume), the larger the RAID-5 log. The RAID-5 log is intended to hold several full-stripe writes simultaneously. The default log size for a RAID-5 volume is four times the full stripe width (the stripe unit size × the number of stripe columns).

5-26

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Planning Log Placement Although logs can be very beneficial to volume recovery after a system crash, if they are not properly managed they can create I/O bottlenecks that negatively impact system performance. You should plan for both RAID-5 logs and DRLs in advance. You should take special care with RAID-5 log placement because the data written to all RAID-5 stripe units is also written to the log. As shown in Figure 5-13, leaving a small amount of free space at the end of all disk drives ensures that you always have alternate locations for log placement or relocation. Volume 01

Vol02_log

Log Space

Log Space

Log Space

Volume 02

Log Space

Log Space

Log Space

Vol01_log

Figure 5-13 Log Space Allocation If possible, a log should not reside on the same disk drive as its related volume. Note – Log sizes are calculated automatically by the VxVM software.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-27

Administering Volume Logs

Adding a Volume Log From the VEA GUI Usually, DRL and RAID-5 logs are added when a new volume is created. However, in some cases it might be preferable to run performance tests before deciding on the placement of logs. To add a DRL or RAID-5 log to an existing volume, click the volume in the grid area. Next, click the Actions menu and select Log Add from its pop-up menu. As shown in Figure 5-14, you can either let VxVM automatically assign a suitable log disk or you can enable manual disk assignment.

Figure 5-14 VEA Add Log Form

5-28

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Adding a Volume Log From the Command Line You can use the vxassist command to add or remove a log from a volume. Both DRL and RAID-5 logs can be added this way. By default, when adding a log, the vxassist command attempts to determine the correct type of log to add and computes an appropriate size for the log. Adding a DRL log to a mirrored volume creates a single log subdisk and a new plex to contain that subdisk. The new plex is then attached to the volume.

Adding a DRL To prevent I/O bottlenecks, a DRL should not reside on a disk drive used by its related volume. It is best to specify the disk drive (media name) where the DRL should be placed. If the DRL location is not specified, the vxassist command assesses the available disk space and decides where to place the log. The following example shows the addition of a DRL to a mirrored volume. # vxassist addlog mirvol ndg-02 # vxprint -g newDG mirvol TY NAME ASSOC KSTATE v mirvol fsgen ENABLED pl mirvol-01 mirvol ENABLED sd ndg-01-01 mirvol-01 ENABLED pl mirvol-02 mirvol ENABLED sd ndg-03-01 mirvol-02 ENABLED pl mirvol-03 mirvol ENABLED sd ndg-02-03 mirvol-03 ENABLED

LENGTH 4096 7182 7182 7182 7182 LOGONLY 33

PLOFFS 0 0 LOG

STATE ACTIVE ACTIVE ACTIVE ACTIVE -

Adding a RAID-5 Log The process for adding a RAID-5 log is the same as for a adding a DRL. The following example shows the addition of a log to a RAID-5 volume. # vxassist addlog raidvol ndg-04 # vxprint -g newDG raidvol TY NAME ASSOC KSTATE v raidvol raid5 ENABLED pl raidvol-01 raidvol ENABLED sd ndg-02-02 raidvol-01 ENABLED sd ndg-01-03 raidvol-01 ENABLED sd ndg-03-03 raidvol-01 ENABLED pl raidvol-02 raidvol ENABLED sd ndg-04-02 raidvol-02 ENABLED

LENGTH 4096 7168 3591 3591 3591 3591 3591

PLOFFS 0 0 0 0

STATE ACTIVE ACTIVE LOG -

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-29

Administering Volume Logs

Removing Volume Logs Using the VEA GUI To remove a volume log using the VEA GUI, click the volume in the grid area. Next, click the Actions menu and select Log Remove from its pop-up menu. Volumes can have more than one log. The Remove Log form, shown in Figure 5-15,allows you to remove logs either by name or by disk drive.

Figure 5-15 VEA Remove Log Form The Quantity/Disk removal method provides two options. You can enter the quantity of logs to be removed, or you can specify one or more disk drives on which to preserve log copies.

5-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Volume Logs

Removing Volume Logs From the Command Line The vxassist command format to remove volume logs is the same regardless of the volume type. The following example shows the removal of logs from both a mirrored volume and a RAID-5 volume. # vxassist remove log mirvol # vxassist remove log raidvol

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-31

Using the VEA GUI to Analyze Volume Structures

Using the VEA GUI to Analyze Volume Structures You use the VEA GUI to visually examine volume structure, disk allocation, and basic performance.

Displaying Volume Layout Details To display volume layout details, click on a volume in the grid area and select Layout View from the pop-up menu. As shown in Figure 5-16, use the right mouse button in the Volume Layout window to select a volume component and examine its properties.

Figure 5-16 Volume Layout Window

5-32

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using the VEA GUI to Analyze Volume Structures

Viewing Disk Volume Mapping and Performance To view volume-to-disk mapping and performance, click on a disk group in the VEA GUI and select Volume Disk from the pop-up menu. The Volume Disk Map, shown in Figure 5-17, displays a map of each volume and its associated disk drives. You can also enable crude performance monitoring by clicking on the View menu and selecting Collect Statistics from the pop-up menu. The disk icons change color to indicate basic levels of I/O activity.

Figure 5-17 Volume Disk Map

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-33

Exercise: Creating Volumes and File Systems

Exercise: Creating Volumes and File Systems In this exercise, you complete the following tasks: ●

Review key lecture points

●

Create a volume

●

Add a volume mirror

●

Add a file system to a volume

●

Add a dirty-region log

●

Resize a volume and file system

●

Create a RAID-5 volume

●

Analyze volumes using the VEA GUI

●

Verify ending lab status

Preparation The purpose of this lab is to have you create and destroy VxVM objects using both the VEA GUI and command-line programs. Each method has advantages. Ask your instructor for the following information: ●

A set of unique volume names for your workgroup: Concat/mirror volume name: __________________________________ RAID-5 volume name: ________________________________________

●

A unique mount point name for your workgroup: Mount point name: ___________________________________________

Your workgroup’s assigned disk drives should be configured as follows: ●

You have two disk groups name dgX and dgY.

●

Each disk group contains three disk drives.

●

None of your assigned disk drives are designated as hot spares.

Note – Substitute your workgroup codes for the X and Y in dgX and dgY.

5-34

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 1 – Reviewing Key Lecture Points Answer the following questions about volume planning, volume access, and volume logging: 1.

Why should you avoid automatic volume naming? a.

Automatic names are limited to 32 variations.

b.

Automatic names cannot be used by some databases.

c.

Automatic names make administration difficult.

d.

Automatic names can be very long.

The answer is c.

2.

Which of the following commands are useful for space allocation research? a.

vxassist maxsize

b.

vxdisksetup -i

c.

vxdisk list

d.

vxdg free

e.

vxdiskadm

The answers are a and d.

3.

Which of the following volume types has the fastest recovery time? a.

Striped volumes

b.

Layered volumes

c.

Mirrored volumes

The answer is b.

4.

Which of the following volume types has the highest disk drive failure tolerance? a.

Concatenated mirror

b.

Mirrored concatenation

c.

RAID 5

d.

Striped mirror

The answer is d.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-35

Exercise: Creating Volumes and File Systems 5.

Which of the following commands modifies volume access attributes? a.

vxdisk define

b.

chmod -R

c.

vxedit set

d.

ln -s

The answer is c.

6.

What is the purpose of dirty region logging? a.

To increase volume write performance

b.

To prevent file system corruption

c.

To speed up volume mirror resynchronization

d.

To control file system access

e.

To prevent internal volume corruption

The answer is c.

7.

What is the purpose of RAID-5 logging? a.

To increase volume write performance

b.

To prevent file system corruption

c.

To speed up volume mirror resynchronization

d.

To control file system access

e.

To prevent internal volume corruption

The answer is e.

5-36

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 2 – Creating a Volume Perform both variations of this task in the order they occur.

Using the VEA GUI to Create a Volume To create a volume using the VEA GUI, complete the following steps: 1.

Display your first disk group (dgX) in the grid area.

2.

In the toolbar, select New Volume.

3.

Configure the New Volume Wizard as follows: ●

Manually select disks to use for the volume.

●

Select only one of the disks for use in the new volume.

●

Enter your assigned concatenated volume name.

●

Select the Concatenated layout.

●

Select Maxsize.

●

After the Maxsize calculation has completed, type 200 in the Size window and select MB from its pull-down menu.

●

Select No file system.

●

Review the final configuration parameters and click Finish.

Note – The Maxsize feature can be useful when you are trying to maximize the size of a new volume and when you have limited disk drive space. 4.

Check the status of the new volume by using the vxprint command.

5.

Verify that your new volume has a single plex with one subdisk and that the volume and plex are ENABLED and ACTIVE.

6.

Display the new volume in the grid area.

7.

Click the new volume in the grid area and select Properties from its pop-up menu.

8.

Examine the volume’s properties and click Cancel when you are finished.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-37

Exercise: Creating Volumes and File Systems

Using the Command Line to Create a Volume To create a volume using the command line, use the man pages, as necessary, to complete the following steps: 1.

Open a window and use the rlogin or telnet command to log into the VxVM server as user root.

2.

Stop the volume you created in the previous procedure. # vxvol -g disk_group stop volume_name

3.

Recursively remove the volume. # vxedit -g disk_group -rf rm volume_name What is the purpose of the vxedit -f option? _____________________________________________________________

4.

5.

Re-create the 200-Mbyte concatenated volume again by using the vxassist command. You must specify the following items: ●

The disk group the volume should be in (-g disk_group)

●

The make option

●

The name and size of the volume (volname 200m)

●

The volume layout (layout=concat)

●

The disk drive (media name) you want to use

Record the command you used to create the volume. _____________________________________________________________

Note – It is not necessary to use the layout= option for a concatenated volume. The default layout is a concatenation. Any other layout requires a specific layout option. For example, layout=mirror or layout=raid5. 6.

Use the vxprint command to verify the volume status is showing ENABLED and ACTIVE.

Note – If there is any problem with your new volume, consult with your instructor.

5-38

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 3 – Adding a Volume Mirror Perform both variations of this task in the order that they occur.

Using the VEA GUI to Add a Mirror To add a mirror to an existing volume, complete the following steps: 1.

Display your new volume in the grid area and click the volume.

2.

Click the Actions menu and select Mirror Add from its pop-up menu, as shown in Figure 5-18.

Figure 5-18 Add Mirror Menu

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-39

Exercise: Creating Volumes and File Systems 3.

Leave the first Add Mirror form configured with its default values which should include the following: ●

Number of mirrors to add: 1

●

Layout=Concatenated

●

Let Volume Manager decide which disks to use

4.

On the VxVM server, check the status of the mirror resynchronization by using the vxtask list command.

5.

On the VxVM server, verify the state of your new mirror by using the vxprint command. You should now see two plexes in your volume. Large mirrors take a while to synchronize. Until the resynchronization is complete, the related plex is in a TEMPRMSD state. Consult the vxinfo man page for volume state definitions.

Note – Remember that you can review the command-line operations in the /var/vx/isis/command.log file on the VxVM server.

Using the Command Line to Add a Volume Mirror To remove a mirror from a volume, and then create a new mirror, complete the following steps: 1.

Ensure that both plexes (mirrors) in the volume you added are fully synchronized and show a status of ENABLED and ACTIVE.

2.

Use the vxassist command to remove one of the mirrors from your volume. For example: # vxassist -g dgX remove mirror vol_01

3.

Use the vxprint command to verify that your volume now has a single plex and subdisk.

4.

Use the vxassist command to re-create the mirrored volume.

5.

Use the alloc= parameter to specify the disk media name on which the new mirror is to be created. For example: # vxassist -g dgX mirror vol_01 alloc=dgX03

5-40

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems 6.

After the vxassist returns, use the vxprint command to verify the volume has two plexes and its status is ENABLED and ACTIVE. Consult with your instructor if you are having problems.

Note – You can also move a volume mirror to a different disk drive if it is poorly placed and is causing a performance problem.

Task 4 – Adding a File System to a Volume Perform both variations of this task in the order they occur.

Using the VEA GUI to Add a File System To add a file system using the VEA GUI, complete the following steps: 1.

Click your mirrored volume in the grid area, and select File System New File System from the pop-up menu.

2.

Configure the New File System form as follows:

3.

●

Ensure that the File system type is ufs.

●

Leave the Allocation at its default value (1024).

●

Enter your assigned mount name.

●

Select Create mount point.

●

Deselect Read only and Honor setuid.

●

Select Add to file system table and mount at boot.

●

Set the fsck pass number to 2.

●

Examine the New file System Details menu

●

Examine the Mount File System Details menu.

●

Click OK.

On the VxVM server, verify that the following are true: ●

The mount point is present in the root directory.

●

Your file system is mounted.

●

The mount entry is in the /etc/vfstab file.

●

The df -kl output seems appropriate for the volume size.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-41

Exercise: Creating Volumes and File Systems 4.

Create some test data in the volume’s file system. # mkfile 2m /mountpoint/myfile

Using the Command Line to Add a File System Review the command-line operations performed by the VEA software in the previous section. Complete the following step: Review the VEA command-line operations recorded in the log file on the VxVM server. # tail -45 /var/vx/isis/command.log The most recent commands are appended to the end of the file. Not all the details are logged in the command file, such as the edits to the /etc/vfstab file. The following is a summary of using the command line to add a file system to an existing volume. # mkfs -F ufs /dev/vx/rdsk/dgX/xvol-01 409600 # mkdir /Junk # vi /etc/vfstab /dev/vx/dsk/dgX/xvol-01 /dev/vx/rdsk/dgX/xvol-01 /Junk ufs 1 yes logging

# mount -F ufs -o logging /dev/vx/dsk/dgX/xvol-01 /Junk

Note – The file system vfstab and mount options enable the UFS logging feature. UFS logging is not necessary, but it offers additional file system protection and is part of the Solaris OS.

5-42

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 5 – Adding a DRL Perform both variations of this task in the order they occur.

Using the VEA GUI to Add a DRL To add a DRL using the VEA GUI, complete the following steps: 1.

Verify there is a disk drive available for the DRL within the same disk group that does not contain either plex of the mirrored volume.

2.

Display your volume in the grid area and click it with the third mouse button.

3.

Select Log Add in the volume pop-up menu.

4.

In the Add Log window, complete the following steps:

5.

a.

Click Manual disk assignment.

b.

Select a disk drive that is not part of the mirrored volume.

c.

Click OK.

Return to the command line on the VxVM server and use the vxprint command to verify the following: ●

The mirrored volume now has a log plex

●

The log is not on the same disk drive as either of the volume mirrors

Note – Look at the subdisk entries to determine log placement. 6.

In the VEA GUI, complete the following steps: a.

Click the volume in the grid area.

b.

Select Properties from the pop-up menu.

c.

Examine both the General and File System tabs.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-43

Exercise: Creating Volumes and File Systems

Using the Command Line to Add a DRL The following example shows the command sequence to remove and add a DRL to a volume. Practice removing and adding a DRL from your volume using the command line. The disk media name you specify should be on a different disk drive than the disk drives used by the volume mirrors. # vxassist remove log volume_name # vxassist addlog volume_name media_name

5-44

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 6 – Resizing a Volume and File System If a volume has a file system, you can increase the size of both the volume and the file system by using either the vxresize command or the VEA GUI. Note – There are other methods of increasing a volume’s size and a file system’s size. However, the VEA GUI and vxresize command reliably increase the size of both the volume and its related file system at the same time. The volume to be resized should be of the type fsgen. The vxprint command displays the volume type. Perform both variations of this task in the order they occur.

Using the Command Line to Resize File Systems To resize file systems using the command line, complete the following steps: 1.

Add 2 Mbytes to the size of your mirrored volume and file system by using the following command: # vxresize -F ufs -g disk_group volume_name +2m

Note – You can also express the +2m as a new volume length without the plus sign. There are also -s and -x options that ensure the requested size value is appropriate. You can also specify disk media names (for example, disk01, disk02) that you want to be used for the new space. 2.

Examine the new volume and file system to ensure that the changes have taken place. Large changes can take a long time.

Note – You cannot shrink a volume with a file system unless the file system is of VxFS type. Read the vxresize man page for a complete description of restrictions.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-45

Exercise: Creating Volumes and File Systems

Using the VEA GUI to Resize File Systems To resize file systems using the VEA GUI, complete the following steps: 1.

2.

In the grid area, complete the following steps: a.

Display your volume.

b.

Click your volume with the third mouse button.

c.

Select Resize Volume in the pop-up menu.

Configure the Resize Volume form, shown in Figure 5-19, as follows: ●

Enter 2 in the Add By window and select MB from its pulldown menu.

●

Let VxVM decide which disks to use for the additional space.

●

Click OK.

Figure 5-19 Resize File System Form 3.

5-46

After the task has completed, verify the results.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 7 – Creating a RAID-5 Volume In this task you create a three-column RAID-5 volume with a separate log disk (four disk drives total) in your second disk group. Complete the following steps: 1.

2.

On the VxVM server, complete the following steps: a.

Unmount the mirrored volume file system.

b.

Stop the volume.

c.

Remove its mount entry from the /etc/vfstab file.

Destroy the mirrored volume disk group (dgX). # vxdg destroy disk_group

3.

Click the remaining disk group with the third mouse button and select Add Disk to Disk Group from its pop-up menu.

4.

Using the Add Disk Wizard, add the three disk drives from the destroyed disk group to the remaining disk group.

5.

On the VxVM server, calculate the available disk space using four of the disk drives in a RAID-5 volume with a log. For example:

# vxassist -g dgB maxsize layout=raid5log dgB01 dgB02 dgB03 dgB04 Maximum volume size: 35348480 (17260Mb) Caution – Do not create a maximum-size volume. 6.

Display the disk drives in your disk group in the grid area.

7.

Select four of the disk drives by simultaneously pressing the Control key while using the left mouse button.

8.

Select the New Volume button in the toolbar.

9.

Configure the New Volume form as follows: ●

Enter your assigned RAID-5 volume name.

●

Enter 200 in the Size field.

●

Select MB from the Size pull-down menu.

●

Choose RAID 5 in the Layout area. The Number of Columns field should automatically be set to 3 with logging is enabled.

●

Leave the default Stripe Unit Size at 32.

●

Click Next.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-47

Exercise: Creating Volumes and File Systems ●

●

Click Create a File System, and complete the following steps: ●

Select a FS Type of ufs.

●

Enter your assigned mount point.

Click Add to File System Table, Mount at Boot and set the fsck pass to 2.

10. Click Next in the Create File System form. 11. Click Finish in the Summary form after you verify that the configuration is correct. 12. On the VxVM server, check the status of the new RAID-5 volume by using the vxprint command. # vxprint -g dgB raidvol TY NAME ASSOC v raidvol raid5 pl raidvol-01 raidvol sd dgB01-01 raidvol-01 sd dgB04-01 raidvol-01 sd dgB05-01 raidvol-01 pl raidvol-02 raidvol sd dgB06-01 raidvol-02

KSTATE ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

LENGTH 409600 416512 208278 208278 208278 3591 3591

PLOFFS 0 0 0 0

STATE ACTIVE ACTIVE LOG -

Note – You can add the log later to better control its placement. The disk drives are not necessarily used in the order you selected them. 13. On the VxVM server, examine the volume creation commands. # tail -25 /var/vx/isis/command.log 14. In the VEA GUI, click the volume with the third mouse button and select Properties from its pop-up menu. 15. Examine both the General and File System tabs. 16. Cancel the Volume Properties window. 17. Edit the /etc/vfstab file, and comment out or delete the obsolete mount entry from the earlier mirrored volume file system mount. You must remove or comment out obsolete mount entries because they cause errors at boot time.

5-48

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems

Task 8 – Analyzing Volumes Using the VEA GUI The following sections detail different VEA GUI features that are used to analyze the structure and performance of VxVM volumes.

Displaying Volume Layout Details To display volume layout details, complete the following steps: 1.

2.

Display your volumes in the grid area and complete the following steps: a.

Click your RAID-5 volume with the third mouse button.

b.

Select Layout View from the pop-up menu.

As shown in Figure 5-20, use the third mouse button to highlight a volume component and examine its properties.

Figure 5-20 Volume Layout Window

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-49

Exercise: Creating Volumes and File Systems 3.

Examine the pop-up menus for each of the different components.

4.

Close the Volume Layout window.

Viewing Disk Volume Mapping and Performance To view volume-to-disk mapping and performance, complete the following steps: 1.

In the grid area, click your disk group with the third mouse button, and select Volume/Disk from the pop-up menu. The Volume/Disk map, shown in Figure 5-21, assists in volume planning and disk utilization assesment.

Figure 5-21 Volume/Disk Map 2.

5-50

Click the View menu with the third mouse button and select Data Gathering Options from the pull-down menu.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems 3.

Perform the following steps: a.

Modify the Refresh Interval to 5 seconds and select Write Requests from the pull-down menu, as shown in Figure 5-22.

b.

Click OK.

Figure 5-22 Data Gathering Options

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-51

Exercise: Creating Volumes and File Systems 4.

Enable performance monitoring by performing the following steps: a.

Click the View menu with the third mouse button.

b.

Select Collect Statistics from the pop-up menu, as shown in Figure 5-23.

Figure 5-23 Enabling Performance Monitoring When performance monitoring is enabled, the relative I/O activity of each volume disk drive is indicated by color changes in the small disk icons shown in Figure 5-23. Red indicates the highest level of I/O activity or slowest performance. This information can be used to identify basic performance bottlenecks in disk group volumes.

5-52

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Creating Volumes and File Systems 5.

Use the mkfile command to create some test data into your file system volume.

6.

Observe the activity levels in the Disk/Volume performance display For example: mkfile 20m /Test/file1

Figure 5-24 Data Activity Indicators

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-53

Exercise: Creating Volumes and File Systems

Task 9 – Verifying Ending Lab Status Type the vxprint command and verify your current VxVM configuration has the following features:

5-54

●

One disk group containing six disks

●

One RAID-5 volume with a log, mounted

●

Two free disk drives left in the disk group

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Volume Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

5-55

Module 6

VERITAS Volume Manager Advanced Operations Objectives Upon completion of this module, you should be able to: ●

Encapsulate and mirror the system boot disk

●

Configure a best practice boot disk

●

Administer hot spares and hot relocation

●

Evacuate all subdisks from a disk drive

●

Move a disk drive without preserving data

●

Move a populated disk drive to a new disk group

●

Backup and restore a disk group configuration

●

Describe how to import a disk group after a system crash

●

Perform a volume snapshot backup

●

Perform an online volume relayout

●

Create VxVM layered volumes

●

Perform basic Intelligent Storage Provisioning administration

●

Replace a failed disk drive

6-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Boot Disk Encapsulation and Mirroring When you install the VxVM software on a system, you can place your system boot disk under VxVM control. Later, you can mirror the boot disk to increase availability. You should optimize your boot disk configuration before starting the encapsulation process.

Optimizing the Boot Disk Hardware Configuration Although there are many possible boot disk configuration variations, this section focuses on the preferred boot disk hardware configuration, which is shown in Figure 6-1.

SCSI SCSI

SOC

rootdg disk group

c0 c1

c2

rootvol-01

rootvol-02

newdg disk group Storage Array

Figure 6-1

Boot Disk Hardware Configuration

The ideal boot disk hardware configuration has the following features:

6-2

●

The boot disk and mirror are on separate interfaces.

●

The boot disk and mirror are not in a storage array.

●

Only the boot disk and mirror are in the rootdg disk group.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Boot Disk Encapsulation Prerequisites For the boot disk encapsulation process to succeed, the following prerequisites must be met: ●

The boot disk must have at least two unused slices.

●

The boot disk must not have any slices in use other than slice 2 and the following: ●

root

●

swap

●

var

●

opt

●

usr

There must be at least 2048 sectors at the beginning or end of the boot disk that are not assigned to any partition. These sectors are needed for the private region. If necessary, VxVM takes the space from the end of the swap partition, but this can create a difficult-to-manage boot disk configuration. The following is an ideal boot disk partition map, before encapsulation, that has five unconfigured partitions and five cylinders of unassigned disk space (3875-3879) at the end of the disk. # format -d c0t0d0 ... ... Part Tag Flag 0 root wm 1 swap wu 2 backup wm 3 unassigned wm 4 unassigned wm 5 unassigned wm 6 unassigned wm 7 unassigned wm

Cylinders 0 - 3399 3400 - 3874 0 - 3879 0 0 0 0 0

Size 3.50GB 500.98MB 4.00GB 0 0 0 0 0

Blocks (3400/0/0) 7344000 (475/0/0) 1026000 (3880/0/0) 8380800 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

Note – If your boot disk does not have any free cylinders, you boot the system from CD-ROM in single-user mode. You use the format utility to modify the swap partition size and relabel the disk.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-3

Boot Disk Encapsulation and Mirroring

Encapsulating the System Boot Disk It is best to use the vxdiskadm utility option 2, Encapsulate one or more disks, to encapsulate the system boot disk. The following example shows pertinent excerpts from a typical dialog. # vxdiskadm Select an operation to perform: 2 Select disk devices to encapsulate: [<pattern-list>,all,list,q,?] c0t0d0 Which disk group [,list,q,?] rootdg There is no active disk group named rootdg. Create a new group named rootdg? [y,n,q,?] (default: y) y Use a default disk name for the disk? [y,n,q,?] (default: y) y A new disk group rootdg will be created and the disk device c0t0d0 will be encapsulated and added to the disk group with the disk name rootdg01 Enter desired private region length [<privlen>,q,?] (default: 2048) 2048 The c0t0d0 disk has been configured for encapsulation. The first stage of encapsulation has completed successfully. You should now reboot your system at the earliest possible opportunity. The encapsulation will require two or three reboots which will happen automatically after the next reboot. To reboot execute the command: shutdown -g0 -y -i6 This will update the /etc/vfstab file so that volume devices are used to mount the file systems on this disk device. You will need to update any other references such as backup scripts, databases,or manually created swap devices.

6-4

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Mirroring the System Boot Disk You use the vxdiskadm utility to add a mirror disk to the rootdg disk group and to mirror the boot disk. The following examples show pertinent excerpts from the vxdiskadm dialog.

Adding a Second Disk to the rootdg Disk Group You use the vxdiskadm utility option 1, Add or initialize one or more disks, to add a second disk drive to the rootdg disk group. The following example shows a completed disk group that is ready for mirroring. # vxdisk -g rootdg list DEVICE TYPE c0t0d0s2 auto:sliced c0t1d0s2 auto:sliced

DISK rootdg01 rootdg02

GROUP rootdg rootdg

STATUS online online nohotuse

Mirroring the Boot Disk You use the vxdiskadm utility option 6, Mirror volumes on a disk, to mirror the encapsulated boot disk. The following example shows pertinent excerpts from the vxdiskadm dialog. # vxdiskadm Select an operation to perform: 6 Mirror volumes on a disk Enter disk name [,list,q,?] rootdg01 Enter destination disk [,list,q,?] (default: any) rootdg02 The requested operation is to mirror all volumes on disk rootdg01 in disk group rootdg onto available disk space on disk rootdg02. VxVM NOTICE V-5-2-229 NOTE: This operation can take a long time to complete. Continue with operation? [y,n,q,?] (default: y) y VxVM vxmirror INFO V-5-2-22 Mirror volume rootvol ... VxVM vxmirror INFO V-5-2-22 Mirror volume swapvol ... VxVM INFO V-5-2-674 Mirroring of disk rootdg01 is complete. Mirror volumes on another disk? [y,n,q,?] (default: n) n

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-5

Boot Disk Encapsulation and Mirroring

Verifying the Completed Boot Disk Configuration The following examples show an encapsulated and mirrored boot disk. A second boot device alias is automatically configured when the boot disk is mirrored.

Verifying the VxVM Boot Device Aliases When a boot disk is encapsulated, the VxVM software automatically creates boot device aliases for use when booting from the encapsulated boot disk root partition or the boot disk mirror, for example: # eeprom nvramrc nvramrc=devalias vx-rootdg01 /pci@1f,4000/scsi@3/disk@0,0:a devalias vx-rootdg02 /pci@1f,4000/scsi@3/disk@1,0:a You cannot boot the system using the device aliases until the OpenBoot use-nvramrc? variable is set to true. After the variable is enabled, you can boot from the primary or mirror boot device aliases, for example: # eeprom “use-nvramrc?”=true # init 0 ok boot vx-rootdg01 # init 0 ok boot vx-rootdg02

Verifying the VxVM Reserved Disk Group Variables VxVM has two reserved variables named defaultdg and bootdg. Unless special action is taken, both of the variables are set to a value of nodg. When you encapsulate the system boot disk and place it in the rootdg disk group, the bootdg variable is automatically updated. The following example shows the VxVM reserved variables: # vxdg defaultdg nodg # vxdg bootdg rootdg

6-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation and Mirroring

Verifying the rootdg Disk Group Structures The following is an example of a typical encapsulated and mirrored boot disk volume structure. # vxprint -g rootdg TY NAME ASSOC dg rootdg rootdg

KSTATE -

LENGTH -

dm rootdg01 dm rootdg02

c0t0d0s2 c0t1d0s2

-

17674902 17678493 -

NOHOTUSE

v pl sd sd pl sd

rootvol rootvol-01 rootdg01-B0 rootdg01-02 rootvol-02 rootdg02-01

root rootvol rootvol-01 rootvol-01 rootvol rootvol-02

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

17139843 17139843 1 17139842 17139843 17139843

0 1 0

ACTIVE ACTIVE ACTIVE -

v pl sd pl sd

swapvol swapvol-01 rootdg01-01 swapvol-02 rootdg02-02

swap swapvol swapvol-01 swapvol swapvol-02

ENABLED ENABLED ENABLED ENABLED ENABLED

531468 531468 531468 531468 531468

0 0

ACTIVE ACTIVE ACTIVE -

PLOFFS -

STATE -

Verifying the Encapsulated Boot Disk Partitioning The following is the boot disk partition map after the encapsulation process is completed. The public region (slice 3) is mapped to the entire disk. The private region (slice 4) is mapped to the last cylinder. The root and swap data are in the same location as before the encapsulation. # format -d c0t1d0 partition> p Current partition table (original): Total disk cylinders available: 4924 + 2 (reserved cylinders) Part Tag 0 root 1 swap 2 backup 3 4 5 unassigned 6 unassigned 7 unassigned

Flag wm wu wm wu wu wm wm wm

Cylinders 0 - 4772 4773 - 4920 0 - 4923 0 - 4923 4922 - 4923 0 0 0

Size 8.17GB 259.51MB 8.43GB 8.43GB 3.51MB 0 0 0

Blocks (4773/0/0) 17139843 (148/0/0) 531468 (4924/0/0) 17682084 (4924/0/0) 17682084 (2/0/0) 7182 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-7

Creating a Best Practice Boot Disk Configuration

Creating a Best Practice Boot Disk Configuration If you encapsulate and mirror your boot disk, you should consider refining the configuration to improve emergency recovery and serviceability. If there are no free cylinders for use as a private region during encapsulation, VxVM takes them from the swap space and places the private region (slice 4) in the middle of the public region address space (slice 3). The private region is protected by a subdisk overlay named rootdg01Priv. Also, block 0 on the boot disk is protected with a subdisk overlay named rootdg01-B0. The following example shows an encapsulated boot disk structure: # format -d c0t0d0 ... Part Tag Flag 0 root wm 1 swap wu 2 backup wm 3 wu 4 wu 5 unassigned wm 6 unassigned wm 7 unassigned wm

Cylinders 0 - 3614 3618 - 3879 0 - 3879 0 - 3879 3615 - 3617 0 0 0

Size 3.72GB 276.33MB 4.00GB 4.00GB 3.16MB 0 0 0

Blocks (3615/0/0) 7808400 (262/0/0) 565920 (3880/0/0) 8380800 (3880/0/0) 8380800 (3/0/0) 6480 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

# vxprint Disk group: rootdg TY NAME dg rootdg

ASSOC rootdg

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm rootdg01

c0t0d0s2

-

8380799

-

-

sd rootdg01Priv -

ENABLED

6479

-

-

v pl sd sd

rootvol rootvol-01 rootdg01-B0 rootdg01-02

root rootvol rootvol-01 rootvol-01

ENABLED ENABLED ENABLED ENABLED

7808400 7808400 1 7808399

0 1

ACTIVE ACTIVE -

v swapvol pl swapvol-01 sd rootdg01-01

swap swapvol swapvol-01

ENABLED ENABLED ENABLED

565920 565920 565920

0

ACTIVE ACTIVE -

6-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating a Best Practice Boot Disk Configuration When the boot disk is mirrored, the structure of the boot disk and the mirror disk are not identical. This can be confusing and can add difficulty to service and recovery situations. The partition maps of a wort-case boot disk and mirror disk configuration are organized in a different manner, for example: # format -d c0t0d0 ... Part Tag Flag 0 root wm 1 swap wu 2 backup wm 3 wu 4 wu 5 unassigned wm 6 unassigned wm 7 unassigned wm

Cylinders 0 - 3614 3618 - 3879 0 - 3879 0 - 3879 3615 - 3617 0 0 0

Size 3.72GB 276.33MB 4.00GB 4.00GB 3.16MB 0 0 0

Blocks (3615/0/0) 7808400 (262/0/0) 565920 (3880/0/0) 8380800 (3880/0/0) 8380800 (3/0/0) 6480 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

# format -d c0t1d0 ... Part Tag Flag 0 root wm 1 swap wu 2 backup wu 3 wu 4 wu 5 unassigned wm 6 unassigned wm 7 unassigned wm

Cylinders 3 - 3617 3618 - 3879 0 - 3879 3 - 3879 0 2 0 0 0

Size 3.72GB 276.33MB 4.00GB 3.99GB 3.16MB 0 0 0

Blocks (3615/0/0) 7808400 (262/0/0) 565920 (3880/0/0) 8380800 (3877/0/0) 8374320 (3/0/0) 6480 (0/0/0) 0 (0/0/0) 0 (0/0/0) 0

The following differences exist between the two disk drives: ●

The VxVM private region (partition 4) is in the middle of the primary boot disk (c0t0d0) and is at the beginning of the mirror disk (c0t1d0).

●

The root and swap partitions on the primary boot disk still remain in the same physical location, aligned with the original partition boundaries. They are offset on the mirror disk.

●

The boot disk is still directly bootable (ok boot disk). The mirror disk must be booted using the nvramrc device alias vx-rootdg02.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-9

Creating a Best Practice Boot Disk Configuration

Establishing Boot Disk Configuration Policies Optimized boot disk configurations can make recovery faster and safer. Administrators can adopt recovery procedures that work for any system under their control. If possible, adhere to the following configuration policies: ●

Document each system’s configuration and keep hard copies of the configurations easily available. Do not rely on having electronic copies available.

●

Keep the /usr and /opt directories on the root partition if possible. The content of these directories is relatively stable and keeping them on the root partition simplifies the configuration. Additionally, the /usr directory contains the VxVM recovery software. Placing the root and /usr software on the same partition reduces the possibility of having a bootable root file system, but losing access to the VxVM recovery software residing on a separate /usr partition.

●

If possible, also leave the /var directory on the root partition.

●

Add an additional disk drive to the rootdg disk group as a hot spare.

Caution – Ensure that the mirror disk is initialized in the VxVM sliced format and not the cdsdisk format. The cdsdisk format must not be used in the rootdg disk group. Consult the Sun BluePrints™ document, Towards a Reference Configuration for VxVM Managed Boot Disks, for a more detailed explanation.

6-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating a Best Practice Boot Disk Configuration

Modifying an Existing Boot Disk Configuration The following section assumes that all disk drives and volume components follow the default VxVM naming conventions. After using the vxdiskadm utility to create a standard encapsulated boot disk and mirror configuration, perform the following steps to reorganize the disks into a best practice configuration. 1.

Un-mirror the primary boot disk by removing the mirror plexes from the mirror disk. Use the Bourne shell. # vxassist -g rootdg remove mirror rootvol !rootdg02 # vxassist -g rootdg remove mirror swapvol !rootdg02

2.

To ensure the correct mirror placement, manually mirror the rootvol and swapvol volumes again in the order shown. # vxrootmir rootdg02 # vxassist -g rootdg mirror swapvol rootdg02

3.

Disassociate all primary boot disk plexes and recursively remove them. # # # # #

4.

vxplex vxplex vxedit vxedit vxedit

-g -g -g -g -g

rootdg rootdg rootdg rootdg rootdg

-v -v -r -r -r

rootvol dis rootvol-01 swapvol dis swapvol-01 rm rootdg01Priv rm rootvol-01 rm swapvol-01

Verify that only the mirror disk plexes remain.

# vxprint -g rootdg TY NAME ASSOC dg rootdg rootdg

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm rootdg01 dm rootdg02

c0t0d0s2 c0t1d0s2

-

8380799 8374320

-

NOHOTUSE

v rootvol pl rootvol-02 sd rootdg02-01

root rootvol rootvol-02

ENABLED ENABLED ENABLED

7808400 7808400 7808400

0

ACTIVE ACTIVE -

v swapvol pl swapvol-02 sd rootdg02-02

swap swapvol swapvol-02

ENABLED ENABLED ENABLED

565920 565920 565920

0

ACTIVE ACTIVE -

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-11

Creating a Best Practice Boot Disk Configuration 5.

Remove the rootdg01 disk from the rootdg disk group and complete the following steps: a.

Reinitialize the rootdg01 disk.

b.

Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01 # vxdisksetup -i c0t0d0 format=simple # vxdg -g rootdg adddisk rootdg01=c0t0d0 6.

Use the vxdiskadm option 6, Mirror volume on a disk, to mirror rootvol and swapvol on the rootdg02 disk drive back to the newly initialized rootdg01 disk drive. # vxdiskadm ... At the prompt below, supply the name of the disk containing the volumes to be mirrored. Enter disk name [,list,q,?] rootdg02 Enter destination disk [,list,q,?] (default: any) rootdg01

7.

Modify the boot-device parameter and enable nvramrc usage. # eeprom boot-device="rootdisk rootmirror" # eeprom "use-nvramrc?"=true

8.

Use the prtvtoc command and the format utility to verify that the primary and mirror boot disk drives have exactly the same partition maps.

9.

Boot the system from each of the available system devices. ok ok ok ok

boot boot boot boot

disk0 disk1. vx-rootdg01 vx-rootdg02

You can now replace a defective boot disk drive in the same manner as any other VxVM disk drive, and then resynchronize the mirrors.

6-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Administering Hot Devices Depending on how the /etc/rc2.d/S95vxvm-recover file is configured, either the older hot-spare daemon, vxsparecheck, starts at boot time or the newer hot-relocation daemon, vxrelocd, starts at boot time. The functionality of the two daemons is different. By default, hot relocation is enabled. Either mode of operation requires that the failed object is redundant. Effectively, the hot-device feature makes a copy of a surviving object mirror.

Selecting Hot Device Operational Mode You can configure either of two hot device modes of operation. Most administrators prefer the newer hot-relocation mode.

Hot-Spare Functionality (Legacy Mode of Operation) In early versions of VxVM, the hot-spare daemon, vxsparecheck, detected and reacted to total disk media failures. It did this by moving all redundant objects on the failed disk drive to a pre-designated spare disk drive in the disk group. Hot-sparing is an older mode of operation, but can still be enabled, if necessary, by editing the /etc/rc2.d/S95vxvm-recover file and commenting out the vxrelocd root & line, and then uncommenting the #vxsparecheck root & line.

Hot-Relocation Functionality The hot-relocation daemon, vxrelocd, detects and reacts to partial disk media failures. It does this by copying the affected subdisk mirror to free space on a different disk drive in the group. Free space can be found on disk drives that have been designated as hot spares. If there are no designated hot spares, VxVM uses available free space on any disk drive in the disk group that does not have the nohotuse flag set. Hot relocation can also be performed for subdisks that are part of a RAID-5 volume.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-13

Administering Hot Devices Hot relocation is enabled by default and goes into effect, without system administrator intervention, when a failure occurs. As shown in Figure 6-2, when a subdisk failure is detected, the contents of the subdisk are reconstructed on the designated hot spare. The volume continues to function with its original full redundancy. Volume

Hot spare

Private

Private

Primary Subdisk

Mirror Subdisk

Figure 6-2

Private Copy

New Subdisk

Subdisk Relocation

The hot-relocation daemon, vxrelocd, detects and reacts to the following types of failures: ●

Disk drive failure This is first detected as an I/O failure from a VxVM object. VxVM attempts to correct the error. If the error cannot be corrected, VxVM tries to access configuration information in the private region of the disk drive. If it cannot access the private regions, it considers the disk drive to have failed. All plexes associated with the failed disk drive are eventually detached. The output of vxprint shows the plex in an IOFAIL state. This state is a hard failure that typically requires replacement of the disk drive.

●

Plex failure This is detected as an uncorrectable I/O error in the plex. For mirrored volumes, the plex is detached. Typically, this type of failure is caused by a block read or write error that cannot be recovered with a series of retry operations. It is a good idea to periodically monitor for these types of errors. They might show up as correctable errors for a while and then become hard errors. Sometimes, recoverable read/write errors are spurious and not seen again.

6-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices ●

RAID-5 subdisk failure This is detected as an uncorrectable I/O error in one of the RAID-5 subdisks. The subdisk is detached. The failure is caused by the same situation as plex failures: unrecoverable block read/write errors.

Hot relocation is not possible if any of the following are true: ●

Subdisks do not belong to a mirrored or RAID-5 volume.

●

Not enough spare disk drive space is available.

●

The only available space for relocation is on a disk drive that contains any portion of the surviving mirror or RAID-5 volume.

●

A mirrored volume has a DRL subdisk as part of its data plex: subdisks belonging to that plex cannot be relocated.

●

The failure is a log plex: a new log plex is created and there is no relocation.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-15

Administering Hot Devices

Evaluating Hot-Device Configurations You can verify the hot-relocation and hot-spare status of disk drives from the command line. The following example shows command-line verification: # vxdg -g dgY nohotuse DISK DEVICE TAG dgY04 c2t1d0s2 c2t1d0

OFFSET 0

LENGTH 17674902

FLAGS n

# vxdg -g dgY spare DISK DEVICE TAG dgY05 c2t3d0s2 c2t3d0

OFFSET 0

LENGTH 17674902

FLAGS s

You can also verify and modify disk drive hot-device status in the VEA grid area, as shown in Figure 6-3.

Figure 6-3

6-16

VEA Hot-Device Flags

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Administering Hot Devices Using Command-Line Programs You use the vxedit command-line program to administer hot devices.

Designating Hot-Spare Disk Drives A disk designated as a spare is used only for hot relocation. The vxassist utility will not allocate a subdisk on that disk unless forced to by command-line arguments. You designate disk drive as spares using the vxedit command, and you verify the spare status of the disk drives with the vxdisk command. The following example shows the command-line process: # vxedit -g # vxdisk -g DEVICE c2t1d0s2 c2t3d0s2 c2t5d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2

dgY set spare=on dgY05 dgY list TYPE DISK GROUP sliced dgY04 dgY sliced dgY05 dgY sliced dgY01 dgY sliced dgY06 dgY sliced dgY02 dgY sliced dgY03 dgY

STATUS online online spare online online online online

Note – If a disk drive is marked as a hot spare, the vxassist utility does not create a subdisk on that disk drive unless the disk drive is specifically designated in command-line arguments.

Blocking Hot Relocation If hot relocation is enabled (the default), you can use any disk drive with free space during the relocation of a failed subdisk if there is no hot-spare space available. If you do not want a disk drive to be used for hot relocation, you can mark it for no hot use as follows: # vxedit -g dgY set nohotuse=on dgY04 # vxdisk -g dgY list DEVICE c2t1d0s2 c2t3d0s2 c2t5d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2

TYPE sliced sliced sliced sliced sliced sliced

DISK dgY04 dgY05 dgY01 dgY06 dgY02 dgY03

GROUP dgY dgY dgY dgY dgY dgY

STATUS online nohotuse online spare online online online online

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-17

Administering Hot Devices

Administering Hot Devices Using the VEA GUI Display a disk group’s disk drives in the grid area, click a disk drive with the third mouse button, and then select Set Disk Usage from its pop-up menu. Set or reset Spare, No hot use, Reserved, or Reserved for Allocator, as shown in Figure 6-4. The Spare and No hot use boxes are mutually exclusive.

Figure 6-4

VEA Hot-Device Administration

The Reserve flag, if set, prevents automatic space allocation by utilities, such as the vxassist program, unless the disk drive is specified on the command line. The Reserved for Allocator flag, if set, reserves a disk for exclusive use by ISP utilities such as the vxpool and vxvoladm commands.

6-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Hot Devices

Controlling Relocation Recovery Time You can reduce the impact of recovery on system performance by instructing the vxrelocd daemon to increase the delay between the recovery of each region of a volume. Use the vxrelocd daemon as follows: # vxrelocd -o slow=500 & The value of slow is passed on to the vxrecover command. The default value is 250 milliseconds.

Monitoring Errors By default, the vxrelocd daemon sends email notification of errors to the server root account. You can modify the account name in the vxrelocd root & line in the etc/rc2.d/S95vxvm-recover file. You can also examine system error logs for evidence of disk drive problems, but the email notification to root is usually sufficient.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-19

Evacuating a Disk Drive

Evacuating a Disk Drive The volume structures on a disk drive that is starting to experience recoverable data errors can be evacuated to a different disk drive before the disk drive fails entirely. Evacuation can reduce the risk of data loss. You can also use evacuation to reduce or eliminate performance bottlenecks that have been identified. Evacuation can only be performed on disk drives within the same group.

Identifying Evacuation Conflicts Before you proceed with disk drive evacuation, carefully investigate the configuration of both the failing disk drive and the new disk drive. You should verify that the evacuation process is not going to create any of the following conflicts: ●

Both volume mirrors are on the same physical disk drive.

●

More than one stripe column of a striped or RAID-5 volume is on the same disk drive.

Preparing for Evacuation Before starting the evacuation process, you must:

6-20

●

Find out with what volume the failing plex or subdisk is associated, and determine the name of the disk drives that are associated with it.

●

Find out what disk group is associated with the failing disk drive.

●

Determine if any other volumes are associated with the failing disk drive.

●

Find a new disk drive with enough free space to perform the evacuation.

●

Check for any volume conflicts associated with the new disk drive.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Evacuating a Disk Drive

Identifying Suitable Evacuation Disk Drives The following example shows how to prepare for disk drive evacuation if you are notified of a correctable read error on a subdisk named dgX02-01 in disk group dgX. # vxprint -g dgX TY NAME ASSOC dg dgX dgX

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm dm dm dm dm dm

dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

c2t16d0s2 c2t18d0s2 c2t1d0s2 c2t3d0s2 c2t5d0s2 c2t20d0s2

-

17674896 17674896 17674896 17674896 17674896 17674896

-

NOHOTUSE SPARE

v pl sd sd sd sd sd

r5demo r5demo-01 dgX03-01 dgX04-01 dgX05-01 dgX01-01 dgX02-01

raid5 r5demo r5demo-01 r5demo-01 r5demo-01 r5demo-01 r5demo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 15360 15360 15360 15360 15360

0 0 0 0 0

ACTIVE ACTIVE -

v pl sd sd sd sd sd

stdemo stdemo-01 dgX03-02 dgX04-02 dgX05-02 dgX01-02 dgX02-02

fsgen stdemo stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 12288 12288 12288 12288 12288

0 0 0 0 0

ACTIVE ACTIVE -

An analysis of the preceding vxprint output indicates the following: ●

The failing subdisk, dgX02-01, is on disk drive dgX02.

●

The dgX02-02 subdisk in another volume is also on dgX02.

●

Disk dgX06 appears to be unused, is marked as a spare, and is a suitable candidate for evacuating disk dgX02.

For this scenario, evacuate the disk dgX02 to disk dgX06. After the evacuation has completed, replace disk dgX02. You might mark the new dgX02 disk drive as the spare and unmark dgX06. You could also migrate the stripe data from dgX06 back to the new dgX02 disk drive.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-21

Evacuating a Disk Drive

Evacuation Using the vxevac Command The vxevac command moves subdisks from the specified disk drive to a new disk drive. If a destination disk drive is not specified, the vxevac command uses any available disk drive. The following example shows a typical vxevac procedure. # vxevac -g dgX dgX02 dgX06 # vxprint -g dgX TY NAME ASSOC dg dgX dgX

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm dm dm dm dm dm

dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

c2t16d0s2 c2t18d0s2 c2t1d0s2 c2t3d0s2 c2t5d0s2 c2t20d0s2

-

17674896 17674896 17674896 17674896 17674896 17674896

-

NOHOTUSE SPARE

v pl sd sd sd sd sd

r5demo r5demo-01 dgX03-01 dgX04-01 dgX05-01 dgX01-01 dgX06-01

raid5 r5demo r5demo-01 r5demo-01 r5demo-01 r5demo-01 r5demo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 15360 15360 15360 15360 15360

0 0 0 0 0

ACTIVE ACTIVE -

v pl sd sd sd sd sd

stdemo stdemo-01 dgX03-02 dgX04-02 dgX05-02 dgX01-02 dgX06-02

fsgen stdemo stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 12288 12288 12288 12288 12288

0 0 0 0 0

ACTIVE ACTIVE -

6-22

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Evacuating a Disk Drive

Evacuation Using the VEA GUI To perform an evacuation using the VEA GUI, complete the following steps: 1.

Select the disk drive that contains the objects and data to be moved.

2.

Select Evacuate Disk from the disk drive’s pop-up menu.

3.

Configure the Evacuate Disk form, as shown in Figure 6-5.

Figure 6-5

VEA Evacuate Disk Form

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-23

Evacuating a Disk Drive

Evacuation Using the vxdiskadm Utility To perform an evacuation using the vxdiskadm utility, complete the following steps: 1.

Start the vxdiskadm utility, and select Option 7, Move volumes from a disk.

2.

Type the media name of the disk drive to be evacuated.

3.

Type the media name of the destination disk drive.

The following example shows a summary of the process. # vxdiskadm Select an operation to perform: 7 Move volumes from a disk Enter disk name [,list,q,?] dgX06 Enter disk name [,list,q,?] dgX02 VxVM NOTICE V-5-2-283 Requested operation is to move all volumes from disk dgX06 in group dgX. NOTE: This operation can take a long time to complete. Continue with operation? [y,n,q,?] (default: y) y VxVM vxevac INFO V-5-2-24 Move volume r5demo ... VxVM vxevac INFO V-5-2-24 Move volume stdemo ... VxVM INFO V-5-2-188 Evacuation of disk dgX06 is complete. Move volumes from another disk? [y,n,q,?] (default: n) n

6-24

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Disk Drives Without Preserving Data

Moving Disk Drives Without Preserving Data You might want to move a VxVM disk drive to a different disk group because the destination disk group needs the additional disk space. As long as the disk drive does not contain any functional data that you need to preserve, the process is fairly easy.

Moving a Disk Drive Using the Command Line If the disk drive you want to move contains an active volume and you do not care if the data is lost, complete the following steps to move the disk drive to a different disk group: 1.

Use the vxprint command to verify that there are no unexpected volumes, or portions of volumes, associated with the disk drive you want to move.

2.

Disable any applications related to the volume.

3.

Unmount any file systems related to the volume.

4.

Stop the volume. You stop volumes using the vxvol command as follows: # vxvol -g old_dg stop vol-02

5.

Delete the volume configuration. Recursively delete all objects in a volume as follows: # vxedit -g old_dg -rf rm vol-02

6.

Remove the disk drive from the disk group. You use the vxdg command to remove a disk drive from a disk group as follows: # vxdg -g old_dg rmdisk olddg-12 Even after the vxdg rmdisk operation, the disk drive is still initialized for VxVM use. The vxdiskunsetup command completely removes a disk drive from VxVM control.

7.

Add the disk drive to a different disk group. You use the vxdg command to add the disk drive to a different disk group as follows: # vxdg -g new_dg adddisk newdg-02=c1t3d0

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-25

Moving Disk Drives Without Preserving Data

Moving a Disk Drive Using the VEA GUI Moving a disk drive to a new disk group is easy using the VEA GUI. You use the following process to reduce errors: 1.

In the grid area, click the third mouse button on the disk group and select Disk/Volume Map from the pop-up menu.

2.

Examine the Volume to Disk Mapping display, shown in Figure 6-6, and verify that there are no unexpected volume structures associated with the disk drive you want to move.

Figure 6-6 3.

Analyzing Volume to Disk Relationships

Use the pop-up menus available on the volume name and disk drive name to perform any of the following actions: ●

Stop any volumes on the disk drive that you want to remove.

●

Delete the volume or volumes.

●

Remove the disk from the disk group. The disk drive is returned to the free disk pool and can now be added to a different disk group.

6-26

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Populated Disk Drives to a New Disk Group

Moving Populated Disk Drives to a New Disk Group Moving populated VxVM disk drives to a new or different disk group is a technique you might use occasionally. One reason to use this technique is if you have mistakenly created all of your volumes in a single disk group. To resolve a performance problem, you would move some of the volumes and their related disk drives to a different disk group. Caution – Do not perform this operation on a production system without first backing up the data on all associated volumes. If this process fails, there is no way to recover data without backup tapes. In this section, a volume called mirvol is moved from a disk group named old_dg to a new disk group named new_dg. The process of moving populated disks is easier if you have VERITAS FastResync. Currently, Sun does not sell, license, or support this option. VERITAS FastResync enables the vxdg move/split/join options.

Evaluating Disk Drive Involvement Before you take any action, you must determine which physical disk drives are part of your target volume. You must also ensure that the disk drives are not being used by other volumes.

Determining Disk Drive Involvement From the Command Line Use the vxprint command as follows to determine the volume to disk drive mapping: # vxprint -g old_dg TY NAME ASSOC dg old_dg old_dg

KSTATE -

LENGTH -

dm olddg-01 dm olddg-02 dm olddg-03

c2t16d0s2 c3t37d0s2 c3t52d0s2

-

17674902 17674902 17674902 -

-

v pl sd pl sd

fsgen mirvol mirvol-01 mirvol mirvol-02

ENABLED ENABLED ENABLED ENABLED ENABLED

204800 208278 208278 208278 208278

ACTIVE ACTIVE ACTIVE -

mirvol mirvol-01 olddg-01-01 mirvol-02 olddg-02-01

PLOFFS -

0 0

STATE -

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-27

Moving Populated Disk Drives to a New Disk Group In the preceding example, the volume mirvol contains two plexes with a single subdisk associated with each plex. The volume is associated with two disk drives, olddg-01 and olddg-02. Note – The VEA GUI’s Disk/Volume Map display can be very helpful when you are trying to determine volume involvement with specific disk drives.

Saving the Configuration Use the vxprint command to save the volume configuration of the mirvol volume in a file named save_mirvol. # vxprint -hmQqrL -g old_dg mirvol > \ /var/tmp/save_mirvol Caution – If you are saving layered volumes that have sub-volumes (such as striped mirror structures), you must add the r and L options to the vxprint command. If you fail to do this, the saved configuration information is incomplete. The vxprint -m option provides detailed configuration information in a format that can be used later by the vxmake utility. Table 6-1 shows the vxprint command options used in this example. Table 6-1 The vxprint Command Options

6-28

Command

Function

-h

Lists complete hierarchies.

-m

Displays information in a format that can be used as input to the vxmake utility.

-Q

Suppresses the disk group header that separates each disk group.

-q

Suppresses headers (in addition to disk group header).

-r

Displays related records of a volume containing subvolumes.

-L

Affects record grouping when used with the -r option.

-g

Specifies the disk group.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Moving Populated Disk Drives to a New Disk Group

Moving the Disk Drives to a New Disk Group Moving the disk drives to a new disk group requires several steps that you have seen earlier in this course. They are: 1.

Unmount appropriate file systems and stop any processes that are accessing the mirvol volume directly. # umount /Data

2.

Stop the volume. # vxvol -g old_dg stop mirvol

3.

Remove the definitions of the structures (volume, plexes, and subdisks) from the configuration database. # vxedit -g old_dg -rf rm mirvol The vxedit command removes the definitions of the volume, plexes, and subdisks from the configuration database for the old disk group, old_dg.

Note – Removing the definitions does not affect the data: it only removes selected records from the configuration database. The -r option recursively removes the volume and all associated plexes and subdisks. 4.

Remove the disk drives from the original disk group. # vxdg -g old_dg rmdisk olddg-01 olddg-02

5.

If the new disk group, new_dg, does not exist, initialize it using one of the disk drives to be moved (disk old_dg-01, in this example). # vxdg init new_dg olddg-01=c2t16d0

Caution – It is critical that all of the disk drives retain their original media names when they are added to the new disk group. 6.

Add the remaining disk drives to the new disk group. # vxdg -g new_dg adddisk olddg-02=c3t37d0

7.

Verify that the disk drives have been added to the new disk group. # vxdisk list | grep new_dg c2t16d0s2 sliced olddg-01 c3t37d0s2 sliced olddg-02 c3t52d0s2 sliced newdg-01

new_dg new_dg new_dg

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

online online online

6-29

Moving Populated Disk Drives to a New Disk Group

Reloading the Volume Configuration To reload the volume configuration, complete the following steps: 1.

Use the vxmake command to reload the saved configuration for the volume vol01. # vxmake -g dgY -d /var/tmp/save_mirvol Earlier the volume configuration was saved in the save_mirvol file. The -d option specifies the description file to use for building subdisks, plexes, and volumes.

Caution – If the disk drives do not have their original media names, the configuration reload fails. 2.

Check the volume status. The volume and plexes should be in a DISABLED state.

# vxprint -g new_dg mirvol TY NAME ASSOC KSTATE LENGTH v mirvol fsgen DISABLED 204800 pl mirvol-01 mirvol DISABLED 208278 sd olddg-01-01 mirvol-01 ENABLED 208278 pl mirvol-02 mirvol DISABLED 208278 sd olddg-02-01 mirvol-02 ENABLED 208278 3.

PLOFFS 0 0

STATE EMPTY EMPTY EMPTY -

Use the vxvol command to bring the volumes back online. # vxvol -g new_dg init active mirvol

Note – An alternative to this procedure is to create a new volume in another disk group, and either dump a backup tape onto it or perform a direct copy from the old volume. 4.

Edit the /etc/vfstab mount information for the volume’s file system and change the disk group in the logical paths.

# vi /etc/vfstab /dev/vx/dsk/new_dg/mirvol 5.

/dev/vx/rdsk/new_dg/mirvol /Data ufs 1 yes -

Mount and test the file system. # mount /Data # ls /Data lost+found

6-30

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Backing Up and Restoring Disk Group Configurations The VxVM software automatically saves disk group configuration backups. You can also manually backup disk group configurations.

Automatic Configuration Backup VxVM keeps updated copies of the configuration status both in memory and written on one or more storage disk drives. This status is updated any time there is a change in the VxVM configuration or availability due to errors or administrative changes. A backup copy of the configuration is automatically updated on the system boot disk when administrative changes are made to the configuration. The primary elements involved are: ●

The vxio software driver

●

The vxconfigd configuration daemon

●

The vxconfigbackupd daemon

Figure 6-7 shows how the kernel configuration table is checked by the vxio driver before the driver attempts to access a virtual structure. The disk-resident copies do not need to be examined. Consult Before Access vxio Driver

Update

Device Access Error

Administrative Modifications Kernel Configuration Table

vxconfigbackupd

vxconfigd /etc/vx/cbr/bk Storage Array Update configdb Data

Figure 6-7

Configuration Monitoring Components

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-31

Backing Up and Restoring Disk Group Configurations

Initial Volume Configuration When VxVM starts, the vxconfigd daemon imports disk groups that belong to the VxVM server. When disk groups are imported, the kernel configuration table is created by the vxconfigd daemon, which reads the disk-resident configdb records.

Status Configuration Changes When the vxio driver is notified of a hard-device error, it disables the volume it was trying to access and updates the VxVM kernel configuration table. The vxio driver also signals the vxconfigd daemon that a configuration change has taken place.

Administrative Configuration Changes System administrator changes to the configuration, such as creating a new disk group, are automatically recorded on the system boot disk by the vxconfigbackupd daemon. A backup directory for each disk group is maintained in the /etc/vx/cbr/bk directory. The following example shows backup files on a system with two disk groups named pridg and secdg. # cd /etc/vx/cbr/bk # ls pridg.1064606899.21.ns-east-115 secdg.1064865853.29.ns-east-115 # # cd pridg.1064606899.21.ns-east-115 # ls 1064606899.21.ns-east-115.binconfig 1064606899.21.ns-east-115.cfgrec 1064606899.21.ns-east-115.dginfo 1064606899.21.ns-east-115.diskinfo

Note – If all array-resident configuration copies are lost, the vxconfigrestore command can be used to rebuild the original structures. The data on the structures is not preserved.

6-32

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Manual Configuration Backup and Restore The vxconfigbackupd daemons are started by the etc/rc2.d/S95vxvm-recover script and record any VxVM disk group configuration changes in the /etc/vx/cbr/bk directory. You use the vxconfigbackup command to backup the current disk group configuration information. You restore a disk group configuration using the vxconfigrestore command. If there is damage to the disk group configuration records that are stored in the private regions of one or more disk drives in a disk group, the disk group import operation might fail. The vxconfigrestore command is used to automatically correct the damaged configuration records or to recreate a disk group from the beginning. When restoring or repairing damaged disk group records, you must meet the following criteria: ●

Failed disk drives must be replaced prior to using the vxconfigrestore command.

●

Replacement disk drives must be initialized for VxVM use prior to using the vxconfigrestore command.

●

All disk drives must have the same physical configuration and logical addresses as when the configuration backup was performed.

Caution – The vxconfigbackup command does not preserve data. File system and raw volume data must be preserved separately using backup commands or applications. Volumes with redundant data, such as mirrored or RAID-5 structure, can recover from the loss of a mirror or a RAID-5 stripe. Non-redundant volume types must be reinitialized for file system use and the data recovered from backup media.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-33

Backing Up and Restoring Disk Group Configurations

Using the vxconfigbackup Command The vxconfigbackup command is used to backup one or more disk groups. If the name of a disk group is not specified with the command, all disk groups are backed up. You use the -l option to specify a backup file location other than the default location in /etc/vx/cbr/bk. The following example shows the use of the vxconfigbackup command. # vxdg list NAME STATE rootdg enabled dgX enabled

ID 1066871088.21.ns-east-104 1066748899.279.ns-east-104

# vxconfigbackup -l /etc/vx/cbr/mybackup dgX Start backing up diskgroup dgX to /etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-104 ... VxVM NOTICE V-5-2-3100 Backup complete for diskgroup: dgX # ls /etc/vx/cbr/mybackup dgX.1066748899.279.ns-east-104 # ls /etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-104 1066748899.279.ns-east-104.binconfig 1066748899.279.ns-east-104.cfgrec 1066748899.279.ns-east-104.dginfo 1066748899.279.ns-east-104.diskinfo

6-34

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Backing Up and Restoring Disk Group Configurations

Using the vxconfigrestore Command When you use the vxconfigrestore command to restore a disk group and its associated structures, the process is performed in two stages: precommit and commit. # mount |grep R5demo /R5demo on /dev/vx/dsk/dgX/r5demo # ls -l /R5demo total 28736 -rw------T 1 root -rw------T 1 root -rw------T 1 root drwx-----2 root # # # #

other other other root

1048576 Oct 27 11:37 file1 3145728 Oct 27 11:37 file2 10485760 Oct 27 19:40 file3 8192 Oct 27 11:35 lost+found

umount /R5demo vxvol -g dgX stop r5demo vxvol -g dgX stop stdemo vxdg deport dgX

# vxdisksetup c2t3d0 (reinitialize the disk) # vxdg import dgX VxVM vxdg ERROR V-5-1-587 Disk group dgX: import failed: Disk for disk group not found # vxconfigrestore -p -l /etc/vx/cbr/mybackup dgX (precommit stage) Installing volume manager disk header for c2t18d0s2 ... Installing volume manager disk header for c2t1d0s2 ... Installing volume manager disk header for c2t20d0s2 ... Installing volume manager disk header for c2t16d0s2 ... Installing volume manager disk header for c2t3d0s2 ... Installing volume manager disk header for c2t5d0s2 ... dgX’s diskgroup configuration is restored Diskgroup can be accessed in read only and can be examined using vxprint in this state. # vxconfigrestore -l /etc/vx/cbr/mybackup -c dgX (commit stage) Committing configuration restoration for diskgroup dgX .... dgX’s diskgroup configuration restoration is committed.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-35

Importing Disk Groups After a System Crash

Importing Disk Groups After a System Crash Depending on the state of a disk group, there are several variations of the import operation that might be useful. The basic import operation is not complicated and can be done using either the vxdg command, the vxdiskadm utility, or the VEA GUI. If a disk group must be imported after a system crash, the process can be more difficult. Following are some of the possible variations. ●

Performing a typical import of a clean disk group: # vxdg import disk_group_name

●

Importing a disk group to another system after a crash: # vxdg -C import disk_group_name The -C option is necessary to clear the old host IDs that were left on the disk drives after the crash. # vxdg -fC import disk_group_name

Caution – The -f option forces an import in the event that all the disk drives are not usable. This option can be dangerous on dual-hosted storage arrays because the disk group might also be imported to another host. A disk group that is imported to two host systems can become corrupted. # vxrecover -g disk_group_name -sb You execute this command after a crash to start the volumes and perform a recovery process. This is done automatically during a reboot. ●

Importing a disk group with a duplicate name: # vxdg -t -n new_disk_group import disk_group_name The -t option makes the new disk group name temporary.

Note – When a disk group is imported, the volumes can be in a disabled state. You can use the vxvol start command to start the volumes.

6-36

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Importing Disk Groups After a System Crash

Importing the rootdg Disk Group After a Crash After a crash it might be necessary to import the rootdg disk group to another system to perform repair operations. This process is a little more complicated because you cannot have two rootdg disk groups on a system. This procedure works only if the boot disk can be easily attached to a different system. Importing a second rootdg disk group requires the use of multiple options to: ●

Assign a new temporary disk group name to rootdg.

●

Clear the original hostid ownership.

●

Use the unique rootdg group identifier: # vxdg -tC -n new_disk_group import group_id

The difficult part is that you must know the unique rootdg group identifier. This value must be known in advance. You can determine the rootdg group identifier with the vxdisk command as follows: # vxdisk -s list Disk: c0t2d0s2 type: sliced flags: online ready private autoconfig autoimport imported diskid: 791000525.1055.boulder dgname: rootdg dgid: 791000499.1025.boulder hostid: boulder The disk group is renamed by the importing host. Note – The vxdisk -s list command lists information for all attached disk drives. Disk drives that belong to a cleanly deported disk group have a blank hostid entry.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-37

Volume Snapshot Operations

Volume Snapshot Operations When you need to back up the data on a volume, such as a file system volume, you can use the VxVM snapshot function to create a copy of the volume. You can then back up the new copy to tape without disrupting service to the original volume.

Snapshot Process You must satisfy the following prerequisites before the snapshot process can be started: ●

You must know the name of the volume to be backed up.

●

You must provide a name for the new temporary snapshot volume.

●

You can specify a specific disk drive to use for the snapshot copy.

●

You must have sufficient unused disk space for the snapshot.

The snapshot process is a two-step process. First, a snapshot mirror of a volume is created. If no disk drives are specified for use by the snapshot operation, VxVM finds free disk space anywhere it can. The resulting snapshot mirror can be poorly constructed. Because the snapshot mirror is only a temporary structure, this might not be a problem. However, it could temporarily interfere with the performance of other volumes. When the snapshot mirror is fully synchronized, its state changes to SNAPDONE. The snapshot mirror continues to maintain synchronization with the parent volume until it is detached. The final operation detaches the temporary snapshot mirror and attaches it to a regular volume with a name of your choosing. You can then perform standard backup procedures on the snapshot volume.

6-38

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations

Using the VEA GUI to Create a Snapshot You can use the snapshot feature from the VEA GUI to create a snapshot. Complete the following steps: 1.

2.

In the grid area, complete the following steps: a.

Click on the volume with the third mouse button.

b.

Select Snap, and then Snap Start from its pop-up menu.

Complete the Snap Start Volume form, as shown in Figure 6-8. It is usually safe to let VxVM decide which disks to use for the Snapshot mirror. The process of creating and synchronizing a new mirror with the existing volume data can take some time depending on the volume size.

Figure 6-8

VEA Snap Start Volume Form

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-39

Volume Snapshot Operations 3.

In the grid area, complete the following steps: a.

Click the volume name with the third mouse button.

b.

Select Snap, and then Snap Shot from its pop-up menu.

c.

Complete the Snap Shot Volume form as shown in Figure 6-9. The new mirror is detached and a separate volume is created from it based on a default naming scheme. You can change the default snapshot name if needed.

Figure 6-9

6-40

VEA Snap Shot Volume Form

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations 4.

Use the vxprint command to verify the status of the new snapshot volume. In the following case, the snapshot mirror/volume is named SNAP-stdemo. The parent volume is named stdemo.

# vxprint -g dgX TY NAME ASSOC dg dgX dgX

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm dm dm dm dm dm dm dm dm dm dm dm

dgX01 dgX02 dgX03 dgX04 dgX05 dgX06 dgX07 dgX08 dgX09 dgX10 dgX11 dgX12

c2t16d0s2 c2t18d0s2 c2t1d0s2 c2t3d0s2 c2t5d0s2 c2t20d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2 c3t37d0s2 c3t50d0s2 c3t52d0s2

-

17674896 17674896 17674896 17674896 17674896 17674896 17674896 17674896 17674896 17674896 17674896 17674896

-

NOHOTUSE -

v pl sd sd sd sd sd

SNAP-stdemo stdemo-02 dgX06-01 dgX07-01 dgX08-01 dgX09-01 dgX10-01

fsgen SNAP-stdemo stdemo-02 stdemo-02 stdemo-02 stdemo-02 stdemo-02

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 12288 12288 12288 12288 12288

0 0 0 0 0

ACTIVE ACTIVE -

v pl sd sd sd sd sd

r5demo r5demo-01 dgX03-01 dgX04-01 dgX05-01 dgX01-01 dgX02-01

raid5 r5demo r5demo-01 r5demo-01 r5demo-01 r5demo-01 r5demo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 15360 15360 15360 15360 15360

0 0 0 0 0

ACTIVE ACTIVE -

v pl sd sd sd sd sd

stdemo stdemo-01 dgX03-02 dgX04-02 dgX05-02 dgX01-02 dgX02-02

fsgen stdemo stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo-01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 12288 12288 12288 12288 12288

0 0 0 0 0

ACTIVE ACTIVE -

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-41

Volume Snapshot Operations 5.

Create a temporary mount point, and complete the following steps: a.

Mount the new snapshot volume.

b.

Back it up to tape.

# mkdir /Temp # mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp # tar tvf /dev/rmt/0 /Temp 6.

Unmount and delete the snapshot volume. # umount /Temp # vxedit -g dgX -rf rm SNAP-stdemo The VEA GUI volume snapshot menu has additional functions, Snap Back and Snap Clear. As shown in Figure 6-10, the Snap Back function rejoins the snapshot mirror to the original volume. It also resynchronizes the data in either direction between the original volume and the snapshot mirror.

Figure 6-10 VEA Snap Back Volume Form The Snap Clear function disassociates the detached snapshot mirror from the original volume. The Snap Back feature cannot be used after the Snap Clear operation has completed.

6-42

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Volume Snapshot Operations

Using the Command Line to Create a Snapshot Using the command line to make snapshots is useful for performing backups from script files. The following example shows the command-line snapshot process. 1.

Create a snapshot of the stdemo volume and verify it has completed.

# vxassist -g dgX -b snapstart stdemo # vxprint -g dgX stdemo ASSOC KSTATE LENGTH TY NAME v pl sd sd sd sd sd pl sd sd sd sd sd

stdemo stdemo-01 dgX03-02 dgX04-02 dgX05-02 dgX01-02 dgX02-02 stdemo-02 dgX06-01 dgX07-01 dgX08-01 dgX09-01 dgX10-01

fsgen stdemo stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo stdemo-02 stdemo-02 stdemo-02 stdemo-02 stdemo-02

2.

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

61440 61440 12288 12288 12288 12288 12288 61440 12288 12288 12288 12288 12288

PLOFFS 0 0 0 0 0 0 0 0 0 0

STATE ACTIVE ACTIVE SNAPDONE -

Use the vxassist snapshot option to detach the temporary snapshot mirror (plex) and associate it with a new volume named SNAP-stdemo.

# vxassist -g dgX snapshot stdemo-02 SNAP-stdemo 3.

Process the volume with the fsck utility to check its integrity, and complete the following steps: a.

Mount the volume.

b.

Perform a backup of the snapshot volume.

c.

Unmount and delete the volume.

# fsck -y /dev/vx/rdsk/dgX/SNAP-stdemo # mkdir /Temp # mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp # tar cvf /dev/rmt/0 /Temp # umount /Temp # vxedit -g dgX -rf rm SNAP-stdemo

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-43

Online Volume Relayout

Online Volume Relayout Online volume relayout is a tool that you can use to correct configuration mistakes or to enhance the performance or reliability of a volume. An important feature of online volume relayout is that a volume and its file system can remain available during the relayout operation. The relayout feature can be used to perform many operations, such as: ●

Adding more stripe columns to a RAID-5 volume

●

Changing the stripe unit size of a volume

●

Changing the type of volume from RAID 5 to mirrored or concatenated

Note – Read the “Changing a VxVM Volume Layout” section of the VERITAS Enterprise Administrator User’s Guide carefully before attempting a relayout of a production volume.

Volume Relayout Prerequisites You must research the following information before starting an online volume relayout process: ●

Identify the new volume layout. This includes concatenated, striped, RAID 5, concatenated mirror, striped mirror, additional columns, and new stripe widths.

●

Research additional permanent disk space that might be needed by the new volume layout.

●

Research the temporary disk space that might be needed during the volume layout.

Note – You can also use the vxassist relayout command to accomplish online volume relayout. Unless you explicity allocate storage, the vxassist command automatically determines where to get the permanent and temporary disk space that might be needed.

6-44

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Online Volume Relayout

Volume Relayout Using the Command Line The following examples show the basic process to add an extra column to an existing RAID-5 volume name r5demo. There is enough additional disk space on the current disk drives being used by the RAID-5 volume for any temporary space needs. For performance reasons, you should specify which disk drive to use for the new column. # vxprint -g dgX r5demo stdemo TY NAME ASSOC KSTATE v r5demo raid5 ENABLED pl r5demo-01 r5demo ENABLED sd dgX03-01 r5demo-01 ENABLED sd dgX04-01 r5demo-01 ENABLED sd dgX05-01 r5demo-01 ENABLED sd dgX01-01 r5demo-01 ENABLED sd dgX02-01 r5demo-01 ENABLED

LENGTH 61440 61440 15360 15360 15360 15360 15360

PLOFFS 0 0 0 0 0

STATE ACTIVE ACTIVE -

v pl sd sd sd sd sd

61440 61440 12288 12288 12288 12288 12288

0 0 0 0 0

ACTIVE ACTIVE -

stdemo stdemo-01 dgX03-02 dgX04-02 dgX05-02 dgX01-02 dgX02-02

fsgen stdemo stdemo-01 stdemo-01 stdemo-01 stdemo-01 stdemo-01

# vxdg -g dgX free DISK DEVICE dgX01 c2t16d0s2 dgX02 c2t18d0s2 dgX03 c2t1d0s2 dgX04 c2t3d0s2 dgX05 c2t5d0s2 dgX06 c2t20d0s2

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

TAG c2t16d0 c2t18d0 c2t1d0 c2t3d0 c2t5d0 c2t20d0

OFFSET 27648 27648 27648 27648 27648 0

LENGTH 17647248 17647248 17647248 17647248 17647248 17674896

FLAGS n -

The RAID 5 and RAID 0 stripes are using a total of 27,648 blocks on each disk drive or about 14 Mbytes. All disk drives in the disk group have more than enough space left for use during the relayout operations. The following example adds an additional column to the RAID-5 volume and specifies that the new column is to be on the disk dgX06. # vxassist -g dgX relayout r5demo layout=raid5 alloc="dgX06” ncol=6

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-45

Online Volume Relayout

Completed RAID 5 Volume Relayout After the completion of the previous volume relayout example, the RAID-5 volume now has the following configuration. # vxprint -g dgX r5demo TY NAME ASSOC v r5demo raid5 pl r5demo-Dp02 r5demo sd dgX06-03 r5demo-Dp02 pl r5demo-01 r5demo sd dgX03-04 r5demo-01 sd dgX04-03 r5demo-01 sd dgX05-03 r5demo-01 sd dgX01-03 r5demo-01 sd dgX02-03 r5demo-01 sd dgX06-02 r5demo-01

KSTATE ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

LENGTH 61440 1920 1920 61440 12288 12288 12288 12288 12288 12288

PLOFFS 0 0 0 0 0 0 0

STATE ACTIVE LOG ACTIVE -

By default, the vxassist layout=raid5 specification adds a log to the RAID-5 volume. Due to performance degradation, RAID-5 volume logs should not reside on the same disk drives as the RAID-5 data. To prevent the unintentional addition of a RAID-5 log, the vxassist layout specification must be more specific. # vxassist -g dgX relayout r5demo layout=raid5,nolog alloc="dgX06” ncol=6

6-46

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Online Volume Relayout

Volume Relayout Using the VEA GUI To perform the same process as in the previous section, complete the following steps: 1.

Display the RAID-5 volume in the grid area, and select Change Layout from its pop-up menu.

2.

Complete the Change Volume Layout form, as shown in Figure 6-11.

Figure 6-11 VEA Change Volume Layout Form In the preceding example, the volume Layout was set to RAID 5 and the Number of Columns was increased to 6. Use the Show Options button to enter additional relayout criteria, such as which disks drives to use.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-47

Online Volume Relayout

Relayout Status Monitor After you fill out the Change Volume Layout form and start the relayout process, a Relayout Status window, shown in Figure 6-12, appears. You use the controls in the Relayout Status window to: ●

Temporarily stop the relayout process (pause)

●

Abort the relayout process

●

Continue the process after a pause

●

Undo the relayout changes (reverse)

The VEA Relayout Status Monitor window also displays the percentage complete status.

Figure 6-12 VEA Relayout Status Monitor Window

Note – The relayout task could fail if the target volume was not originally created using the VEA GUI or the vxassist command.

6-48

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Creating Layered Volumes A layered volume is built on one or more other volumes. The underlying volumes are mirrored. Layered volume are RAID 1+0 format and are called striped mirror or concatenated mirror configurations.

Layered Volume Disk Requirements By default, the mirrors in layered volumes are created with DRLs. For best performance and reliability, all components of a layered volume should be on different disk drives. A striped mirror volume, shown in Figure 6-13, requires a minimum of four disk drives to implement. Placing the DRLs on separate disk drives increases the disk requirement to six. Volume Stripe

Log

Mirror

Stripe

Mirror

Log

Mirror

Mirror

Figure 6-13 Striped Mirror Disk Requirements The disk requirements for a concatenated mirror volume structure, shown in Figure 6-14, are the same as for striped mirror structures. Volume Subdisk

Mirror

Mirror

Log

Mirror

Mirror

Log

Subdisk

Figure 6-14 Concatenated Mirror Disk Requirements

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-49

Creating Layered Volumes

Evaluating Available Disk Space There are several methods available for evaluating available disk space. The vxdg free command is a good tool for performing a basic evaluation. In the following example, each of the disk drives in the dgX disk group have approximately 8 Gbytes of available space. To calculate the available free space in Mbytes on a given disk drive, divide its LENGTH value by 2048 (17674896/2048 = 8630.32 Mbytes = 8.63 Gbytes). # vxdg -g dgX free DISK DEVICE dgX01 c2t16d0s2 dgX02 c2t18d0s2 dgX03 c2t1d0s2 dgX04 c2t3d0s2 dgX05 c2t5d0s2 dgX06 c2t20d0s2

TAG c2t16d0 c2t18d0 c2t1d0 c2t3d0 c2t5d0 c2t20d0

OFFSET 0 0 0 0 0 0

LENGTH 17674896 17674896 17674896 17674896 17674896 17674896

FLAGS n -

You can also use the vxassist maxsize option as follows to calculate maximum available space for a specific structure. The example defaults to two mirrors and logging. # vxassist -g dgX maxsize \ layout=stripe-mirror ncolumn=2 \ dgX01 dgX02 dgX03 dgX04 dgX05 dgX06 Maximum volume size: 35348480 (17260Mb)

Note – If you use the VEA GUI New Volume wizard to configure layered volumes, you can use the Max Size button to estimate maximum available space.

6-50

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Creating Layered Volumes From the Command Line If you create layered volumes using the vxassist command, you should specify the layout, number of columns, and whether logging is to be configured. The default is to not create DRLs. An example of using the vxassist command to create a layered volume follows. # vxassist -g dgX make strmirr 500m \ layout=stripe-mirror ncolumn=2 \ dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

Use the vxprint command as follows to display the layered volume structure. Output not related to the volume is omitted for clarity. # vxprint -g dgX ... ... v strmirr fsgen pl strmirr-03 strmirr sv strmirr-S01 strmirr-03 sv strmirr-S02 strmirr-03

ENABLED ENABLED ENABLED ENABLED

1024000 1024000 512000 512000

0 0

ACTIVE ACTIVE -

v pl sd pl sd

strmirr-L01 strmirr-P01 dgX03-02 strmirr-P02 dgX05-02

fsgen strmirr-L01 strmirr-P01 strmirr-L01 strmirr-P02

ENABLED ENABLED ENABLED ENABLED ENABLED

512000 512000 512000 512000 512000

0 0

ACTIVE ACTIVE ACTIVE -

v pl sd pl sd

strmirr-L02 strmirr-P03 dgX04-02 strmirr-P04 dgX01-02

fsgen strmirr-L02 strmirr-P03 strmirr-L02 strmirr-P04

ENABLED ENABLED ENABLED ENABLED ENABLED

512000 512000 512000 512000 512000

0 0

ACTIVE ACTIVE ACTIVE -

Note – Layered volume structures are difficult to understand in the vxprint output format. Use the VEA GUI Layout View function to see a visual representation of layered volume structures.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-51

Creating Layered Volumes

Creating Layered Volumes Using the VEA GUI Use the VEA GUI New Volume wizard, show in Figure 6-15, to configure layered volumes. Select either Concatenated Mirrored or Striped Mirrored in the Layout section. By default, the New Volume wizard configures two columns, two mirrors for each column, and enables logging (DRLs).

Figure 6-15 Creating Layered Volumes Using the VEA GUI

6-52

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Layered Volumes

Identifying Layered Volume Subcomponents The striped mirror volume components, shown in Figure 6-16, consist of several layers. The lower levels of the layered volumes are ready-made configurations designed to provide the highest level of availability without increasing the administrative complexity.

Volume vol01 10 GB

Striped Plex vol01-03 10 GB Col/Width:2x64

Subdisk vol01-S01 Column: 0

Sub-Volume vol01-L01 5 GB

Subdisk vol01-S02 Column: 1

Sub-Volume vol01-L02 5 GB

Sub-Plex vol01-P01 5 GB

Subdisk disk01-01 5 GB

Sub-Plex vol01-P02 5 GB

Subdisk disk02-01 5 GB

Sub-Plex vol01-P03 5 GB

Subdisk disk03-01 5 GB

Sub-Plex vol01-P04 5 GB

Subdisk disk04-01 5 GB

Figure 6-16 Striped Mirror Volume Structure The concatenated mirror volume components shown in Figure 6-17 also consist of several layers.

Volume vol01 10.5 GB

Concat Plex vol01-03 10.5 GB

Subdisk vol01-S01 1.6 GB

Sub-Volume vol01-L01 1.6 GB

Subdisk vol01-S02 8.9 GB

Sub-Volume vol01-L02 8.9 GB

Sub-Plex vol01-P01 1.6 GB

Subdisk disk01-01 1.6 GB

Sub-Plex vol01-P02 1.6 GB

Subdisk disk02-01 1.6 GB

Sub-Plex vol01-P03 8.9 GB

Subdisk disk03-01 8.9 GB

Sub-Plex vol01-P04 8.9 GB

Subdisk disk04-01 8.9 GB

Figure 6-17 Concatenated Mirror Volume Structure

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-53

Basic Intelligent Storage Provisioning Administration

Basic Intelligent Storage Provisioning Administration The VxVM Intelligent Storage Provision (ISP) feature is specifically targeted at managing large storage installations composed of a huge number of hardware RAID LUNs or SAN devices.

Primary ISP Components Traditional VxVM volume creation, using software such as the vxassist command or the VEA GUI, relies on the administrator. The administrator must analyze storage resources and configure volumes in a manner that satisfies the required performance or reliability needs. With the advent of hardware RAID LUN technology, such as the Sun StorEdge 3510/6020/9910 models and related SAN technology, a system administrator might be faced with analyzing thousands of devices whose underlying characteristics and capabilities are hidden and unknown. The ISP system provides a storage allocation engine that automatically chooses which designated storage to use based on the capabilities you specify when creating new ISP volumes (called application volumes). The components, commands, volumes, and storage designated for ISP use cannot be used by traditional VxVM commands such as vxassist, vxdiskadm, and vxvol. Basic ISP components include the following: ●

Data pools (only for data volumes)

●

Clone pools (only for snapshots)

●

Application volumes

●

Pre-defined templates

All ISP operations are performed using the vxvoladm command, the vxpool command, or the VEA GUI. There are no other commands or tools used to create ISP pools or application volumes.

6-54

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

ISP Storage Pools You configure both data pools and clone pools within a disk group. A data storage pool is created within a standard disk group. One or more LUNs from a disk group are assigned to a named storage pool. Any subsequent storage pools created in the same disk group are automatically defined as clone pools. Clone pools are used only to hold full-sized instant snapshots of data pool volumes in the same disk group. If the instant snapshot feature is not licensed on your system, clone pools have no use.

ISP Application Volumes Application volumes reside only in an ISP data storage pool. You create application volumes using either the vxvoladm command or VEA GUI.

Predefined ISP Templates You associate ISP templates with storage pools so that volumes created in a storage pool are restrained by a fixed set of configuration rules. The hierarchy of ISP templates is shown in Figure 6-18. Storage Pool Template Sets (6)

Policies

2 Each Storage Pool Templates (11)

autogrow selfsufficient

5-15 Each Volume Templates (21) 1-2 Each Capabilities (25) 0-2 Each Variables (8)

Figure 6-18 ISP Template Hierarchy

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-55

Basic Intelligent Storage Provisioning Administration

Using Storage Pool Set Templates Storage pool set templates describe both data pool and clone pool characteristics within a disk group. There are six pre-defined storage pool sets. By default, the first pool created in a disk group is a data pool and all subsequent pools created in a disk group are clone pools. Each storage pool set provides two storage pool definitions. For example, the storage pool set, mirrored_data_striped_clones, provides the mirrored_volumes storage pool definition for the data pool and the striped_volumes storage pool definition for the clone pool.

Creating Storage Pool Sets Using the VEA GUI As shown in Figure 6-19, the VEA GUI Organize Disk Group Wizard organizes existing disk groups using one of the pre-defined storage pool set templates. Default data pool and clone pool names can be modified. You assign disk group disks to the pools afterwards.

Figure 6-19 Selecting Storage Pool Sets

6-56

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Creating Storage Pool Sets Using the vxpool Command The vxpool command associates a storage pool set template with an existing disk group. In the following example, the storage pool set template mirrored_data_striped_clones is associated with the dgSP disk group. # vxpool listpoolsets mirrored_data_striped_clones mirrored_prefab_raid5_data_mirrored_clones mirrored_prefab_stripe_data_striped_clones prefab_mirrored_data_prefab_striped_clones stripe_mirrored_data_striped_clones striped_prefab_mirrored_data_striped_clones # vxpool -g dgSP organize mirrored_data_striped_clones By default, the data and clone pool names are a variation of the pool templates, mirrored_volumes and striped_volumes. After initial pool creation, you use the vxpool command to associate disk group media names with each pool. You can also modify the default storage pool names. An example follows. # vxpool -g dgSP adddisk mirrored_volumes1 \ dm=dgSP01,dgSP03,dgSP04,dgSP02,dgSP07,dgSP06 # vxpool -g dgSP adddisk striped_volumes1 \ dm=dgSP08,dgSP12,dgSP13,dgSP09,dgSP10,dgSP11 # vxpool -g newDG2 # vxpool -g newDG2

rename mirrored_volumes1 dp_01 rename striped_volumes1 cp_01

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-57

Basic Intelligent Storage Provisioning Administration

Using Storage Pool Templates When first created, data pools or clone pools are associated with a single storage pool template such as mirrored_volumes or raid5_volumes. Each storage pool template references a list of predefined volume templates. You can create storage pools without assigning a template, and later use the vxpool command to associate a particular template. Each storage pool template references 5-15 volume templates. The following example shows the use of the vxpool command to create a storage pool, assign disks to it, and associate a template with the pool. The first pool created in a disk group is automatically a data pool. # vxdisk -g dgX list DEVICE TYPE c2t1d0s2 auto:cdsdisk c2t3d0s2 auto:cdsdisk c2t5d0s2 auto:cdsdisk c2t16d0s2 auto:cdsdisk c2t18d0s2 auto:cdsdisk c3t32d0s2 auto:cdsdisk c3t33d0s2 auto:cdsdisk c3t35d0s2 auto:cdsdisk c3t37d0s2 auto:cdsdisk c3t50d0s2 auto:cdsdisk

DISK dgX03 dgX04 dgX05 dgX01 dgX02 dgX06 dgX07 dgX08 dgX09 dgX10

GROUP dgX dgX dgX dgX dgX dgX dgX dgX dgX dgX

STATUS online online online online online online online online online online

# vxpool listpooldefinitions any_volume_type mirror_stripe_volumes mirrored_prefab_raid5_volumes mirrored_prefab_striped_volumes mirrored_volumes prefab_mirrored_volumes prefab_raid5_volumes prefab_striped_volumes raid5_volumes stripe_mirror_volumes striped_prefab_mirrored_volumes striped_volumes # vxpool -g dgX create r5pool \ dm=dgX01,dgX02,dgX03,dgX06,dgX07,dgX08 \ pooldefinition=raid5_volumes

6-58

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration # vxpool -g dgX list r5pool

# vxpool -g dgX print TY NAME TYPE st r5pool data dm dgX08 dm dgX07 dm dgX06 dm dgX03 dm dgX02 dm dgX01 -

AUTOGROW diskgroup -

SELFSUFFICIENT pool -

DESCRIPTION Volume has parity -

By default, the autogrow policy for pools is set to 2 (diskgroup). The pool can be grown by bringing in additional storage from the disk group outside of the storage pool. By default, the selfsufficient policy is set to level 1 (pool). It only allows the use of templates that have been manually assigned to the storage pool. Storage pool attributes can be modified after initial pool creation. See the vxpool man page for more details. The vxprint command provides a method for evaluating current storage pools. In the following example, a disk group named dgX, containing 10 disk drives, has a data pool named r5pool that uses 6 of the disk group’s drives. # vxprint -g dgX TY NAME ASSOC dg dgX dgX

KSTATE -

LENGTH -

PLOFFS -

STATE ALLOC_SUP

dm dm dm dm

dgX04 dgX05 dgX09 dgX10

c2t3d0s2 c2t5d0s2 c3t37d0s2 c3t50d0s2

-

17679776 17679776 17679776 17679776

-

-

st dm dm dm dm dm dm

r5pool dgX01 dgX02 dgX03 dgX06 dgX07 dgX08

c2t16d0s2 c2t18d0s2 c2t1d0s2 c3t32d0s2 c3t33d0s2 c3t35d0s2

-

17679776 17679776 17679776 17679776 17679776 17679776

-

DATA -

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-59

Basic Intelligent Storage Provisioning Administration

Using Application Volume Templates There are currently 21 predefined application volume templates. Each storage pool template references 5-15 volume templates. Each predefined volume template references one or two volume capabilities. For instance, the storage pool template raid5_volumes refers to a set of ten volume templates. Most of the volume templates contain the name of a single capability. Many of the capabilities use the same name as the volume templates. The capabilities define one or more variables that can be used when creating application volumes. Many of the variables have default values. The following are capabilities for the Raid5Volume template. Volume Template: Raid5Volume Provides capabilities: Raid5Capability, Raid5LogMirroring Variables:

ncols

Minimum number of columns

nlogs

Number of logs

nmaxcols

Maximum number of columns

The Raid5Capability capability provides the ncols and nmaxcols variables which define the minimum and maximum number of columns to create for a RAID-5 volume. The default value for ncols is 8 and for nmaxcols is 20. The Raid5LogMirroring capability provides the nlogs variable which defines the number of logs to create for a RAID-5 volume. The default value is 1. Another volume template named LogsOnSeparateComponents has the following capabilities: Volume Template: LogsOnSeparateComponents Provides capabilities: LogsOnSeparateComponents Variables:

component Name of component

The component variable can be set to Controller or Enclosure to separate multiple logs. The default is Enclosure.

6-60

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Deriving Volume Template Capabilities Table 6-2 shows capabilities resulting from applying the mirrored_volumes storage pool template to a disk group. Many of the variables have default values. Table 6-2

Volume Capability Derivation Storage Pool: mirrored_volumes

Associate Volume Templates

Capability Name

ArrayProductId

Same name

Volume uses storage with the same productid

ConfineLogs ToSimilarStorage

Same name

Logs confined to same storage by variable: name default: Enclosure

ConfineMirrors ToSimilarStorage

Same name

Mirrors confined to same storage by variable: name default: Enclosure

Confine ToSimilarStorage

Same name

Volumes confined to same storage by variable: name default: VendorName

Confine ToSpecificStorage

Same name

Volume confined to specific storage by variable: name (no default) variable: value (no default)

DataMirroring

Same name

Number of mirror set by variable: nmirs default: 2

DCOLogMirroring

Same name

Number of DCO log mirrors set by variable: nlogs default:1

InstantSnapshottable

Same name

This volume supports instant snapshots.

LogsOn SeparateComponents

Same name

Separate volume logs by component. variable: component default: Enclosure

MirrorsOn SeparateComponents

Same name

Separate volume mirrors by component. variable: component default:Enclosure

MultipathingThrough MultiplePaths

Same name

Number of paths that can fail. variable: npaths default=2

Capability Variables

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-61

Basic Intelligent Storage Provisioning Administration

Creating Application Volumes Using the vxvoladm Command You use the vxvoladm command to create application volumes. The following example shows the use of the vxvoladm command to create a default RAID-5 volume in a storage pool named r5pool. # vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume # vxprint -g dgX r5vol TY NAME ASSOC v r5vol raid5 pl r5vol-01 r5vol sd dgX03-01 r5vol-01 sd dgX05-01 r5vol-01 sd dgX04-01 r5vol-01 sd dgX02-01 r5vol-01 sd dgX06-01 r5vol-01 sd dgX08-01 r5vol-01 sd dgX07-01 r5vol-01 sd dgX09-01 r5vol-01 sd dgX10-01 r5vol-01 pl r5vol-02 r5vol sd dgX01-01 r5vol-02

KSTATE ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

LENGTH 20480 20480 2560 2560 2560 2560 2560 2560 2560 2560 2560 8640 8640

PLOFFS 0 0 0 0 0 0 0 0 0 0

STATE ACTIVE ACTIVE LOG -

The initial results are unexpected because the autogrow policy allowed the use of disk group disks outside of the pool and the default for the nmaxcols variable is 20. The result is a 9 column RAID-5 volume that uses all of the disk group disks, even those outside of the defined pool. # vxvoladm -g dgX remove volume r5vol When the r5vol volume is removed, the extra disks are automatically removed from the r5pool. # vxpool -g dgX getpolicy r5pool AUTOGROW SELFSUFFICIENT diskgroup pool # vxpool -g dgX setpolicy r5pool autogrow=pool # vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume \ capability=’Raid5Capability(nmaxcols=4)’

6-62

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Creating Application Volumes Using the VEA GUI You create application volume using the VEA GUI by clicking a disk group that has storage pools configured and selecting New Volume from its pop-up menu. When you create a new volume in a disk group that contains configured storage pools, the New Volume Wizard is aware that a storage pool exists. It automatically displays all possible volume configuration capabilities based on templates associated with the data pool in the disk group. Initially, none of the capabilities are enabled. If you do not enable any of the capabilities, the completed volume is a simple concatenation.

Selecting Volume Capabilities As shown in Figure 6-20, some capabilities display variables, such as number of mirrors, that you can modify if needed.

Figure 6-20 Selecting Volume Capabilities

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-63

Basic Intelligent Storage Provisioning Administration

Saving Custom Volume Templates The next application volume window display, shown in Figure 6-21, summarizes your capability choices and offers the opportunity to save the capability configuration as a custom volume template. If you save a custom template, the next time you create a volume, you are offered the opportunity to use your custom template and bypass the manual capability process.

Figure 6-21 Saving Custom Volume Templates

6-64

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration

Interpreting Application Volume Configurations As shown in the following example, disk group disks, pools, and their related application volumes are grouped together by the vxprint command. # vxprint -g dgSP TY NAME ASSOC dg dgSP dgSP

KSTATE -

LENGTH -

dm dgSP05

c2t22d0s2

-

17679776 -

-

st dm dm dm dm dm dm

mirrored_volumes1 dgSP01 c2t16d0s2 dgSP02 c2t18d0s2 dgSP03 c2t1d0s2 dgSP04 c2t20d0s2 dgSP06 c2t3d0s2 dgSP07 c2t5d0s2

-

17679776 17679776 17679776 17679776 17679776 17679776

DATA -

v pl sd pl sd

datapool_vol01 fsgen ENABLED 2097152 datapool_vol01-01 datapool_vol01 ENABLED 2097152 dgSP04-01 datapool_vol01-01 ENABLED 2097152 0 datapool_vol01-02 datapool_vol01 ENABLED 2097152 dgSP01-01 datapool_vol01-02 ENABLED 2097152 0

ACTIVE ACTIVE ACTIVE -

st dm dm dm dm dm dm

striped_volumes1 dgSP08 c3t32d0s2 dgSP09 c3t33d0s2 dgSP10 c3t35d0s2 dgSP11 c3t37d0s2 dgSP12 c3t50d0s2 dgSP13 c3t52d0s2

CLONE -

-

17679776 17679776 17679776 17679776 17679776 17679776

PLOFFS -

-

-

STATE ALLOC_SUP

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-65

Replacing Failed Disk Drives

Replacing Failed Disk Drives One of the most common VxVM errors that you might see is detached plex messages. This error message usually means that one of the mirrors in a volume has encountered a disk drive error while being accessed. You must identify the physical path to the failed disk drive before you proceed. The most common tools you use to do this are: ●

The vxprint command

●

The vxdisk command

●

The /var/adm/messages file

Failure Behavior A plex (mirror) is detached if a persistent I/O error is encountered. There are several things to be aware of before proceeding: ●

Disk block read errors might affect one subdisk, while other subdisks on the same physical disk drive remain functional.

●

Errors are not detected until read or write operations are attempted.

●

Severe disk drive errors, such as general access failures, result in relocation of all redundant subdisks associated with the failed disk. They are relocated to either designated hot spare disks or to any disk that does not have the nohotuse flag set.

Hardware RAID Behavior Hardware RAID storage units, such as the Sun StorEdge T3 array, present LUNs to attached systems. Each LUN is actually a portion of a hardware RAID structure that is monitored internally in the storage array for disk failures. Typically, the internal RAID volumes are redundant, such as RAID 5 or RAID 1, and the storage array internally relocates the failing data to a designated spare drive. Hardware RAID internal failures are usually transparent to the VxVM software. Hardware RAID storage devices usually notify the user root mail account of internally detected problems.

6-66

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives

Evaluating Failure Severity By default, the VxVM software sends failure notification email to the user root account. The extent of failures must be evaluated in a timely manner. It is important to assess failures in a timely manner before a problem escalates to a much worse situation. The first line of defence is to examine system error logs and verify volume status.

Use root Email Notification Typically, a series of root account email error notifications follow a device failure. After the initial error notification, several more message are usually sent as VxVM attempts to relocate failed volume components. The following example shows an initial error notification. # mail ? From [email protected] Fri Nov 7 21:11:59 2003 Date: Fri, 7 Nov 2003 21:11:58 -0500 (EST) From: Super-User Message-Id: <[email protected]> To: [email protected] Subject: Volume Manager failures on host ns-east-104 Content-Length: 239 Failures have been detected by the VERITAS Volume Manager: failed disks: dgX04 failed plexes: mirvol_01-02 The Volume Manager will attempt to find spare disks, relocate failed subdisks and then recover the data in the failed plexes.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-67

Replacing Failed Disk Drives

Examining Errors in the /var/adm/messages File The following messages are seen when a disk drive is experiencing access problems, such as hard write errors. Nov 7 21:11:40 ns-east-104 vxio: [ID 245403 kern.warning] WARNING: VxVM vxio V-5-0-151 error on Plex mirvol_01-02 while writing volume mirvol_01 offset 16 length 4 Nov 7 21:11:40 ns-east-104 vxio: [ID 786473 kern.warning] WARNING: VxVM vxio V-5-0-4 Plex mirvol_01-02 detached from volume mirvol_01 Nov 7 21:11:40 ns-east-104 vxio: [ID 628984 kern.warning] WARNING: VxVM vxio V-5-0-386 dgX04-01 Subdisk failed in plex mirvol_01-02 in vol mirvol_01 Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 976563 daemon.notice] V5-1-768 Offlining config copy 1 on disk c2t3d0s2: Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 672837 daemon.notice] Reason: Disk write failure Nov 7 21:11:41 ns-east-104 vxvm:vxconfigd: [ID 905431 daemon.notice] V5-1-7909 Detached disk dgX04

Checking Volume Status The vxprint command is the easiest way to check the status of all volume structures. In the following excerpt, the status of two plexes in a volume is bad. One of the plexes is a log. # vxprint Disk group: sdg0 TY NAME dg sdg0

ASSOC sdg0

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm disk0 dm disk7

c4t0d0s2 c5t0d0s2

-

8368512 8368512

-

-

v pl sd pl sd pl sd

fsgen vol0 vol0-01 vol0 vol0-02 vol0 vol0-03

ENABLED DISABLED ENABLED ENABLED ENABLED DISABLED ENABLED

524288 525141 525141 525141 525141 LOGONLY 5

0 0 LOG

ACTIVE IOFAIL ACTIVE IOFAIL -

6-68

vol0 vol0-01 disk0-01 vol0-02 disk7-01 vol0-03 disk0-02

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives

Note – If a hot spare is available and the volume is redundant, the structure is relocated to a different disk drive and you never see such volume states. When VxVM detects a disk drive failure, it can place a failed plex in a number of different states. The two most common states for a failed plex are: ●

DETACHED/IOFAIL

●

DISABLED/NODEVICE

The DETACHED/IOFAIL state is not so severe. It is relatively easy to identify the failed device path by examining the vxprint -ht command output as follows: # vxprint -ht | grep sd0b-01 sd sd0b-01 lv0b-01 disk7 0

525141

0

c5t0d0

ENA

In the previous example, the VxVM media name is disk7 and the physical path is c5t0d0. When the VxVM software loses complete contact with a disk drive, the physical path in the vxprint -ht command’s output might be blank. At those times, you must determine the media name of the failed disk drive from the vxprint command, and then use the vxdisk list command to associate the media name with the physical device. # vxdisk list DEVICE TYPE c0t0d0s2 auto:sliced c0t1d0s2 auto:sliced c2t1d0s2 auto:sliced c2t3d0s2 auto:sliced c2t5d0s2 auto:sliced c2t16d0s2 auto:sliced c2t18d0s2 auto:sliced c2t20d0s2 auto:sliced dgX04

DISK rootdg01 rootdg02 dgX03 dgX05 dgX01 dgX02 dgX06 dgX

GROUP STATUS rootdg online rootdg online nohotuse dgX online online dgX online dgX online dgX online dgX online failed nohotuse was:c2t3d0s2

When a disk drive fails and becomes detached, the VxVM software cannot find the disk drive, but it still knows the physical path. This information is the origin of the failed was status. This status means that the disk drive has failed and that the physical path is the value displayed in the STATUS column.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-69

Replacing Failed Disk Drives

General Disk Drive Replacement Process When a disk drive containing redundant volumes fails, usually all of the subdisks are relocated to a hot spare and the redundant volume continues to function normally. Subdisk relocation messages are sent to the user root email account. The VxVM software requires that a specific recovery process is followed to replace the failed disk drive, and then relocate the subdisks back to the new disk drive. Some storage arrays also require that specific disk replacement processes are followed. Disk drive failures in hardware RAID storage, such as the Sun StorEdge T3 arrays, are frequently transparent to VxVM. This is because their internal LUN structures are redundant and an internal spare is automatically substituted for the failed drive. The overall replacement process is typically a mixture of software and hardware replacement procedures. Following is a summary of the process. 1.

If appropriate, try to resolve temporary problems before taking more complex action. If the disk drive is detached due to a secondary problem, such as a loose cable, you can try to reattach it using the vxreattach command. Reattachment might resolve all problems. If you feel the disk is still at risk, you can evacuate it and perform a hardware replacement.

2.

Remove the failed disk drive from VxVM control. Use the vxdiskadm utility option 4, Remove a disk for replacement, to remove the physical disk from its disk group while retaining its media name. If you intend to hot-swap the failed disk without rebooting the system, you might also use the vxdiskadm utility option 11, Disable (offline) a disk device, to stop all VxVM access, such as polling.

3.

Perform the storage-specific hardware replacement procedure. The hardware replacement procedures vary widely depending on the storage type and model. Always consult the technical manuals for your particular array model.

6-70

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Replacing Failed Disk Drives 4.

Return the failed disk drive to VxVM control. Use the vxdiskadm utility option 10, Enable (online) a disk device, to enable VxVM access to the replacement disk. Use the vxdiskadm utility option 5, Replace a failed or removed disk, to associate the new disk with its media name. You initially select None as a replacement device. You are then asked if you want to initialize the new disk drive.

Note – In some cases you might need to scan for new disk drives using either the vxdiskconfig or vxdctl enable command. 5.

Use the vxdiskadm utility option 14, Unrelocate subdisks back to a disk, to move the relocated volume components back to the original disk location. The only information you furnish is the media name of the replacement disk, VxVM.

Note – If a failed volume is non-redundant, you cannot recover it. You must manually recreate the volume structure, using tools such as vxconfigrestore, and restore the data from backup tapes.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-71

Exercise: Performing Advanced Operations

Exercise: Performing Advanced Operations In this exercise, you complete the following tasks: ●

Review key lecture points

●

Encapsulate the system boot disk

●

Mirror the system boot disk

●

Perform an online volume relayout

●

Evacuate a disk drive

●

Move a populated volume

●

Perform a snapshot backup

●

Create a layered volume

●

Replace a failed disk drive

●

Use Intelligent Storage Provisioning

●

Configure a best practice boot disk

Preparation If your lab environment uses a central VxVM server instead of standalone workstations, the ‘‘Encapsulating the System Boot Disk’’ section on page 6-4 and the ‘‘Mirroring the System Boot Disk’’ section on page 6-5 must be performed as a demonstration on the VxVM server. The demonstration is typically performed by the instructor. The tasks require a second internal disk drive that is the same model and capacity as the system boot disk.

6-72

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 1 – Reviewing Key Lecture Points Answer the following questions about boot disk mirroring, hot devices, and layered volumes. 1.

Which of the following are features of a preferred boot disk configuration? a.

The boot disk should be located in a storage array.

b.

The boot disk should be mirrored.

c.

The boot disk should not be part of the rootdg disk group.

d.

The boot disk should have at least two unused slices.

e.

The boot disk should be at least 18 Gbytes in size.

The answers are b and d.

2.

Which answer most accurately describes the best practice boot disk configuration process? a.

Initialize, copy, mirror, delete

b.

Encapsulate, mirror, delete, copy

c.

Copy, delete, encapsulate, mirror

d.

Mirror, encapsulate, initialize, copy

The answer is b.

3.

What is the primary benefit of a best-practice boot disk configuration? a.

It improves boot disk performance

b.

It standardizes recovery procedures

c.

It simplifies boot disk prerequisites

The answer is b.

4.

What is the key feature of hot relocation? a.

Whole disk drive relocation

b.

Cylinder group relocation

c.

Subdisk relocation

d.

Sub-volume relocation

The answer is c.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-73

Exercise: Performing Advanced Operations 5.

Can designated hot spares be used for hot relocation? a.

Yes

b.

No

The answer is a.

6.

What is a key prerequisite for both hot spares and hot relocation? a.

Volumes must be striped

b.

Volumes must be failure tolerant

c.

Volumes must be mirrored

d.

Volumes must be striped mirror

The answer is b.

7.

Can a designated hot spare be used for typical volume creation? a.

Yes

b.

No

The answer is b.

8.

What is the default hot-device mode of operation? a.

Hot relocation

b.

Hot spare

The answer is a.

9.

Which of the following commands performs hot-device administration? a.

vxdisk set

b.

vxtask set

c.

vxedit set

d.

vxdisksetup

The answer is c.

6-74

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 10. Which of the following commands starts a layered volume? a.

vxvol

b.

vxrecover

c.

vxdctl

d.

vxdiskadm

The answer is b.

11. What is the primary purpose of the ISP software? a.

To create complex volume structures

b.

To increase overall system performance

c.

To manage complex SAN environments

The answer is c.

12. Which of the following commands is used to create ISP application volumes? a.

vxvol

b.

vxvoladm

c.

vxassist

d.

vxdiskadm

The answer is b.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-75

Exercise: Performing Advanced Operations

Task 2 – Encapsulating the System Boot Disk If your lab configuration uses a central VxVM server, this task must be performed as a demonstration (probably by the instructor). 1.

Before proceeding with this task, ensure that your boot disk meets the following requirements: ●

The boot disk must have at least two unused slices.

●

Ideally, the boot disk does not have any slices in use other than the root and swap partitions.

●

There should be a minimum of 2048 sectors (blocks) at the beginning or end of the boot disk that are not assigned to a partition. A standard lab boot disk should either have one or two unassigned cylinders or have them assigned to an unused partition, usually slice 7. If necessary, use the format utility to zero out the small partition at the end of the boot disk. Do not proceed until your boot disk meets this requirement.

2.

Start the vxdiskadm utility, select option 2, and answer the preliminary questions as follows. Substitute the logical path to your disk drive.

# vxdiskadm Select an operation to perform: 2 Encapsulate one or more disks Menu: VolumeManager/Disk/Encapsulate

Select disk devices to encapsulate: [<pattern-list>,all,list,q,?] c0t0d0 Which disk group [,list,q,?] rootdg There is no active disk group named rootdg. Create a new group named rootdg? [y,n,q,?] (default: y) y Use a default disk name for the disk? [y,n,q,?] (default: y) y A new disk group rootdg will be created and the disk device c0t0d0 will be encapsulated and added to the disk group with the disk name rootdg01 Enter desired private region length [<privlen>,q,?] (default: 2048) 2048

6-76

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 3.

Reboot the Solaris OS. # init 0 ok boot

4.

Observe the following system messages as the encapsulation process proceeds through reconfiguration reboots.

VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup... VxVM vxvm-reconfig INFO V-5-2-324 The Volume Manager is now reconfiguring (partition phase)... VxVM vxvm-reconfig INFO V-5-2-499 Volume Manager: Partitioning c0t0d0 as an encapsulated disk. /dev/dsk/c0t0d0s1 was dump device -invoking dumpadm(1M) -d swap to select new dump device VxVM vxvm-reconfig INFO V-5-2-323 The Volume Manager is now reconfiguring (initialization phase)... VxVM vxvm-reconfig INFO V-5-2-497 Volume Manager: Adding rootdg01 (c0t0d0) as an encapsulated disk. VxVM vxcap-vol INFO V-5-2-89 Adding volumes for c0t0d0. .. Starting new volumes... VxVM vxcap-vol INFO V-5-2-444 Updating /etc/vfstab... Remove encapsulated partitions... VxVM vxroot INFO V-5-2-328 The Volume Manager will now set up your Boot Disk as a managed disk. VxVM vxroot INFO V-5-2-290 Saving original configuration... 80 blocks VxVM vxvm-reconfig NOTICE V-5-2-393 The system will now be rebooted. syncing file systems... done rebooting... Rebooting with command: boot ... VxVM INFO V-5-2-3247 starting special volumes ( swapvol rootvol )... VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup... vxvm: NOTE: Setting partition /dev/dsk/c0t0d0s1 as the dump device.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-77

Exercise: Performing Advanced Operations 5.

Log into the system as user root and verify the boot disk environment is correctly configured. # vxprint # eeprom nvramrc # eeprom “use-nvramrc?”=true # vxdg defaultdg nodg # vxdg bootdg rootdg

Task 3 – Mirroring the System Boot Disk Perform the following steps only if you have already encapsulated your boot disk and have a disk drive available that is identical to the primary system boot disk. In a production system, it would be best to take the system offline before starting a procedure such as the following. Single-user mode would be ideal. Caution – This procedure assumes two identical disk drives. The primary boot disk address is c0t0d0, and the mirror disk is c0t1d0. Ensure that you are using the correct address or disk media name for each step in this procedure. 1.

Locate an appropriate mirror disk and add it to the rootdg disk group using the VxVM medianame rootdg02. # vxdisk list # vxdisksetup -i c0t1d0 # vxdg -g rootdg adddisk rootdg02=c0t1d0

2.

Start the vxdiskadm utility and select option 6, Mirror Volumes on a disk.

# vxdiskadm Select an operation to perform: 6 Mirror volumes on a disk At the prompt below, supply the name of the disk containing the volumes to be mirrored. Enter disk name [,list,q,?] list Disk group: rootdg

6-78

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations DM NAME dm rootdg01 dm rootdg02

DEVICE c0t0d0s2 c0t1d0s2

TYPE auto auto

PRIVLEN 2048 4063

PUBLEN 8380799 8374320

STATE -

Enter disk name [,list,q,?] rootdg01 Enter destination disk [,list,q,?] (default: any) rootdg02 VxVM vxmirror INFO V-5-2-22 Mirror volume rootvol ... VxVM vxmirror INFO V-5-2-22 Mirror volume swapvol ... 3.

Use the vxprint command to examine your completed boot disk configuration.

# vxprint -g rootdg ns-east-104# vxprint -g rootdg TY NAME ASSOC KSTATE dg rootdg rootdg -

LENGTH -

dm rootdg01 dm rootdg02

c0t0d0s2 c0t1d0s2

-

17674902 17678493 -

NOHOTUSE

v pl sd sd pl sd

rootvol rootvol-01 rootdg01-B0 rootdg01-02 rootvol-02 rootdg02-01

root rootvol rootvol-01 rootvol-01 rootvol rootvol-02

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

17139843 17139843 1 17139842 17139843 17139843

0 1 0

ACTIVE ACTIVE ACTIVE -

v pl sd pl sd

swapvol swapvol-01 rootdg01-01 swapvol-02 rootdg02-02

swap swapvol swapvol-01 swapvol swapvol-02

ENABLED ENABLED ENABLED ENABLED ENABLED

531468 531468 531468 531468 531468

0 0

ACTIVE ACTIVE ACTIVE -

4.

PLOFFS -

STATE -

Examine the OpenBoot PROM boot device aliases and verify that an alias for the boot disk mirror was added. # eeprom nvramrc nvramrc=devalias vx-rootdg01 /pci@1f,4000/scsi@3/disk@0,0:a devalias vx-rootdg02 /pci@1f,4000/scsi@3/disk@1,0:a

5.

Halt the Solaris OS and boot from the VxVM boot disk device alias. # init 0 ... ok boot vx-rootdg01

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-79

Exercise: Performing Advanced Operations

Task 4 – Performing an Online Volume Relayout In this task, you use the volume relayout feature of the VEA GUI to add two columns to an existing 3-column RAID-5 volume. This task can also be accomplished using the command line, but the syntax is complex. It is best to unmount the volume file system before performing an online relayout operation. To perform a volume relayout, complete the following steps: 1.

Click the third mouse button on your RAID-5 volume in the VEA GUI grid area and select Change Layout from the pop-up menu. Let the VxVM software select additional disk drives.

2.

In the Change Volume Layout form, change the Columns entry to 5 (as shown in Figure 6-22) and click OK.

Figure 6-22 Change Volume Layout Form

6-80

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 3.

When the status window appears, as shown in Figure 6-23, try pausing and continuing the relayout operation.

Figure 6-23 Relayout Status Monitor 4.

After the relayout has completed, examine the new volume structure using the vxprint command and verify that the results are what you anticipated.

5.

Unmount the RAID-5 volume and delete it.

6.

Create a two-disk mirrored volume, 200 Mbytes in size, with a mounted file system.

7.

Use the mkfile command to create some test files in the mirrorvolume’s file system. # mkfile 10m /Test/file1 # mkfile 20m /Test/file2

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-81

Exercise: Performing Advanced Operations

Task 5 – Evacuating a Disk Drive To evacuate a disk drive, complete the following steps: 1.

Start the vxdiskadm utility on the VxVM server, and select option 7, Move volumes from a disk.

2.

Move the contents of one of the disk drives in your new mirrored volume to another unused disk drive in your disk group.

Caution – If you do not specify a destination disk drive for the move, VxVM uses any available disk space in the disk group. This might result in a poorly configured volume with performance problems.

Task 6 – Moving a Populated Volume In this task, you move your mirrored file system volume into another existing disk group. You must first create a second disk group. Complete the following steps: 1.

On the VxVM server, use the vxprint command to determine the media names and logical paths of the two disk drives being used in your mirrored file system. # vxprint -g old_dg -ht

2.

Record the media names and logical paths of the two disk drives being used in your mirrored volume. For example: dgB01, c3t4d0. Media name: __________

__________

Logical path: __________

__________

3.

Remove an unused disk drive from your current disk group and use it to create a second disk group.

4.

Use the vxprint command to save the mirrored volume configuration. # vxprint -hmQq -g old_dg volumename > volumename.save

Note – Do not locate the volumename.save file on a disk drive that is being relocated. 5.

Unmount any file systems associated with the mirrored volume.

6.

Stop the mirrored volume. # vxvol -g old_dg stop volumename

6-82

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 7.

Remove the definitions of the structures (volume, plexes, and subdisks) from the configuration database. # vxedit -g old_dg -r rm volumename

8.

Remove the disk drives from the original disk group. # vxdg -g old_dg rmdisk olddg01 olddg02

Note – The disk drives should now be in the free disk pool. 9.

Add the two disk drives to the new disk group.

10. Rename the disk drives so that they have their original media names. Caution – Step 10 is critical. If the disk drives do not have their original media names, the configuration reload fails. 11. Use the vxmake command to reload the saved configuration for the volumename volume. # vxmake -g new_dg -d volumename.save The -d option specifies the description file to use for building subdisks, plexes, and volumes. 12. Use the vxvol command to bring the mirrored volume back online. # vxvol -g new_dg init active volumename 13. Mount the mirrored volume file system to return the mirrored volume to service. 14. Unmount the mirrored volume and destroy its disk group. 15. Add all of your disk drives into a single disk group again.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-83

Exercise: Performing Advanced Operations

Task 7 – Performing a Snapshot Backup To perform a snapshot backup, complete the following steps: 1.

Create a two-disk mirrored volume, 200 Mbytes in size, with a mounted file system.

2.

Use the mkfile command to create some test files in the mirrorvolume’s file system. # mkfile 10m /Test/file1 # mkfile 20m /Test/file2

3.

Click the mirrored volume in the VEA GUI grid area, and select Snapshot Interactive from the pop-up menu.

Note – Although it is not absolutely necessary, you can assign disk drives for temporary use in the Volume Snapshot form. 4.

Click Snapstart on the Volume Snapshot form to start the snapshot process. This can take some time depending on the volume size.

5.

Click Snapshot when the mirror copy is completed. This detaches the new mirror and creates a separate volume from it.

6. v SNAP-vol02 pl vol02-02 sd dgY07-01

Use the vxprint command to verify that a volume structure similar to the following is present: fsgen SNAP-vol02 vol02-02

7.

204800 208278 208278

0

ACTIVE ACTIVE -

-

fsck -y /dev/vx/rdsk/dgY/SNAP-vol02 mkdir /vol02_backup_081202 mount /dev/vx/dsk/dgY/SNAP-vol02 /vol02_backup_081202 cd / tar cvf /dev/rmt/0 ./vol02_backup_081202

Unmount and delete the snapshot volume. # umount /vol02_backup_081202 # vxedit -g dgY -rf rm SNAP-vol02

6-84

-

Back up the new snapshot volume to tape (if possible). The following example shows the process. # # # # #

8.

ENABLED ENABLED ENABLED

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 8 – Creating a Layered Volume To perform this task, you need a minimum of five unused disk drives in a disk group. To create a striped-mirror RAID 1+0 volume, complete the following steps: 1.

Unmount all volume-related file systems.

2.

Stop all volumes in your disk group and delete them.

3.

Display the disk drives in your disk group in the VEA GUI grid area, and select all six of them by simultaneously clicking the mouse button and pressing the Control key.

4.

Click New Volume in the toolbar.

5.

Fill out the New Volume Wizard form as follows:

6.

a.

Enter a volume name, such as strmir_vol01.

b.

Enter a size of 200m or larger.

c.

Select the Stripe Mirrored layout.

d.

Leave the number of columns at 2.

e.

Leave the number of mirrors at 2.

f.

Leave logging enabled

g.

Add a file system that mounts at boot time.

Click Next and Finish in the next two forms. It can take quite a while to complete a large layered volume. Keep checking the status of the volume completion either from the command line or by using the Task Monitor.

7.

Use the vxprint command to verify that your RAID 1+0 volume is structured using sub-volume components.

8.

Click the layered volume in the VEA GUI grid area and select Layout View from its pop-up menu.

9.

Resize the Volume Layout window until you can see all of the layered volume components.

10. Close the Volume Layout window when you finish examining the volume’s structure. 11. Test your new volume’s file system by copying data into it.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-85

Exercise: Performing Advanced Operations

Task 9 – Replacing a Failed Disk Drive You can decided whether or not to perform this procedure as a group demonstration.

To simulate a failed disk drive replacement, complete the following steps: 1.

Unmount and delete your RAID 1+0 volume.

Caution – Before proceeding, ensure that all of the disks in the disk group are initialized in a sliced format. If the disks are in a cdsdisk format, the failure simulation will not work. If necessary, destroy the disk group and recreate it using the vxdiskadm utility. 2.

Create a two-disk mirrored volume, 200 Mbytes in size, with a mounted file system, with logging disabled (no DRLs).

3.

Use the mkfile command to create a test file in the mirrored volume’s file system. # mkfile 10m /Test/file1

4.

Double-click on the volume in the VEA GUI object tree to display detailed information about the volume’s disk drives.

5.

Complete the following steps: a.

Simulate a disk drive failure by zeroing-out the VTOC of one of the mirrored volume disks.

b.

Substitute the physical path to one of your mirrored volume disks.

# fmthard -s /dev/null /dev/rdsk/c2t3d0s2 6.

Create a second test file in your mirrored volume’s file system. # mkfile 20m /Test/file2 You should see some plex-related errors.

7.

Complete the following steps: a.

Type mail on the VxVM server and press the Return key to view each new message.

b.

Type q when you are done. Unless you disabled hot-relocation use on all of your disk drive, VxVM relocates the mirror on the failed disk drive to a different disk drive.

6-86

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 8.

Use the vxprint command to search for disabled plexes. The mirrored volume should show a normal status because the failed plex was relocated to a different disk drive.

9.

Complete the following steps: a.

Use the vxdisk list command to verify the failed disk drive.

b.

Record the logical path to the failed disk drive.

Failed disk drive logical path: __________________________________ 10. Remove the failed disk drive from VxVM control. 11. Use the vxdiskadm utility option 4, Remove a disk for replacement, to remove the physical disk from its disk group while retaining its media name. Do not select a replacement disk. Note – If you intend to hot-swap the failed disk without rebooting the system, you might also use the vxdiskadm utility option 11, Disable (offline) a disk device, to stop all VxVM access, such as polling. 12. Perform the storage-specific hardware replacement procedure. The hardware replacement procedures vary widely depending on the storage type and model. Always consult the technical manuals for your particular array model. 13. Return the failed disk drive to VxVM control. a.

Use the vxdiskadm utility option 10, Enable (online) a disk device, to enable VxVM access to the replacement disk.

b.

Use the vxdiskadm utility option 5, Replace a failed or removed disk, to associate the new disk with its media name. You initially select none as a replacement device. You are then asked if you want to initialize the new disk drive. Do not use FMR for plex resync.

Note – In some cases you might need scan for new disk drives using the vxdctl enable command.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-87

Exercise: Performing Advanced Operations 14. Use the vxdiskadm utility option 14, Unrelocate subdisks back to a disk, to move the relocated volume components back to the original disk location and complete the following steps: ●

The only information you furnish is the media name of the replacement disk, VxVM

●

Answer no to Unrelocate to a new disk.

●

Answer no to Use -f option.

15. Use the vxprint command to verify the mirrored volume is returned to its original configuration. The letters UR are added to the relocated subdisk name.

Task 10 – Using Intelligent Storage Provisioning Perform the following steps to configure ISP storage pools and ISP application volumes. 1.

Unmount and delete all volumes in your disk group.

2.

Click your disk group in the VEA GUI object tree and select Organize DiskGroup from its pop-up menu.

3.

Click on the Mirrored Data and Striped Snapshot organization category in the Organize DiskGroup Wizard window.

4.

Use the default data pool and clone pool names.

5.

Review the summary information and click Finish when you are satisfied with the configuration.

6.

Display your disk group in the VEA GUI grid area and click the Storage Pools tab.

7.

Complete the following steps:

8.

6-88

a.

Click the mirrored_volumes1 data pool with the third mouse button and select Properties from its pop-up menu.

b.

Examine the properties and capabilities associated with the mirrored_volumes1 data pool.

Complete the following steps: a.

Click the mirrored_volumes1 data pool with the third mouse button and select Add/Remove Disks from its pop-up menu.

b.

Add all six of your disk drive to the mirrored_volumes1 data pool.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations 9.

Click your disk group in the VEA GUI object tree and select New Volume from its pop-up menu.

10. Create a mirrored ISP application volume as follows: a.

Click Next in the Select User Template window.

b.

Click Data Redundancy and Data Mirroring capabilities, and then click Next.

c.

Leave the Let Volume Manager Decide button enabled, and then click Next.

d.

Complete the following steps: 1.

Enter a volume name.

2.

Set the size to 200 Mbytes

3.

Click Next.

e.

Do not create a file system on the volume.

f.

Complete the following steps: 1.

Enter a user template name.

2.

Click Save.

3.

Click Finish.

g.

Use the VEA GUI and the vxprint command to examine the resulting volume structure and storage pool organization.

h.

Delete the mirrored ISP application volume.

11. Create another application volume in your data storage pool using additional capabilities. a.

Click Next in the Select User Template window.

b.

Click Mirrored DCO Logs, Data Redundancy and Data Mirroring capabilities, and then click Next.

c.

Leave the Let Volume Manager Decide button enabled, and then click Next.

d.

Complete the following steps: 1.

Enter a volume name.

2.

Set the size to 200 Mbytes.

3.

Click Next.

e.

Do not create a file system on the volume.

f.

Click Finish.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-89

Exercise: Performing Advanced Operations

Note – DCO logging is associated with a feature not currently licensed or supported by Sun Microsystems. 12. Click your disk group in the VEA GUI object tree and select New Storage Pool from its pop-up menu. 13. Create another storage pool in your disk group as follows: a.

Enter a storage pool name and click Next.

b.

Click the RAID-5 storage pool template, and then click Next.

c.

Examine the summary information, and then click Finish.

14. Display your storage pools in the VEA GUI grid area and verify that the new storage pool is a clone pool. 15. Delete all of your storage pools and organize your disk group again using a different storage pool set template, such as the Striped Mirrored Data and Striped Snapshots template. 16. Create one more application volume using the new storage pool capabilities.

6-90

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Performing Advanced Operations

Task 11 – Configuring a Best Practice Boot Disk Do not perform this task unless you have encapsulated and mirrored your boot disk in an earlier task. The following section assumes that all disk drives and volume components follow the default VxVM naming conventions. To reorganize the disks into a best practice configuration, complete the following steps: 1.

Un-mirror the primary boot disk by removing the mirror plexes from the mirror disk. Use the Bourne shell. # vxassist -g rootdg remove mirror rootvol !rootdg02 # vxassist -g rootdg remove mirror swapvol !rootdg02

2.

To ensure the correct mirror placement, manually mirror the rootvol and swapvol volumes again in the order shown. # vxrootmir rootdg02 # vxassist -g rootdg mirror swapvol rootdg02

3.

Disassociate all primary boot disk plexes and recursively remove them. # # # # #

4.

vxplex vxplex vxedit vxedit vxedit

-g -g -g -g -g

rootdg rootdg rootdg rootdg rootdg

-v -v -r -r -r

rootvol dis rootvol-01 swapvol dis swapvol-01 rm rootdg01Priv rm rootvol-01 rm swapvol-01

Verify that only the mirror disk plexes remain.

# vxprint -g rootdg TY NAME ASSOC dg rootdg rootdg

KSTATE -

LENGTH -

PLOFFS -

STATE -

dm rootdg01 dm rootdg02

c0t0d0s2 c0t1d0s2

-

8380799 8374320

-

NOHOTUSE

v rootvol pl rootvol-02 sd rootdg02-01

root rootvol rootvol-02

ENABLED ENABLED ENABLED

7808400 7808400 7808400

0

ACTIVE ACTIVE -

v swapvol pl swapvol-02 sd rootdg02-02

swap swapvol swapvol-02

ENABLED ENABLED ENABLED

565920 565920 565920

0

ACTIVE ACTIVE -

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-91

Exercise: Performing Advanced Operations 5.

Remove the rootdg01 disk from the rootdg disk group, and complete the following steps: a.

Reinitialize the rootdg01 disk.

b.

Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01 # vxdisksetup -i c0t0d0 format=simple # vxdg -g rootdg adddisk rootdg01=c0t0d0 6.

Use the vxdiskadm option 6, Mirror volume on a disk, to mirror rootvol and swapvol on the rootdg02 disk drive back to the newly initialized rootdg01 disk drive. # vxdiskadm ... At the prompt below, supply the name of the disk containing the volumes to be mirrored. Enter disk name [,list,q,?] rootdg02 Enter destination disk [,list,q,?] (default: any) rootdg01

7.

Modify the boot-device parameter and enable nvramrc usage. # eeprom boot-device="rootdisk rootmirror" # eeprom "use-nvramrc?"=true

8.

Use the prtvtoc command and the format utility to verify that the primary and mirror boot disk drives have exactly the same partition maps.

9.

Boot the system from each of the available system devices. ok ok ok ok

boot boot boot boot

disk0 disk1. vx-rootdg01 vx-rootdg02

You can now replace either the primary boot disk drive or its mirror in the same manner as any other VxVM disk drive, and just resynchronize the mirrors.

6-92

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, highlight just the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS Volume Manager Advanced Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

6-93

Module 7

VERITAS File System Basic Operations Objectives Upon completion of this module, you should be able to: ●

Describe basic VxFS features

●

Install the VxFS software

●

Create VxFS file systems

●

Use extended VxFS mount options

●

Perform online VxFS administration tasks

7-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic VxFS Features

Basic VxFS Features The VxFS software is targeted at commercial environments where high performance and availability are important, and large volumes of data must be managed. The VxFS extent-based space allocation scheme increases performance by reducing the number of I/O operations required to read and write large amounts of data. The VxFS intent log feature provides fast recovery following a system crash or reboot. A file system check can be completed in seconds, regardless of the file system size.

Extent-Based Space Allocation Standard UFS file systems use block-based allocation schemes and provide good random access to files and reasonable latency on small files. For larger files, however, this block-based architecture limits throughput. The VxFS file system improves file system performance by using a different allocation scheme name extent-based allocation. Disk space is allocated by the system in 512-byte sectors, which are grouped together to form a logical block. VxFS supports logical block sizes of 1024, 2048, 4096, or 8192 bytes. The default block sizes vary according to file system size as follows: ●

1024 block size for file systems up to 4-Tbytes in size

●

2048 block size for file systems up to 8-Tbytes in size

●

4096 block size for file systems up to 16-Tbytes in size

●

8192 block size for file systems beyond 16-Tbytes in size

An extent is one or more adjacent blocks of data within a file system. It is presented as an address-length pair, which identifies the starting block address and the length of the extent (in blocks). When data is added to a file on a VxFS system, it is grouped in extents as opposed to being allocated a block at a time (as is done with UFS file systems). By allocating disk space in extents, disk I/O to and from a file can be done in units of multiple blocks considerably faster than block-at-a-time operations.

7-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic VxFS Features

File System Intent Logging After a system failure, the structural integrity of UNIX file systems is verified by the fsck program during the system reboot operation. This means checking the entire structure of a file system, verifying that it is intact, and correcting any inconsistencies that are found. This process can be very time consuming. The VxFS file system provides recovery only seconds after a system failure by using a tracking feature called intent logging. Intent logging is a logging scheme that records pending changes to the file system structure. During system recovery from a failure, the intent log for each file system is scanned and operations that were pending are completed. The file system can then be mounted without a full structural check of the entire system. When the disk has a hardware failure, the intent log might not be enough for recovery and, in such cases, a full fsck check must be performed. However, when the failure is due to software rather than hardware, a system can be recovered in seconds. The default intent log size is currently 64 Mbytes. The fsadm command can be used to dynamically modify the intent log size. Larger intent logs can improve system performance because they reduce the number of times the log wraps around. An intent log that is too large can increase file system recovery time after a system failure. Note – See the fsadm_vxfs and mkfs_vxfs man pages for more information.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-3

Installing the VxFS Software

Installing the VxFS Software You use the pkgadd command to install the VxFS software. The VxFS packages should be installed in the order shown. # pkgadd -d . The following packages are available: 1 VRTSalloc VERITAS Volume Manager: VERITAS Intelligent Storage Provisioning (sparc) 4.0,REV=12.06.2003.03.07 2 VRTSap VERITAS Action Provider (sparc) 2.00.015 3 VRTSddlpr VERITAS Device Discovery Layer Services Provider (sparc) 4.0,REV=12.06.2003.02.03 4 VRTSfppm VERITAS File Placement. (sparc) 4.0,REV=GA03 5 VRTSfsdoc VERITAS File System Documentation (sparc) 4.0,REV=GA04 6 VRTSfsman VERITAS File System - Manual Pages (sparc) 4.0,REV=GA04 7 VRTSfspro VERITAS File System Management Services Provider (sparc) 4.0,REV=GA04 8 VRTSmuob VERITAS Enterprise Administrator Service Localized Package (sparc) 3.2.514.0 9 VRTSob VERITAS Enterprise Administrator Service (sparc) 3.2.514.0 10 VRTSobgui VERITAS Enterprise Administrator (sparc) 3.2.514.0 11 VRTStep VERITAS Task Exec Provider (sparc) 1.20.025 12 VRTSvlic VERITAS License Utilities (sparc) 3.02.005d 13 VRTSvmdoc VERITAS Volume Manager (user documentation) (sparc) 4.0,REV=12.06.2003.01.34 14 VRTSvmman VERITAS Volume Manager, Manual Pages (sparc) 4.0,REV=12.06.2003.01.34 15 VRTSvmpro VERITAS Volume Manager Management Services Provider (sparc) 4.0,REV=12.06.2003.01.35 16 VRTSvxfs VERITAS File System (sparc) 4.0,REV=GA04 17 VRTSvxvm VERITAS Volume Manager, Binaries (sparc) 4.0,REV=12.06.2003.01.35 Select package(s) you wish to process (or ’all’ to process all packages). (default: all) [?,??,q]: 16 4 5 6 11 2

7-4

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating VxFS File Systems

Creating VxFS File Systems You use the same process as with the UNIX UFS file system to create VxFS file systems. To create VxFS file system on top of VxVM volumes, you can use either the VEA GUI New File System Wizard or the standard file system command-line programs, such as mkfs. The primary difference is that you must specify a file system type of vxfs. You create a VxFS file system using the mkfs command as follows. # mkfs -F vxfs /dev/vx/rdsk/dgX/mirvol The following mkfs command options can affect system performance when used appropriately: ●

mkfs -F vxfs -o bsize You use the bsize option to decrease or increase the file system block size during initial file system creation. For most applications it is best to use the default VxFS block size, which is automatically calculated based on file system size.

●

mkfs -F vxfs -o logsize You use the logsize option to decrease or increase the size of the intent log during initial VxFS file system creation. Increasing the size of the intent log can improve performance, but the log must never be larger than half of virtual memory (physical memory + swap).

●

mkfs -F vxfs -o largefiles/nolargefiles If the largefiles option is enabled, a persistent largefiles bit is written on the file system. The largefiles bit (flag) allows the creation of files larger than 2 Gbytes. The default mode of operation is largefiles.

Note – Consult the mkfs_vxfs man pages for more information on other VxFS mkfs command options, including inosize and ninode.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-5

Extended VxFS Mount Options

Extended VxFS Mount Options The VxFS software provides a number of mount options that might be appropriate under special circumstances. You should not use many of the options unless there is a clearly defined need. The most commonly used VxFS mount options control intent log behavior and how the VxFS software responds to media errors.

Intent Log Behavior The log, delaylog, tmplog, logiosize, and nodatinlog mount options have varying effects on VxFS file system performance and integrity, depending on specific needs. ●

mount -F VxFS -o log The log option guarantees all file system changes are logged before a system call returns. Recent changes are not lost.

●

mount -F VxFS -o delaylog The default logging mode is delaylog. Most system call changes are logged before returning. Some changes are logged shortly after the system call returns. After a system failure, the most recent change might be lost. This behavior is similar to UNIX UFS file system behavior.

●

mount -F VxFS -o tmplog When the tmplog mode of operation is enabled, most intent log changes are made after the system calls return. This mode can increase file system performance, but places recent data changes at a higher level of risk.

●

mount -F VxFS -o logiosize=size The logiosize option can increase performance by adjusting intent log write size to match the stripe size of read-modify-write (RAID 5) devices. Valid size values are 512, 1024, 2048, 4096, or 8192.

●

mount -F VxFS -o nodatinlog The nodatainlog option should not be used on systems capable of relocating bad disk blocks. The Solaris OS uses bad block relocation to relocate disk blocks that are having recoverable read or write errors. If enabled, nodatainlog decreases the file system performance by approximately 50 percent.

7-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Extended VxFS Mount Options

Error Handling Behavior The ioerror option has several sub-options that are designed to handle errors in a way that is appropriate for the related storage technology. ●

mount -F VxFS -o ioerror=disable The disable sub-option disables all file system data, and metadata reads and writes when an error is detected. You use this sub-option when the underlying storage is redundant, such as mirroring or RAID 5. You should use the disable policy for cluster file systems.

●

mount -F VxFS -o ioerror=nodisable The nodisable sub-option does not disable reads or writes on error detection. It posts appropriate error messages and sets numerous file system error flags. This behavior is similar to previous versions of VxFS.

●

mount -F VxFS -o ioerror=wdisable/mwdisable The wdisable sub-option disables all write operations to file system and metadata, but read status is set to degraded. The mwdisable sub-option disables writes to metadata, and all other read and write status is degraded. The wdisable and mwdiable sub-options are used in environments where read errors are more likely to persist than write errors, such as when using non-redundant storage. The mwdisable policy is the default for locally mounted file systems.

Other VxFS Mount Options There is also a blkclear option that is designed for high security environments. The blkclear mode of operation guarantees that uninitialized storage (old data) never remains in a file system. This mode of operations can degrade file system performance by as much as 10 percent. Note – Consult the mount_vxfs man pages for further details about other available mount options, some of which relate to VxFS features that are not licensed by Sun.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-7

Online File System Administration

Online File System Administration You can defragment, resize, and back up VxFS file systems while they are online and in use.

Online Defragmentation The UFS software uses the concept of cylinder groups to limit fragmentation. These are self-contained sections of a file system that are composed of inodes, data blocks, and bitmaps, that indicate free inodes and data blocks. Allocation strategies in UFS attempt to place inodes and related data blocks near each other. This strategy reduces fragmentation, but does not eliminate it. Over time, the original ordering of free resources can be lost. As files are added and removed, gaps between used areas of the disk can still occur. The VxFS software fsadm utility is used to defragment a disk without requiring that the disk be unmounted first. The fsadm utility removes unused space from directories, makes small files contiguous, and consolidates free blocks for use.

Online Resizing When UFS file systems are too small or become too large for their assigned disk space, the following methods are used to correct the problem: ●

Users are moved to new or different file systems.

●

Subdirectories are moved to other file systems.

●

The file systems are backed up and restored to a different file system.

VxFS with VxVM enables a file system to be expanded or reduced in size while it is being accessed.

Online Backup and Restore You use the VxFS vxdump utility to dump volume snapshots to tape. Volumes are restored using the vxrestore utility.

7-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Exercise: Configuring VxFS In this exercise, you complete the following tasks: ●

Review key lecture points

●

Install the VxFS software

●

Create a VxFS file system

●

Resize a VxFS file system

●

Defragment a VxFS file system

●

Backup and restore a VxFS file system

●

Use VxFS extended mount options

Preparation VxFS is a separately licensed option. You must install a license key to activate the software. Ask your instructor for a VxFS temporary license key. Record the temporary key. VxFS license key: __________________________________________________ You can get VxVM, VxFS, and Shared Disk Group temporary licenses at the Sun Business Partners Web site at http://webhome.ebay/partnersoftware/.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-9

Exercise: Configuring VxFS

Task 1 – Reviewing Key Lecture Points Answer the following questions about VxFS features and mount options. 1.

What is the primary advantage of extent-based space allocation? a.

Faster recovery times after system failures

b.

Reduced data fragmentation

c.

Increased throughput in write-intensive applications

d.

Reduced data loss after system failures

The answer is c.

2.

What is the primary advantage of intent logging? a.

Simplified file system administration

b.

Enhanced space utilization

c.

Improved file system performance

d.

Faster recovery time after system failures

The answer is d.

3.

Which of the following VxFS mount options is used to increase data security? a.

log

b.

delaylog

c.

blkclear

d.

nodatinlog

e.

convosync

The answer is c.

4.

Which of the following VxFS mount options should be used with cluster file systems? a.

log

b.

delaylog

c.

blkclear

d.

nodatinlog

The answer is a.

7-10

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Task 2 – Installing the VxFS Software To install the VxFS software, complete the following steps: 1.

Log in as user root on the system attached to the storage arrays.

2.

Change to the VxVM installation package location furnished by your instructor.

3.

Install the VxFS packages in the order shown and answer yes to all questions. # pkgadd -d . VRTSvxfs VRTSfppm VRTSfsdoc VRTSfsman \ VRTStep VRTSap

4.

Install the VxFS license key furnished by your instructor. # vxlicinst VERITAS License Manager vxlicinst utility version 3.02.005 Copyright (C) VERITAS Software Corp 2002. All Rights reserved. Enter your license key : 3JZU-YBPO-CR28-RWWP-P5RO-P12

5.

Reboot the VxFS server. There are no VxFS-specific boot messages.

6.

Log in to the VxFS server and verify that the VxFS program files are installed. # ls /opt/VRTSvxfs/sbin cp qiostat cpio qlogadm fcladm qlogckd fsadm qlogclustadm fsapadm qlogdb fscat qlogdetach fsckpt_restore qlogdisable fsckptadm qlogenable fsclustadm qlogmk fsvoladm qlogprint getext qlogrm ls qlogstat mv qlogtrace qioadmin setext qiomkfile vxdump

vxedquota vxenablef vxfsckd vxfsconvert vxfsstat vxlsino vxquot vxquota vxquotaoff vxquotaon vxrepquota vxrestore vxtunefs vxupgrade

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-11

Exercise: Configuring VxFS 7.

Verify that the VxFS technical manuals are installed. # ls /opt/VRTS/docs vxfs_ag.pdf vxfs_ig.pdf

8.

Verify that the VxFS man pages have been added to the /opt/VRTS/man directory. # man vxquot # man vxdump

9.

Verify that the VxFS driver is loaded. # modinfo | grep vxfs

10. Verify that the special file system programs that are required early in the boot process are present. # ls /etc/fs/vxfs mount qlogattach

qlogck qlogrec

system.preinstall

11. Move to the VxFS administration guide directory. # cd /opt/VRTS/docs 12. View the contents of the manuals with the Adobe Acrobat Reader (acroread). 13. Complete the following steps: a.

Examine Appendix B, “Kernel Messages,” of the VERITAS File System 4.0 Administrator’s Guide.

b.

Examine the Description and Action sections associated with each of the error messages.

Note – Use the Control-F sequence to enable the Adobe Acrobat Find window.

7-12

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS

Task 3 – Creating a VxFS File System You use VEA to perform the following file system operations. Study VxFS options in the VERITAS File System 4.0 Administrator’s Guide as necessary. To create a VxFS file system, complete the following steps: 1.

Delete all volumes in your disk group and add all of your disks to the group.

2.

Use the VEA GUI to create a mirrored volume as follows: a.

Use all 6 of your disk drives for the volume.

b.

Click Concatenated layout and select the Mirrored check box.

c.

Set the total number of mirrors at 2.

d.

Disable logging.

e.

Click Max Size to calculate the maximum size and round-off the result that is returned.

f.

Do not create a file system on the volume.

3.

Click your new volume and select File System then New File System from its pop-up menu.

4.

Configure the New File System window as follows: a.

Set the file system type to vxfs.

b.

Leave the block size set to the default value (1024).

c.

Enter a file system mount point.

d.

Click Add to file system table check box, and complete the following steps:

e. 5.

1.

Select Mount at boot.

2.

Set the fsck pass number to 2.

Click OK.

In the VEA GUI grid area, complete the following steps: a.

Click your VxFS volume and select Layout View from its popup menu.

b.

Examine the concatenated mirror volume structure and close the Layout View window when you have finished.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-13

Exercise: Configuring VxFS 6.

On the VxVM server, complete the following steps: a.

Examine the VEA GUI command log.

b.

Observe the volume size and the log size information in the Output section of the mkfs command.

# tail -40 /var/vx/isis/command.log c.

Verify that the largefiles option was specified with the mkfs command.

Task 4 – Resizing a VxFS File System A unique feature of VxFS is the ability to decrease the size of an existing file system that is overly large for its intended application. To decrease the size of the VxFS file system created previously in this exercise, complete the following steps. 1.

Use the df -kl command to verify the amount of space available in your new VxFS file system.

# df -kl Filesystem kbytes .... /dev/vx/dsk/dgX/mirvol 26214400

used

avail

capacity

Mounted on

39890

24538611

1

/VXFS

In the preceding example, the file system is approximately 26-Gbytes in size and about 40-Mbytes of space is in use by inode and intent log space. 2.

Use the mount command to verity that your VxFS file system was mounted using the delaylog, largefiles, and ioerror=mwdisable mount options.

3.

Verify that you can create a file larger than 2-Gbytes in your file system. # mkfile 2500m /VXFS/file1

7-14

4.

Delete all test files from your VxFS file system.

5.

Click on the VxFS volume in the VEA GUI Object tree, and select Resize Volume from its pop-up menu.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS 6.

Complete the Resize Volume form as follows: a.

Enter a new volume size of 1 Gbyte.

b.

Let volume manager decide which disks to use.

c.

Click on the OK button to complete the resize operation.

d.

Verify the new VxFS file system size by using the df -kl command.

Task 5 – Defragmenting a VxFS File System See the fsadm_vxfs man page for further information on the fsadm defragmentation utility. Perform the following steps to defragment your VxFS file system: 1.

Use the fsadm command to verify the current fragmentation in your VxFS file system. Substitute the name of your file system.

# fsadm -D /VXFS Directory Fragmentation Report Dirs Total Searched Blocks total 2 0

Immed Dirs 0

Immeds to Add 2

Dirs to Reduce 0

Blocks to Reduce 0

The Blocks to Reduce value is a general indicator that defragmentation might be of value. Some directories are fragmented in a way that does not respond to defragmentation. 2.

Perform the following command sequence to create some fragmentation in your VxFS file system. Substitute the name of your file system. # # # # # #

3.

cp cp rm cp cp cp

-r -r -r -r -r -r

/usr/dt /VXFS /usr/appserver /VXFS /VXFS/dt /usr/iplanet /VXFS /usr/share /VXFS /usr/dt /VXFS

(151 Mbytes) (75 Mbytes) (65 Mbytes) (78 Mbytes)

Verify the level of fragmentation and available space in your VxFS file system. # fsadm -D /VXFS # df -kl

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-15

Exercise: Configuring VxFS 4.

Defragment your VxFS file system. Substitute the name of your file system. # fsadm -de /VXFS

5.

Verify your VxFS file system’s fragmentation and available space have improved. # fsadm -D /VXFS # df -kl

Task 6 – Backing Up and Restoring a VxFS File System To use the Snapshot feature to prepare for a VxFS file system backup, complete the following steps: 1.

In the VEA grid area, click your VxFS volume and select Snap then Snap Start from its pop-up menu.

2.

Let volume manager decide what disks to use for the Snapshot, and then click OK.

3.

On the VxVM server, use the vxprint command to verify that the Snapshot structure is complete.

4.

In the VEA grid area, click your VxFS volume and select Snap then Snap Shot from its pop-up menu.

5.

Examine the Snap Shot Volume form and complete the following steps: a.

Enter a custom volume name or use the default volume name.

b.

Click OK. The Snapshot mirror is detached and used to create the Snapshot volume.

6.

In the VEA GUI grid area, click the Snapshot volume and select File System then Mount File System from its pop-up menu.

7.

Enter a mount point name in the Mount File System form and click OK.

8.

Verify that the Snapshot volume’s file system is mounted and available. # df -kl

7-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS 9.

Ask your instructor if there is a tape unit available to perform the following dump and restore procedure. Substitute the name of your Snapshot file system and VxFS file system. # vxdump -0f /dev/rmt/0 /SNAP # cd /VXFS # rm -r appserver dt iplanet share # ls lost+found # vxrestore -f /dev/rmt/0

Note – The vxdump and vxrestore utilities are functionally the same as the Solaris OS ufsdump and ufsrestore utilities.

Task 7 – Using VxFS Extended Mount Options To investigate the effects of selected VxFS file system mount options, complete the following steps: 1.

If you have not done so already, remove all of the files from your VxFS file system. # cd /VXFS # rm -r appserver dt iplanet share # ls lost+found # cd /

2.

Create a test file in your VxFS file system and record the amount of time it takes to complete the operation. # ptime mkfile 100m /VXFS/file1 real 7.348 user 0.076 sys 3.459 # rm /VXFS/file1

3.

Unmount your VxFS file system and mount it again using the blkclear security option. # umount /VXFS # mount -F vxfs -o blkclear /VXFS

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-17

Exercise: Configuring VxFS 4.

Use the mount command to verify that the VxFS file system’s mount options are now read, write, setuid, blkclear, delaylog, largefiles, and ioerror=mwdisable.

5.

Run a performance test on your VxFS file system again. # ptime mkfile 100m /VXFS/file1 real 1:10.047 user 0.078 sys 5.145 # rm /VXFS/file1

6.

Unmount your VxFS file system and mount it again with no special options. # mount -F vxfs /VXFS

7-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, then just highlight the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

VERITAS File System Basic Operations Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

7-19

Module 8

VERITAS Volume Manager Performance Management Objectives Upon completion of this module, you should be able to: ●

Describe performance improvement techniques

●

Use the vxstat and vxtrace performance analysis tools

●

Describe RAID-5 write performance characteristics

8-1 Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques

Performance Improvement Techniques Periodic reassessment of volume performance is necessary on any system. The access to any data structure can degrade over time to the point of poor performance.

Data Assignment Strategies When deciding where to locate a file system, you usually attempt to balance I/O load among available disk drives. The success of this process is limited by the difficulty of anticipating future usage patterns. Figure 8-1 shows how data assignment mistakes can lead to a performance problem.

Controller

Controller

c3

c4

Array

Array Heavy-Use Volume

Low-Use Volume

Heavy-Use Volume

Low-Use Volume

Heavy-Use Volume

Low-Use Volume

Heavy-Use Volume

Low-Use Volume

Figure 8-1

Data Assignment Bottleneck

In general, do not place file systems that have heavy I/O loading on the same disk drives. Separate them into different storage arrays on various controllers. Also, the placement of logs can be critical to performance. This is especially true of RAID-5 logs.

8-2

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques The following solutions can be used to resolve the problem demonstrated in Figure 8-1 on page 8-2: ●

Swap some of the heavy-use volumes with the low-use volumes.

●

Move one of the heavy-use subdisks to a different location.

Note – Swapping volume locations is probably a better solution because it eliminates having two heavily used volumes on a single disk drive. Another type of performance problem can occur when a log plex is placed on the same disk drive as its associated data plex. In the case of RAID-5 logs, you should always consider that the data written to all RAID-5 columns must also be written to the log. In a six-column RAID-5 volume, this configuration could increase the I/O rate of the log disk drive by as much as 600 percent. As shown in Figure 8-2, leaving unused space on all disk drives ensures that you always have alternate locations to which to move logs. This is why the Maxsize calculations of the vxassist command or VEA might not be wise to use. Volume 01

Column 0

Column 1

Column 2

Column 3

Vol02_log

Log Space

Log Space

Log Space

Volume 02

Column 0

Column 1

Column 2

Column 3

Log Space

Log Space

Log Space

Vol01_log

Figure 8-2

RAID-5 Log Placement

The log placement shown in Figure 8-2 would not work well if both volumes were heavily accessed. The configuration would work best if at least one of the volumes has low write activity.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-3

Performance Improvement Techniques

Volume Structure Strategies Sometimes, performance problems are not due to physical volume locations and can be greatly reduced by reconfiguring the volume structures. In many cases, this can be accomplished using the VxVM Volume Relayout feature.

Striping Striping distributes data across multiple devices to improve access performance. Striping improves performance for both read and write operations. If you can identify the most heavily accessed volumes (for file systems or database tables) during the initial design stages, then you can eliminate performance bottlenecks by striping them across several devices. The example in Figure 8-3 shows a volume (Hot_Vol) that was identified as being a data-access bottleneck. The volume is striped across four disk drives, leaving the remainder of those four disk drives free for use by less heavily used volumes.

Hot_Vol Stripe 0

Hot_Vol Stripe 1

Light Use

Light Use

Hot_Vol Stripe 2 Light Use

Hot_Vol Stripe 3 Light Use

Light Use

Figure 8-3

8-4

Using Striping to Improve Performance

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques

Mirroring Mirroring stores multiple copies of data on a system. Mirroring is primarily used to protect against data loss due to physical media failure. It also improves the chance of data recovery in the event of a system crash. In some cases, mirroring can also be used to improve system performance. Mirroring heavily accessed data not only protects the data from loss due to disk drive failures, but it can also improve I/O performance. Unlike striping, however, performance gained through the use of mirroring depends on the read/write ratio of the disk drive accesses. If the system workload is primarily write-intensive (for example, greater than 30 percent writes), then mirroring can result in somewhat reduced performance.

RAID 0+1 RAID 0+1 is also referred to as a mirror-stripe configuration. When used together, mirroring and striping provide the advantage of both spreading the data across multiple disk drives and providing redundancy of data.

Layered Volumes Striped mirror (RAID 1+0) volume configurations can improve performance significantly. Additionally, striped mirror configurations can tolerate a higher percentage of disk drive failures and recovery times are greatly reduced.

RAID 5 RAID 5 provides the advantage of read performance that is similar to that of striping, while also providing data protection using a distributed parity scheme. The disadvantage of RAID 5 is relatively slow write performance. RAID 5 is not generally seen as a performance improvement mechanism except in cases of read-intensive applications. However, RAID-5 volume structures can be adjusted to dramatically improve performance for some write-intensive applications.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-5

Performance Improvement Techniques

Read Policy Strategies To provide optimal read performance for different types of mirrored volumes, you can select the following read policies: ●

The round-robin read policy (round) Read requests to a mirrored volume are satisfied in a round-robin manner from all plexes in the volume. You select this policy when there is no significant performance advantage by using any particular mirror.

●

The preferred-plex read policy (prefer) Read requests to a mirrored volume are satisfied from one specific plex (presumably the plex with the highest performance), unless that plex has failed.

●

The default read policy (select) The appropriate read policy is automatically selected for the configuration. For example, preferred-plex is selected when there is only one striped plex associated with the volume, and round-robin is selected in most other cases.

In the example in Figure 8-4, you set the read policy of the volume labeled Hot_Vol to prefer for the striped plex labeled Plex 1. In this way, read requests are directed to the striped plex that has the best performance characteristics.

Hot_Vol Stripe 0 Plex 1

Hot_Vol Stripe 1 Plex 1

Hot_Vol Stripe 2 Plex 1

Disk 1

Disk 2

Disk 3

Figure 8-4

Hot_Vol Plex 2

Disk 4

Preferred-Plex Read Policy

You can change volume read policies either from the command line or using the VEA GUI.

8-6

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Performance Improvement Techniques In the VEA GUI, highlight the volume in the grid area, and click Props in the toolbar. In the General properties tab, you can choose one of following fixed read policy options: ●

Based on Layout

●

Round Robin

●

Prefer (preconfigured)

From the command line, use the vxvol command as follows: # vxvol -g dgY rdpol prefer vol01 vol01-03

Hardware Configuration Strategies For increased performance, availability, or both, striping and mirroring should be done across system boards, controllers, and targets. You can gain the highest level of performance or reliability by striping or mirroring across system boards, as shown in Figure 8-5. Host System System Board

Controller c3

System Board

Controller c4

Perferred stripe or mirror configuration

t1

t1 t2

t2

t3 Array

Figure 8-5

t3 Array

High Availability and Performance Cabling

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-7

Using Performance Analysis Tools

Using Performance Analysis Tools The VxVM software continuously gathers performance statistics about all devices and objects under its control. The types of information include: ●

A count of operations

●

The number of blocks transferred

●

The average operation time (which reflects the total time through the VxVM software and is not suitable for comparison against other statistics programs)

The statistics include the number of reads, writes, atomic copies, verified reads, verified writes, plex reads, and plex writes for each volume. As a result, one write to a two-plex volume results in at least five operations: one for each plex, one for each subdisk, and one for the volume. VxVM also maintains other statistical data, such as information about read and write failures. The statistics are continuously gathered starting with the system boot operation. The statistics are reset prior to a testing operation.

Gathering Volume Performance Statistics The vxstat command displays statistical information about different types of VxVM physical and logical objects. You can use the following options to control the output: ●

vxstat -g disk_group Displays volume statistics for the specified disk group.

●

vxstat -g disk_group vol01 Displays statistics for the specified volume.

●

vxstat -g disk_group -d Displays disk drive level statistics for the specified disk group.

●

vxstat -g disk_group -d disk01 Displays statistics for the specified disk drive.

After performance data has been gathered, you can analyze it to determine, and to optimize, your system configuration for efficient use of system resources.

8-8

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Performance Analysis Tools A volume or disk drive with elevated read or write access times is not necessarily a problem. If the slow response is not causing any apparent problems for users or applications, then there might not be anything that needs fixing.

Preparing for Analysis Before obtaining statistics, clear (reset) all existing statistics by using the vxstat -r command. Clearing statistics eliminates any differences between volume or disk drives due to volumes being created. It also removes statistics that are not typically of interest, such as information about booting. After clearing the statistics, allow the system to run during typical system activity. When monitoring a system that is used for multiple purposes, try not to exercise any one application more than it would usually be exercised. It can also be beneficial to take periodic snapshots of the volume statistics to help identify the source of irregular system-load problems. A single volume with excessive I/O rates can cause performance degradation on other volumes associated with the same disk drives.

Analyzing Volume Statistics Use the vxstat command as follows to help identify volumes with an unusually large number of operations or excessive read or write times: # vxstat -g bench OPERATIONS BLOCKS AVG TIME(ms) TYP NAME READ WRITE READ WRITE READ WRITE vol acct 473 11 57252 44 4.0 20.9 vol brch 23 11 92 44 33.0 20.0 vol ctrl 92773 121312064799783 78102463 276.9 457.2 vol hist1 23 11 92 44 97.0 24.5 vol hist2 23 11 92 44 54.8 22.7 vol hist3 23 11 92 44 103.5 25.5 vol log1 9 27217 9 409716 16.7 21.1 vol log2 7 8830 7 159769 15.7 24.3 vol rb1 123 13 492 52 30.7 83.1 vol rb2 23 11 92 44 149.1 25.5 vol sys 26933 86156 17768834463215.0310.7 vol t11r 23 11 92 44 39.6 24.5

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-9

Using Performance Analysis Tools

Analyzing Disk Drive Statistics You can also use the vxstat command to summarize operations according to physical disk drives. For example: # vxstat -g bench -d OPERATIONS TYP NAME READ WRITE dm c3t98d0 14330 140370 dm c3t100d0 13881 140370 dm c3t113d0 0 0 dm c3t115d0 0 0 dm c3t117d0 0 0

8-10

BLOCKS READ WRITE 120348 986785 117971 986785 0 0 0 0 0 0

AVG TIME(ms) READ WRITE 15.4 185.6 15.4 187.7 0.0 0.0 0.0 0.0 0.0 0.0

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Using Performance Analysis Tools

Gathering Application Performance Statistics The vxtrace command displays detailed trace information about errors or I/O operations. This level of detail is generally not necessary, but is included here for completeness. You can use the following options to control the display: ●

vxtrace -o disk Traces all physical disk drive I/O operations.

●

vxtrace -o disk c3t98d0 Traces all I/O operations to the physical disk drive c3t98d0.

●

vxtrace hist2 Traces all virtual device I/O operations associated with the volume hist2.

●

vxtrace -o dev hist2 Traces virtual disk device I/O to the device associated with volume hist2.

After identifying a volume that has an I/O-related problem, you can use the vxtrace command to determine which system process is responsible for the I/O requests. The volume of interest in this example is named ctrl. # vxtrace -o dev ctrl 40122 START write vdev ctrl block 16 len 40122 END write vdev ctrl op 40122 block 40123 START write vdev ctrl block 16 len 40123 END write vdev ctrl op 40123 block 40124 START write vdev ctrl block 16 len 40124 END write vdev ctrl op 40124 block 40125 START write vdev ctrl block 16 len 40125 END write vdev ctrl op 40125 block ^C

4 concurrency 16 len 4 time 4 concurrency 16 len 4 time 4 concurrency 16 len 4 time 4 concurrency 16 len 4 time

1 1 1 2 1 4 1 0

pid 10689 pid 10689 pid 10689 pid 10689

# ps -ef |grep 10689 oracle 10689 1 0 20:05:21 ? 0:03 ora_ckpt_bench

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-11

RAID-5 Write Performance

RAID-5 Write Performance The RAID-5 write process is controlled according to how much data is going to be written into a full stripe width. The optimum write performance is obtained when full stripes are written.

Read-Modify-Write Operations When less than 50 percent of the data disk drives are undergoing write operations in a single I/O, the read-modify-write sequence is used. This is the default operation for RAID-5 volumes. As shown in Figure 8-6, the read-modify-write sequence involves several steps, including: 1.

The stripes to be modified are read into a buffer.

2.

The parity information is read into a buffer.

3.

Exclusive OR (XOR) operations are performed.

4.

The new data and parity are written in a single write operation. New Data 1

1

1

1

0

XOR

1

0

1

1

0

1

Stripe Unit 0

Stripe Unit 1

Stripe Unit 2

Stripe Unit 3

Stripe Unit 4

Parity

Figure 8-6

8-12

Read-Modify-Write Operation

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID-5 Write Performance At least three I/O operations are necessary in the example shown in Figure 8-6 on page 8-12. Also, additional XOR calculations are necessary to account for the data in stripe units 2, 3, and 4 that was not read. Generally, the read-modify-write method is the least efficient way of writing to RAID-5 structures.

Reconstruct-Write Operations If more than 50 percent of the data stripe will be modified, use the reconstruct-write method. As shown in Figure 8-7, the reconstruct-write method involves different steps, including: 1.

Only unaffected data is read into a buffer.

2.

XOR is applied to the new data and the unaffected data.

3.

New parity and data are written in a single write operation. New Data 0

1

1

0

XOR 0

1

1

0

1

0

1

1

0

1

Stripe Unit 0

Stripe Unit 1

Stripe Unit 2

Stripe Unit 3

Stripe Unit 4

Parity

Figure 8-7

0

Reconstruct-Write Operation

Only two I/O operations are necessary in the example shown in Figure 8-7. Generally, the reconstruct-write operation is more efficient than the read-modify-write sequence.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-13

RAID-5 Write Performance

Full-Stripe Write Operations When large write operations that cover an entire data stripe are issued, the read-modify-write and reconstruct-write procedures are bypassed in favor of a full-stripe-write operation. A full-stripe write operation is faster than the other RAID-5 write procedures because it does not require any read operations. As shown in Figure 8-8, a full-stripe write operation consists of the following steps: 1.

XOR is applied to the new data to produce new parity.

2.

The new data and parity are written in a single write operation. New Data 0

1

0

0

1

XOR 0

1

1

0

1

0

1

0

1

1

0

1

Stripe Unit 0

Stripe Unit 1

Stripe Unit 2

Stripe Unit 3

Stripe Unit 4

Parity

Figure 8-8

Full-Stripe-Write Operation

Only a single write operation is necessary in the example shown in Figure 8-8. Note – In some cases, it is beneficial to reduce the number of RAID-5 columns to force more full-stripe write operations. This reduction can enhance overall write performance for some applications that use random-length write operations.

8-14

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Exercise: Demonstrating Performance Differences In this exercise, you complete the following tasks: ●

Review key lecture points

●

Perform a RAID-5 write performance test

●

Perform a striped volume write performance test

Preparation Unless your instructor says otherwise, the instructor conducts the performance demonstrations on the VxVM server system. First complete the questions in ‘‘Task 1 – Reviewing Key Lecture Points’’ on page 8-16.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-15

Exercise: Demonstrating Performance Differences

Task 1 – Reviewing Key Lecture Points Answer the following questions about performance planning and monitoring. 1.

What are the two most important areas of interest when researching volume placement? a.

Volume structure

b.

Volume activity levels

c.

Volume size

d.

Volume log locations

The answers are b and d.

2.

Why is it important to leave a small amount of free space on each disk drive? a.

To relocate the private region

b.

To adjust cylinder boundaries

c.

To relocate log space

d.

To create snapshot volumes

The answer is c.

3.

Which RAID structure usually has the best write performance? a.

RAID 5

b.

Mirrored

c.

Concatenated

d.

Striped

The answer is d.

4.

When is the preferred-plex read policy used? a.

When a RAID-5 stripe is on a faster disk drive

b.

When one mirror has better read performance

c.

When no mirror has a performance advantage

The answer is b.

8-16

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences 5.

Which of the following commands sets read policies? a.

vxedit

b.

vxvol

c.

vxdisk

The answer is b.

6.

What are the advantages of mirroring across controllers? a.

Availability

b.

Performance

c.

Administration

d.

Tuning

The answers are a and b.

7.

What is the primary VxVM command for displaying performance statistics? a.

vxtrace

b.

vxtask

c.

vxstat

The answer is c.

8.

What should be done prior to starting a VxVM volume performance test? a.

Clear the VxVM statistics

b.

Empty the system logs

c.

Initialize the syslogd daemon

The answer is a.

9.

What data pattern gives the best RAID-5 write performance? a.

Less than 30 percent stripe-width writes

b.

Near 80 percent stripe-width writes

c.

Full stripe-width writes

The answer is c.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-17

Exercise: Demonstrating Performance Differences

Task 2 – Performing a RAID-5 Write Performance Test The following demonstration is performed by the instructor unless you have been given other directions. The performance test uses the dd command to write directly to an unmounted raw VxVM volume. This demonstration is designed to be performed on a 30-Mbyte RAID-5 5+0 volume named r5vol that is in a disk group named dgX, and a 5-column striped volume named stdemo that is also 30 Mbytes in size. There is a lab demonstration file named r5demo.sh that is used to perform a series of performance tests. Alternately, you can enter the command manually to perform the performance test.

Create the Demonstration Volume and Test Data Complete the following steps: 1.

Log in to the VxVM server as user root.

2.

Complete the following steps:

3.

a.

Create a disk group named dgX containing a minimum of 5 disks.

b.

Use the vxdiskadm utility to ensure the disk type is sliced.

Create a 5-column, no log, RAID-5 volume with 30 Mbytes. # vxassist -g dgX make r5demo 30m \ layout=raid5,nolog \ dgX01 dgX02 dgX03 dgX04 dgX05

Note – The default stripe unit size is 32 blocks (16 Kbytes). 4.

Check for space, and then create a 20-Mbyte test file. # df -kl / # mkfile 20m /testfile

8-18

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Enable Classroom-Wide Display Complete the following steps: Note – In this task, all steps are directed toward the instructor. The words you and your refer to the instructor, not the students. The instructor should complete the following steps: 1.

Verify that the students remotely log in to the VxVM server.

2.

Verify that each workgroup gives you their terminal identifier by typing the tty command. Record the identifiers here: __________ __________ __________ __________ __________ __________

3.

Direct the output of your window to all of the students terminals. For example:

# script /dev/null | tee /dev/pts/2 | tee /dev/pts/3

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-19

Exercise: Demonstrating Performance Differences

Perform a RAID 5 Volume Demonstration Complete the following steps: 1.

Run the r5demo.sh demonstration test file on the VxVM server. You can also manually repeat the following command sequence, each time incrementing the block size of the dd command in the following sequence: 17, 33, 49, 65, 80, 64.

# vxstat -g dgX -r r5demo # /usr/proc/bin/ptime dd if=/testfile \ of=/dev/vx/rdsk/dgX/r5demo bs=17k # vxstat -g dgX -f MWF -v r5demo

Note – As you move past the 50-percent stripe write into full-stripe write, the I/O should move through the three write categories (M, W, and F). The full stripe width is 16 Kbytes times 5 columns, which is equal to 80 Kbytes. Tell the class that a full-stripe write is only four of the five columns. One column is always used for parity. This is evident in the last test loop using 64-Kbyte transfer size.

The vxstat -f MWF options have the following meanings:

2.

8-20

M

Read-modify-write statistics

W

Reconstruct write statistics

F

Full-stripe write statistics

Kill the tee processes to end the demonstration. Do not kill the script process, it terminates after the tee processes are gone.

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Demonstrating Performance Differences

Task 3 – Performing a Striped Volume Write Performance Test If time permits, create a second volume in a striped format with 5 columns, and run the performance test again. Use the same test file data. Complete the following steps: 1.

Use the vxassist command to create a 5-column striped volume. # vxassist -g dgX make stdemo 30m \ layout=striped \ dgX01 dgX02 dgX03 dgX04 dgX05

2.

Run the r5demo.sh script again, but substitute the name and path of the new striped volume, stdemo.

Enter disk group name (default: dgX) Enter the name of the demo volume (default: r5demo) stdemo Enter the raw path to the demo volume (default: /dev/vx/rdsk/dgX/r5demo) /dev/vx/rdsk/dgX/stdemo Enter data file location (default: /testfile)

Note – The vxstat command does not display any statistics for the striped volume, but the ptime results are informative.

VERITAS Volume Manager Performance Management Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

8-21

Exercise Summary

Exercise Summary

!

Discussion – Take a few minutes to discuss what experiences, issues, or discoveries you had during the lab exercises.

?

Manage the discussion here based on the time allowed for this module, which was given in the “About This Course” module. If you find you do not have time to spend on discussion, highlight just the key concepts students should have learned from the lab exercise. ●

Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots or especially confusing areas at this time. ●

Interpretations

Ask students to interpret what they observed during any aspects of this exercise. ●

Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience. ●

Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.

8-22

VERITAS Volume Manager 4.0 Administration Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D