Vxvm
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Vxvm as PDF for free.
More details
- Words: 7,124
- Pages: 237
VERITAS Volume Manager Basic Administration and Trouble shooting by ES Support, Chennai
What is Volume Manger?
VERITAS Volume Manager (VxVM) is an online storage management tool that provides a logical volume management layer which overcomes the physical restrictions of hardware disk devices by spanning volumes across multiple spindles.
What is Volume Manger?
Through the support of RAID redundancy techniques, VxVM protects against disk and hardware failures, while providing the flexibility to extend the capabilities of existing hardware.
Benefits of VERITAS Volume Manager
Manageability
Availability
Performance
Scalability
Types of Objects
Physical Storage Objects
Logical or Virtual Objects
Physical Storage Objects
Logical or Virtual Objects
Disk Groups
VM Disks
Sub Disks
Plexes
Volumes
How Volume Manager Works?
With Volume Manager, you enable virtual data storage by bringing a disk under Volume Manager control. To bring a disk under Volume Manager control means that Volume Manager creates virtual objects and establishes logical connections between those objects and the underlying physical objects, or disks.
How Volume Manager Works?
How Volume Manager Works?. . .
1. Volume Manager removes all of the partition table entries from the VTOC, except for partition table entry 2 (backup slice). Partition table entry 2 contains the entire disk, including the VTOC, and is used to determine the size of the disk.
How Volume Manager Works?. . .
2. Volume Manager then rewrites the VTOC and creates two partitions on the physical disk. One partition contains the private region, and the other contains the public region.
How Volume Manager Works?. . .
Private region: The private region stores information, such as disk headers, configuration copies, and kernel logs, that Volume Manager uses to manage virtual objects. The private region represents a small management overhead. The minimum size for the private region is 1024 sectors (512K) for disks with active configuration databases, but VxVM uses 2048 sectors (1024K) by default. This default value is rounded up to the next cylinder boundary.
How Volume Manager Works?. . .
Partition Tags: VxVM sets the partition tags, the numeric values that describe the file system mounted on a partition, for the public and private regions:
Tag 14 is always used for the public region of the disk. Tag 15 is always used for the private region of the disk.
If the disk has no partitions that are being placed under Volume Manager control, then Volume
How Volume Manager Works?. . .
Volume Manager updates the VTOC with information about the removal of the existing partitions and the addition of the new partitions during the initialization process.
Summary of Virtual Object Relationships
Summary of Virtual Object Relationships . . .
Volume Layouts
Planning a First-Time VxVM Setup
to place under Volume Manager control?
Do you want to use enclosure-based naming?
Do you want to exclude any disks from Volume Manager control?
Do you want to suppress dynamic multipathing on any disks?
When you place disks under Volume Manager control, do you want to preserve or eliminate data in existing file systems and partitions?
Manager control, do you want to preserve or eliminate data in existing file systems and partitions?. ..
When you place a disk under Volume Manager control, you can either preserve the data that exists on the physical disk (encapsulation) or eliminate all of the data on the physical disk (initialization).
Encapsulation
Saving the data on a disk brought under Volume Manager control is called disk encapsulation
Encapsulation . . .
Contain the required minimum unpartitioned free space of 1024 sectors (512K) (By default, VxVM uses 2048 sectors (1024K).)
Contain an s2 slice that represents the full disk ( The s2 slice cannot contain a file system.)
Encapsulation . . .
The partitions are converted to subdisks that are used to create the volumes that replace the Solaris partitions.
Initialization
Eliminating all of the data on a physical disk brought under Volume Manager control is called disk initialization
Any disks that are encapsulated or initialized during installation are placed in the disk group rootdg. If disks are left alone during installation, they can be placed under Volume Manager control later and assigned to disk
Do you want to place the system root disk under Volume Manager control?
Existing /, /usr, and /var partitions are converted to volumes without removing the partitions. Other partitions are converted to volumes, and then partitions are removed. The existing swap area is converted to a volume. If there is insufficient space for the private region on the boot disk, Volume Manager takes sectors from the swap area of the disk, which makes the private region overlap the public region. The swap partition remains the same size, and the swap volume is resized to
Note: Volume Manager preserves a copy of the original VTOC of any disk that is encapsulated in /etc/vx/reconfig.d/disks.d/cx tydz /vtoc, where cxtydz is the SCSI address of the disk.
Typical Initial VxVM Setup
VxVM Licensing
# vxlicense –c license a key
;Adding a
# vxlicense –p ;Viewing installed license keys
VxVM Standard Packages
VRTSvxvm ;Driver & Utilities VRTSlic ;Licensing VRTSvmdoc ;Documentation VRTSvmman ;Manual Pages VRTSvmdev ;Developer kit VRTSvmsa ;Storage Admin (GUI)
Steps to Add VxVM Packages
# # # # # #
pkgadd pkgadd pkgadd pkgadd pkgadd pkgadd
-d -d -d -d -d -d
. . . . . .
VRTSlic VRTSvxvm VRTSvmsa VRTSvmdoc VRTSvmman VRTSvmdev
The vxinstall Program
# vxinstall ;to configure VxVM after package installation
The vxinstall Process
The vxinstall Process. . .
VxVM User Interfaces
Volume Manager Storage Administrator (VMSA)
Command Line Interface (CLI)
Volume Manager Support Operations (vxdiskadm)
Monitoring the Command Log
# tail -f /var/opt/vmsa/logs/command
CLI Directories
/etc/vx/bin
/usr/sbin
/usr/lib/vxvm/bin
Managing Disks
Placing a Disk Under Volume Manager Control
Placing a Disk Under Volume Manager Control. . .
Placing a Disk Under Volume Manager Control . ..
# vxdisksetup -i c1t0d0
The -i option writes a disk header to the disk, making the disk directly usable
# vxdg -g newdg adddisk newdg02=c2t0d0
Evacuating a Disk
Evacuating a disk moves the contents of the volumes on a disk to another disk. The contents of a disk can be evacuated only to disks in the same disk group that have sufficient free space. You must evacuate a disk if you plan to
# vxevac -g datadg datadg02 datadg03
Removing a disk
If a disk is no longer needed in a disk group, you can remove the disk. After you remove a disk from a disk group, the disk cannot be accessed. When removing a disk from a disk group, you have two options: Move the disk to the free disk pool. With this option, the disk remains under Volume Manager control. Send the disk back to an uninitialized state. With this option, the disk is no longer under Volume Manager control. You cannot remove the last disk in a disk group. To remove the last disk in a disk
# vxdg -g newdg rmdisk newdg02
vxdiskunsetup –C c1t0d0
Rename the disk
# vxedit -g datadg rename datadg01 datadg03
Managing Disk Group
A disk group is created when you place at least one disk in the disk group. When you add a disk to a disk group, a disk group entry is added to the private region header of that disk. Because a disk can only have one disk group entry in its private region header, one disk group does not "know about" other disk groups, and therefore disk groups cannot share resources, such as disk drives, plexes, and volumes. A volume with a plex can belong to only one disk group, and subdisks and plexes of a
When you add a disk to a disk group, VxVM assigns the disk media name to the disk and maps this name to the disk access record. In addition, the host name is also recorded in the private region. This information is written to the
Disk media name: A disk media name is the logical disk name assigned to a drive by VxVM. VxVM uses this name to identify the disk for volume operations, such as volume creation and mirroring.
Disk access record: A disk access record is a record of how a disk maps to a physical location and represents the UNIX path to the device. Disk access records are dynamic and can be recreated when vxdctl enable is run.
Once disks are placed under Volume Manager control, storage is managed in terms of the logical configuration. File systems mount to logical volumes, not to physical partitions. Logical names, such as /dev/vx/[r]dsk/diskgroup_name/volum e, replace physical locations, such as /dev/[r]dsk/c0t4d2s5
Whenever the VxVM configuration daemon is started (or vxdctl enable is run), the system reads the private region on every disk and establishes the connections between disk access records and disk media names.
The rootdg Disk Group
The rootdg disk group is a special disk group that is created when you install VxVM during the vxinstall process. VxVM requires that the rootdg disk group exist and that it contain at least one disk. It is recommended that at least two disks are in the rootdg disk group so that the VxVM configuration database can be maintained on at least two disks. If you want your
Disk Groups and High Availability
Creating Disk Group
# vxdg init newdg \ newdg01=c1t1d0s2
Designating a Disk As a Hot-Relocation Spare
vxedit -g diskgroup set spare=on|off datadg02
Making a Disk Group Unavailable
# umount /filesystem1
# umount /filesystem2
# vxdg deport newdg
# vxdg -n newerdg deport newdg
# vxdg -h server1 deport newdg
Importing a Deported Disk Group
# vxdg import newdg
# vxdg -n newerdg import newdg
# vxdg -f import newdg
Moving Disk Groups between Systems
Renaming a Disk Group
# vxdg -n mktdg deport datadg # vxdg import mktdg # vxvol -g mktdg startall (or)
# vxdg deport datadg # vxdg -n mktdg import datadg # vxvol -g mktdg startall
Destroying a Disk Group
# vxdg destroy newdg
Viewing Disk Group Information
Upgrading the Disk Group Version
#vxdg upgrade datadg
Selecting a Volume Layout
Concatenated Layout
Concatenation: Advantages
Removes size restrictions: Concatenation removes the restriction on size of storage devices imposed by physical disk size.
Better utilization of free space: Concatenation enables better utilization of free space on disks by providing for the ordering of available discrete disk space on multiple disks into a single addressable volume.
Simplified administration:
Concatenation: Disadvantages
No protection against disk failure: Concatenation does not protect against disk failure. A single disk failure may result in the failure of the entire volume.
Striped Layout
Striping: Advantages
Parallel data transfer: Striping is useful if you need large amounts of data written to or read from the physical disks quickly by using parallel data transfer to multiple disks.
Load balancing: Striping is also helpful in balancing the I/O load from multiuser applications across multiple disks.
Improved performance: Improved performance is obtained by increasing the effective bandwidth of the I/O path to the data. This may be achieved by a single volume I/O operation spanning across
Striping: Disadvantages
No redundancy: Striping alone offers no redundancy or recovery features.
Disk failure: Striping a volume increases the chance that a disk failure will result in failure of that volume. For example, if you have three volumes striped across two disks, and one of the disks is used by two of the volumes, then if that one disk goes down, both volumes will go down.
Mirrored Layout
Mirroring: Advantages
Improved reliability and availability: With concatenation or striping, failure of any one disk can make the entire plex unusable. With mirroring, data is protected against the failure of any one disk. Mirroring improves the reliability and availability of a striped or concatenated volume.
Improved read performance: Reads benefit from having multiple places from
Mirroring: Disadvantages
Requires more disk space: Mirroring requires twice as much disk space, which can be costly for large configurations. Each mirrored plex requires enough space for a complete copy of the volume's data.
Slightly slower write performance: Writing to volumes is slightly slower, because multiple copies have to be written in parallel. The overall time the write operation takes is determined by the time needed to write to the slowest disk
RAID-5
RAID-5: Advantages
Redundancy through parity: With a RAID-5 volume layout, data can be re-created from remaining data and parity in case of disk failure. Requires less space than mirroring: RAID-5 stores parity information, rather than a complete copy of the data. Improved read performance: RAID-5 provides similar improvements in read performance as in a normal striped layout. Fast recovery through logging: RAID5 logging minimizes recovery time in case
RAID-5: Disadvantages
Slow write performance: The performance overhead for writes can be substantial, because a write can involve much more than simply writing to a data block. A write can involve reading the old data and parity, computing the new
Creating a Concatenated Volume
# vxassist -g datadg make datavol 10g
# vxassist -g datadg make datavol layout=nostripe 10g
# vxassist -g datadg make datavol 10g datadg02 datadg03
Creating a Striped Volume
# vxassist -g acctdg make payvol 20m layout=stripe
# vxassist -g acctdg make expvol 20m layout=stripe ncol=3 stripeunit=64K acctdg01 acctdg02 acctdg03
Creating a RAID-5 Volume
# vxassist -g acctdg make expvol 20m layout=raid5
# vxassist -g acctdg make expvol 20m layout=raid5 stripeunit=32K acctdg01 acctdg02 acctdg03 acctdg04
Creating a Mirrored Volume
# vxassist -g datadg make datavol 5g layout=mirror
# vxassist -g datadg make datavol 5g layout=stripe,mirror
# vxassist -g datadg -b make datavol 5g layout=stripe,mirror nmirror=3
Estimating Volume Size
# vxassist -g datadg maxsize
# vxassist -g datadg maxsize layout=raid5
vxassist -g datadg maxgrow datavol
Displaying Volume information
# # # # # #
vxprint vxprint vxprint vxprint vxprint vxprint
–Aht –g rootdg –dt –st –pt -vt
Removing a Volume
Removing a Volume . . .
# vxassist -g datadg remove volume datavol
# vxedit –g datadg –rf rm datavol
Adding Mirror
Adding Mirror . . .
# vxassist -g datadg mirror datavol
# vxassist -g datadg mirror datavol datadg03
vxmirror -g datadg -a
# /etc/vx/bin/vxmirror -d yes
# vxassist make nomirrorvol 20m nmirror=1
Mirroring across Controllers and Enclosures
# vxassist -g datadg make datavol 10g layout=mirror nmirror=2 mirror=ctlr ctlr:c2 ctlr:c3
# vxassist -g datadg make datavol 10g layout=mirror nmirror=2 mirror=enclr enclr:emc1 enclr:emc2
Removing a Mirror
# vxassist -g datadg remove mirror datavol datadg02 (or)
# vxplex -g datadg dis datavol-02
# vxedit -g datadg -rf rm \ datavol-02
Excluding Storage from Volume Creation
# vxassist -g datadg make datavol 5g !datadg05
# vxassist -g datadg make datavol 5g !ctlr:c2
# vxassist -g datadg make datavol 5g ctlr:c1 !target:c1t5
# vxassist -g datadg maxsize layout=raid5 nlog=2 !datadg07 !datadg08
Adding File System to a Volume
# newfs dev/vx/rdsk/datadg/datavol
# mount /dev/vx/dsk/datadg/datavol /data
Volume Maintenance . . .
Volume Maintenance . . .
Managing Volume Tasks
Managing Volume Tasks . . .
# # # # # #
vxtask -l list ;long format vxtask -h list ; hierarchically vxtask -p list ; paused state vxtask -r list ; running state vxtask -a list ; aborted state vxtask list convertop1 ;specific task tag
Monitoring a Task
This is same as vxtask list command and addition to that we can use additional option like –w intervel
Controlling the tasks
# vxtask pause 198
# vxtask resume 198
# vxtask abort 198
# vxtask abort convertop1
Disk Encapsulation
On a Solaris system VxVM uses the volume table of contents (VTOC) to determine disk size (partition 2), then creates two partitions on the physical disk:
One partition contains the private region. The private region stores VxVM information, such as disk headers, configuration copies, and kernel logs. Tag 15 is always associated with the private region. When a disk is encapsulated, tag 15 is always associated to a slice other than slice 3. The other partition contains the public region. The public region is used for storage space
Root Disk Encapsulation
Data Disk Encapsulation Requirements
At least two partition table entries must be available on the disk.
One partition is used for the public region. One partition is used for the private region.
A minimum of 1024 sectors of unpartitioned free space, rounded up to the nearest cylinder boundary, must be available either at the beginning or at the end of the disk. VxVM uses 2048 sectors
Boot Disk Encapsulation Requirements
Boot disk encapsulation has the same requirements as data disk encapsulation, with one important distinction: when encapsulating the root disk, the private region can be created from the swap area, which reduces the swap area by the size of the private region. The private region is created at the beginning of swap, and the swap partition
Encapsulating Root
Select disk devices to encapsulate: [<pattern-list>,all,list,q,?] c0t0d0 Which disk group [,list,q,?] (default: rootdg) Encapsulate other disks? [y,n,q,?] (default: n) n
VTOC: Before Encapsulating Root Disk
VTOC: After Encapsulating Root Disk
As part of the root encapsulation process, the /etc/system file is updated to include information that tells VxVM to boot up on the encapsulated volumes. The following two lines are added to the /etc/system file: rootdev:/pseudo/vxio@0:0 set
/etc/vfstab: Before Root Encapsulation
/etc/vfstab: After Root Encapsulation
Mirroring the Root Disk
The boot disk must be encapsulated by VxVM in order to be mirrored.
To mirror the root disk, you must provide another disk with enough space to contain all of the root partitions (/, /usr, /var, /opt, and swap).
You can only use disks in the rootdg disk group for the boot disk and alternate boot
Why Create an Alternate Boot Disk
# /etc/vx/bin/vxrootmir secrootmir
# vxassist mirror homevol secrootmir
# /etc/vx/bin/vxmirror rootdisk secrootmir
# prtconf -vp | grep bootpath
bootpath: '/sbus@3,0/SUNW,socal@d,100 00/sf@0,0/ ssd@w2100002037590098,0:a'
# prtconf -vp | grep vx vx-disk01: '/sbus@3,0/SUNW,socal@d,10000/ sf@0,0/ ssd@w21000020374fe71f,0:a' vx-rootdisk: '/sbus@3,0/SUNW,socal@d,10000/ sf@0,0/ ssd@w2100002037590098,0:a'
Unencapsulating a Root Disk
# vxunroot To convert the root, swap, usr, var, opt, and home file systems back to being accessible directly through disk partitions instead of through volume devices, you use the vxunroot utility. Other changes that were made to ensure the booting of the system from the root
For vxunroot to work properly, the following conditions must be met: All but one plex of rootvol, swapvol, usr, var, opt, and home must be removed (using vxedit or vxplex). One disk in addition to the root disk must exist in rootdg.
To convert a root volume back to partitions:
Ensure that the rootvol, swapvol, usr, and var volumes have only one associated plex each. The plex must be contiguous, nonstriped, nonspanned, and nonsparse.
# vxprint -ht rootvol swapvol usr var
# vxplex -o rm dis plex_name
# /etc/vx/bin/vxunroot
This command changes the volume entries in /etc/vfstab to the underlying disk partitions for the rootvol, swapvol, usr, and var volumes. The command also modifies /etc/system and prompts for a reboot so that disk partitions are mounted instead of volumes for the root, swap, usr, and var
VxVM Architecture
VxVM is a device driver that is placed between the UNIX operating system and the SCSI device drivers. When VxVM is installed, UNIX invokes the VxVM device drivers instead of the SCSI device drivers. VxVM determines which SCSI drives are involved in the requested I/O and delivers the I/O request to those
VxVM Daemons
vxconfigd When a system is booted, the command vxdctl enable is automatically executed to start the VxVM configuration daemon, vxconfigd. VxVM reads the /etc/vx/volboot file to determine disk ownership, then automatically imports rootdg and all other disk groups owned by this host. vxconfigd reads the kernel log to determine the state of VxVM objects. vxconfigd reads the configuration database on the disks, then uses the kernel log to update the state information
VxVM Daemons . . .
vxiod vxiod—VxVM I/O kernel threads provide extended I/O operations without blocking calling processes. By default, 10 I/O threads are started at boot time, and at least one I/O thread must continue to run at all times.
VxVM Daemons . . .
vxrelocd vxrelocd is the hot relocation daemon that monitors events that affect data redundancy. If redundancy failures are detected, vxrelocd automatically relocates affected data from mirrored or RAID5 subdisks to spare disks or other free space within the disk group. vxrelocd also notifies the system
V M Disks – Private Region
The disk header contains the disk label, disk group information, host ID, and pointers to the private and public regions. You can display disk header information by using vxdisk list diskname. The configuration database contains VxVM object definitions. The size of the configuration database is approximately 70 percent of the private region. Kernel logs contain configuration changes, including information about log
Types of VxVM Disks
A simple disk is a disk that is created dynamically in the kernel and has public and private regions that are contiguous inside a single partition. A sliced disk is a disk that has separate slices for the public and private regions. A NOPRIV disk is a disk that does not contain a private region
VxVM Configuration Database
The VxVM configuration database stores all disk, volume, plex, and subdisk configuration records. The vxconfig device (/dev/vx/config) is the interface through which all changes to the volume driver state are performed. This device can only be opened by one process at a time, and the initial volume configuration is downloaded into the kernel
The configuration database is stored in the private region of a VxVM disk.
The VxVM configuration is replicated within the disk group so that sufficient copies exist to protect against loss of the configuration in case of physical disk failure. VxVM attempts to store at least four copies for each disk group.
Displaying Disk Group Configuration Data
To display the status of the configuration database for a disk group
# vxdg list acctdg
In the output, there are four disks that have configuration databases that are active (online), and there is one disk that does not have an active copy of the data (disabled).
The size of the configuration database for a disk group is the size of the smallest private region in the disk group. In the example, permlen=2630.
Log entries are on all disks that have databases. The log is used by the VxVM kernel to keep the state of the drives
Displaying Disk Configuration Data
# vxdisk –g newdg list newdg01
Term Device devicetag type hostid controlling this
Description Full UNIX device name of disk Device name used by VxVM to reference the physical disk Method of placing disk under VxVM controlDefault is sliced. Name of system that manages the disk groupIf blank, no host is currently
group disk name VM disk media name and internal ID group name Disk group name and internal ID flags Settings that describe status and options for the disk Pubpaths Paths for block and character device files of public region of disk Privpaths Paths for block and character device files of private region of disk version Version number of header format iosize The size of I/O to private region public, private slices Partition (slice) number, offset from beginning of partition, and length of partition (public offset=0, private offset=1) update time Date, time, and sequence number of last update to private region headers Offset to two copies of the private region header configs count Number of config database copies kept in the private region logs count Number of kernel logs kept in the private region Defined regions Location of configuration database(s) and kernel logs in the private regionBecause the database or logs can be split, there could be multiple pieces. Therefore, the offset is the starting location within the private region where this piece of the database begins. Copy represents which copy of the database that this piece is a part of. Note: There are "multiple pieces" of the config database because one is READ-ONLY and the other is READ-WRITE. When you perform tasks in VxVM, only the RW piece's data changes. The other piece contains the control files for the config database. Multipathing information If dynamic multipathing is enabled and there are multiple paths to the disk,
How Does vxconfigd Work?
The VxVM configuration daemon must be running in order for configuration changes to be made to the VxVM configuration database. If vxconfigd is not running, VxVM will operate properly, but configuration changes are not allowed.
vxconfigd Modes
Enabled Enabled is the normal operating mode in which most configuration operations are allowed. Disk groups are imported, and VxVM begins to manage device nodes stored in /dev/vx/dsk and /dev/vx/rdsk.
Disabled In the disabled mode, most operations are not allowed. vxconfigd does not retain configuration information for the imported disk groups and does not maintain the volume and plex device directories. Certain failures, most commonly the loss of all disks or configuration copies in the rootdg disk group, will cause vxconfigd to enter the disabled state automatically.
Booted The booted mode is part of normal system startup, prior to checking the root file system. The booted mode imports the rootdg disk group and waits for a request to enter the enabled mode. Volume device node directories are not maintained, because it may not be possible to write to the root file system.
The vxdctl Utility
vxconfigd is invoked by startup scripts during the boot procedure. To manage some aspects of vxconfigd, you can use the vxdctl utility.
# vxdctl mode mode: enabled This command displays the status of the configuration daemon. If the configuration daemon is not running, it must be started in order to make configuration changes. Disk failures are also configuration changes, but there is another way of tracking them if the daemon is down (kernel logs).
If vxconfigd is running, but not enabled, the following message is displayed: mode: disabled To enable the configuration daemon, you type: # vxdctl enable
mode: not-running
# vxconfigd
Once started, vxconfigd automatically becomes a background process.
# vxdctl stop
To send a kill -9 to vxconfigd:
# vxdctl -k stop
# vxdctl disable
To prevent configuration changes from occurring, but to allow administrative commands to be used, you can disable the daemon
# vxdctl license [init]
To display the list of VxVM features which are currently available based on known licensing information
This command checks /etc/vx/elm files in order to display currently licensed features
By adding the init argument, you can request that vxconfigd reread any persistently stored license information. If licenses have expired, some features may become unavailable. If new
The volboot File
The /etc/vx/volboot file contains a host ID that is used by VxVM to establish ownership of physical disks. This host ID is used to ensure that two or more hosts that can access disks on a shared SCSI bus will not interfere with each other in their use of those disks. This host ID is important in the generation of unique ID strings
The volboot file also contains a list of disks to scan in search of the rootdg disk group. At least one disk in this list must be both readable and a part of the rootdg disk group, or VxVM will not be able to start up correctly.
Never edit the volboot file manually. If you do so, its checksum will be invalidated.
To view the decoded contents of the volboot file
# vxdctl list
Changing the Host ID
If you change your host name in UNIX, you need to change your host ID in the volboot file. To change the host ID in the volboot file and on all disks in disk groups currently imported on the machine # vxdctl hostid hostid # vxdctl enable
If some disks are inaccessible at the time of a hostid operation, it may be necessary to use the vxdisk clearimport operation to clear out the old host ID on those disks when they become reaccessible. Otherwise, you may not be able to re-add those disks to their disk groups.
Caution: Be careful when using this command. If the system crashes before the hostid operation completes, some disk groups may not reimport automatically
Re-Creating the volboot File
To re-create the volboot file because it was removed or invalidated
# vxdctl init [hostid]
If a hostid operand is specified, then this string is used; otherwise, a default host ID is used. The default host ID is the network node name for the host
Introduction to Recovery
What Is Resynchronization ?
Minimizing the Impact of Resynchronization
DRL - Dirty region logging for Mirror Logging b – for RAID 5
DRL
How Does DRL Work? DRL logically divides a volume into a set of consecutive regions and keeps track of the regions to which writes occur. A log is maintained that contains a status bit representing each region of the volume. For any write operation to the volume, the regions being written are marked dirty in the log before the data is written. If a write causes a log region to become dirty when it was previously clean, the log is synchronously written to disk before the write operation can occur. On system restart, VxVM recovers only those regions of the volume that
RAID 5 Logging
RAID-5 volumes use RAID-5 logs to keep a copy of the data and parity currently being written.
Without logging, data not involved in any active writes can be lost or silently corrupted if both a disk in a RAID-5 volume and the system fail.
If this double-failure occurs, there is no way of knowing if the data being written to the data portions of the disks or the parity being written to the parity portions have actually been
RAID-5 logging is used to prevent corruption of data during recovery by immediately recording changes to data and parity to a log area. The new data and parity are then written to disk.
Logs are associated with a RAID-5 volume by being attached as log plexes. More than one log plex can exist for each RAID-5 volume, in which case the
Disk Failure
Permanent disk failure:
When a disk is corrupted and no longer usable, the disk must be logically and physically removed, and then replaced with a new disk. With permanent disk failure, data on the disk is lost. Example: VTOC is damaged.
Temporary disk failure:
Temporary disk failure: When communication to a disk is interrupted, but the disk is not damaged, the disk can be logically removed, then reattached as the replacement disk. With temporary (or intermittent) disk failure, data still exists on the disk. Example: Power is disrupted.
Impact of Disk Failure
VxVM is designed to protect your system from the impact of disk failure through a feature called hot relocation. The hot-relocation feature of VxVM automatically detects disk failures and restores redundancy to failed VxVM objects by moving subdisks from failed disks to other disks.
Note: Hot relocation is only performed for redundant (mirrored or RAID-5) subdisks on a failed disk. Nonredundant subdisks on a failed disk are not relocated, but the system administrator is notified of the failure.
Partial Disk Failure
When a partial disk failure occurs (that is, a failure affecting only some subdisks on a disk), redundant data on the failed portion of the disk is relocated. Existing volumes on the unaffected portions of the disk remain accessible. With partial disk failure, the disk is not removed from VxVM control and is labeled as FAILING, rather than as FAILED. Before removing a FAILING disk for replacement, you must evacuate any remaining volumes on the disk.
How Does Hot Relocation Work?
The vxrelocd daemon starts during system startup and monitors VxVM for failures involving disks, plexes, or RAID-5 subdisks. When a failure occurs, vxrelocd triggers a hotrelocation attempt and notifies the system administrator, through email, of failures and any relocation and recovery actions.
The vxrelocd daemon is started from the S95vxvm-recover file. The argument to vxrelocd is the list of people to e-mail notice of a relocation (default is root). To disable vxrelocd, you can place a "#" in front of the line in the S95vxvm-recover file.
The hot-relocation feature is enabled by default. No system administrator action is needed to start hot relocation when a failure occurs.
A successful hot-relocation process involves
Failure detection: Detecting the failure of a disk, plex, or RAID-5 subdisk Notification: Notifying the system administrator and other designated users and identifying the affected Volume Manager objects Relocation: Determining which subdisks can be relocated, finding space for those subdisks in the disk group, and relocating the subdisks (The system administrator is notified of the success or failure of these actions. Hot relocation does not guarantee the same layout of data or the same performance after relocation.) Recovery: Initiating recovery procedures, if necessary, to restore the volumes and data (Again, the system administrator is notified of
Setting Up Spare Disks
vxedit -g datadg set spare=on datadg03
vxedit -g datadg set spare=off datadg03
vxedit -g datadg set nohotuse=on datadg04
vxedit -g datadg set nohotuse=off datadg04
Including Spare Disks in Space Availability
# vxassist -g mydg -r maxsize layout=stripe ncolumns=3
To include spare disks when determining how much space is available using the maxsize or maxgrow options, you add the -r flag in the vxassist command
Reserving Disks
A spare disk is not the same as a reserved disk. You can reserve a set of disks for special purposes, such as to avoid general use of a particularly slow or a particularly fast disk
# vxedit set reserve=on datadg05
# vxedit set reserve=off datadg05
After you type this command, vxassist does not allocate space from the selected disk unless that disk is specifically mentioned on the vxassist command line.
# vxassist make vol03 20g datadg05
overrides the reservation and creates a 20-MB volume on datadg05. However, the command
# vxassist make vol04 20g
does not use datadg05, even if there is no free space on any other disk.
Disk Replacement Tasks
Replacing a failed or corrupted disk involves two main operations :
1. Disk replacement When a disk fails, you replace the corrupt disk with a new disk. The disk used to replace the failed disk must be either an uninitialized disk or a disk in the free disk pool. The replacement disk cannot already be in a disk group. If you want to use a disk that exists in another disk group, then you must remove the disk from the disk group and place it back into the free disk pool
2. Volume recovery When a disk fails and is removed for replacement, the plex on the failed disk is disabled, until the disk is replaced. Volume recovery involves:
Starting disabled volumes
Note: A volume remains started, and does not need to be restarted, if it has a RAID-5 or mirrored layout, that is, if the volume has one remaining active plex.
Resynchronizing mirrors Resynchronizing RAID-5 parity
After successful recovery, the volume is available for use again. Redundant (mirrored or RAID-5) volumes can be recovered by VxVM. Nonredundant (unmirrored) volumes must be restored from backup.
Adding a New Disk
Before VxVM can use a new disk, you must ensure that Solaris recognizes the disk. When adding a new disk, follow these steps to ensure that the new disk is recognized:
1.Connect the new disk. 2. Get Solaris to recognize the disk: # drvconfig # disks Note: In Solaris 7 and later, use devfsadm, a onecommand replacement for drvconfig and disks. 3. Verify that Solaris recognizes the disk: # prtvtoc /dev/dsk/device_name 4. Get VxVM to recognize that a failed disk is now working again: # vxdctl enable 5. Verify that VxVM recognizes the disk: # vxdisk list After Solaris and VxVM recognize the new disk, you can then use the disk as a replacement disk.
Replace a failed disk
Select a removed or failed disk [,list,q,?] datadg02
The following devices are available as replacements: c1t0d0s2 c1t1d0s2 Choose a device, or select "none" [<device>,none,q,?] (default: c1t0d0s2)
After you confirm the operation, the following status message is displayed:
Replacement of disk datadg02 in group datadg with disk device c1t0d0s2 completed successfully.
Replacing a Disk: CLI
# vxdg -k -g datadg adddisk datadg01=c1t1d0s2
The -k switch forces VxVM to take the disk media name of the failed disk and assign it to the new disk
For example, if the failed disk datadg01 in the datadg disk group was removed, and you want to add the new device
Note: Exercise caution when using the -k option to vxdg. Attaching the wrong disk with the -k option can cause unpredictable results in VxVM
The vxunreloc Utility
The hot-relocation feature detects I/O failures in a subdisk, relocates the subdisk, and recovers the plex associated with the subdisk.
VxVM also provides a utility that unrelocates a disk, that is, moves relocated subdisks back to their original disk. After hot relocation moves subdisks from a failed disk to other disks, you can
Unrelocation is performed using the vxunreloc utility, which restores the system to the same configuration that existed before a disk failure caused subdisks to be relocated.
Viewing Relocated Subdisks
When a subdisk is hotrelocated, its original disk media name is stored in the orig_dm_name field of the subdisk record files. You can search this field to find all the subdisks that originated from a failed disk using the vxprint command
For example, to display all the subdisks that were hot-relocated from datadg01 within the datadg disk group:
# vxprint -g datadg -se `sd_orig_dmname="datadg01"'
Unrelocating a Disk: vxdiskadm
Recovering a Volume
The vxreattach utility reattaches disks to a disk group and retains the same media name. This command attempts to find the name of the drive in the private region and to match it to a disk media record that is missing a disk access record. This operation may be necessary if a disk has a transient failure, for example, if a drive is turned off and then back on, or if the Volume
vxreattach tries to find a disk in the same disk group with the same disk ID for the disks to be reattached. The reattach operation may fail even after finding the disk with the matching disk ID if the original cause (or some other cause) for the disk failure still exists.
/etc/vx/bin/vxreattach [-bcr] [dm_name]
-b Performs the reattach operation in the background -c Checks to determine if a reattach is possibleNo operation is performed, but the disk group name and the disk media name at which the disk can be reattached are displayed. -r Attempts to recover stale plexes of any
The vxrecover Command
To perform volume recovery operations from the command line, you use the vxrecover command. The vxrecover program performs plex attach, RAID-5 subdisk recovery, and resynchronize operations for specified volumes (volume_name), or for volumes residing on specified disks (dm_name). You can run vxrecover
Note: The vxrecover command will only work on a started volume. A started volume displays an ENABLED state in vxprint -ht.
Recovery operations are started in an order that prevents two concurrent operations from involving the same disk. Operations that involve unrelated disks will run in parallel.
# vxrecover -bs -g datadg datadg01
After replacing the failed disk datadg01 in the datadg disk group, and adding the new disk c1t1d0s2 in its place, you can attempt to recover the volumes.
# vxrecover -b -g datadg datadg01
To recover, in the background, any detached subdisks or plexes that resulted from replacement of the disk datadg01 in the datadg disk group
# vxrecover -v -g datadg datadg01
To monitor the operations during the recovery
Protecting VxVM Configuration
Precautionary Tasks
To protect the VxVM configuration, you can perform two precautionary tasks: 1. Save a copy of the VxVM configuration using the vxprint command.
You can also use the vxprint command to save the VxVM configuration to a file and later use that file to recover removed volumes. By saving the output of the vxprint command to a file, you can then use the vxmake command with the saved file to restore the configuration if needed.
When saving the VxVM database configuration, you use the -m option. This option displays all information about each record in a format that is useful as input to the vxmake utility. To view the saved records, you can use the -D - option. This option reads a configuration from the standard
Saving the Configuration Database
# vxprint -g diskgroup -hmQqr > backup.DGname (or)
# vxprint -g diskgroup -hmvpsQqr > backup.DGname
Displaying a Saved Configuration
# vxprint -D - -rht < backup.DGname
Recovering a Lost Volume
# vxprint -D - -rhtmqQ lostvolume < backup.DGname > restoredvolume
This command creates object definitions for a restored volume out of the object definitions in the lost volume
To implement the object definitions of restoredvolume into a real volume
# vxmake -g diskgroup -d restoredvolume
To start the restored volume, and recover its plexes.
# vxrecover -Es restoredvolume
What is Volume Manger?
VERITAS Volume Manager (VxVM) is an online storage management tool that provides a logical volume management layer which overcomes the physical restrictions of hardware disk devices by spanning volumes across multiple spindles.
What is Volume Manger?
Through the support of RAID redundancy techniques, VxVM protects against disk and hardware failures, while providing the flexibility to extend the capabilities of existing hardware.
Benefits of VERITAS Volume Manager
Manageability
Availability
Performance
Scalability
Types of Objects
Physical Storage Objects
Logical or Virtual Objects
Physical Storage Objects
Logical or Virtual Objects
Disk Groups
VM Disks
Sub Disks
Plexes
Volumes
How Volume Manager Works?
With Volume Manager, you enable virtual data storage by bringing a disk under Volume Manager control. To bring a disk under Volume Manager control means that Volume Manager creates virtual objects and establishes logical connections between those objects and the underlying physical objects, or disks.
How Volume Manager Works?
How Volume Manager Works?. . .
1. Volume Manager removes all of the partition table entries from the VTOC, except for partition table entry 2 (backup slice). Partition table entry 2 contains the entire disk, including the VTOC, and is used to determine the size of the disk.
How Volume Manager Works?. . .
2. Volume Manager then rewrites the VTOC and creates two partitions on the physical disk. One partition contains the private region, and the other contains the public region.
How Volume Manager Works?. . .
Private region: The private region stores information, such as disk headers, configuration copies, and kernel logs, that Volume Manager uses to manage virtual objects. The private region represents a small management overhead. The minimum size for the private region is 1024 sectors (512K) for disks with active configuration databases, but VxVM uses 2048 sectors (1024K) by default. This default value is rounded up to the next cylinder boundary.
How Volume Manager Works?. . .
Partition Tags: VxVM sets the partition tags, the numeric values that describe the file system mounted on a partition, for the public and private regions:
Tag 14 is always used for the public region of the disk. Tag 15 is always used for the private region of the disk.
If the disk has no partitions that are being placed under Volume Manager control, then Volume
How Volume Manager Works?. . .
Volume Manager updates the VTOC with information about the removal of the existing partitions and the addition of the new partitions during the initialization process.
Summary of Virtual Object Relationships
Summary of Virtual Object Relationships . . .
Volume Layouts
Planning a First-Time VxVM Setup
to place under Volume Manager control?
Do you want to use enclosure-based naming?
Do you want to exclude any disks from Volume Manager control?
Do you want to suppress dynamic multipathing on any disks?
When you place disks under Volume Manager control, do you want to preserve or eliminate data in existing file systems and partitions?
Manager control, do you want to preserve or eliminate data in existing file systems and partitions?. ..
When you place a disk under Volume Manager control, you can either preserve the data that exists on the physical disk (encapsulation) or eliminate all of the data on the physical disk (initialization).
Encapsulation
Saving the data on a disk brought under Volume Manager control is called disk encapsulation
Encapsulation . . .
Contain the required minimum unpartitioned free space of 1024 sectors (512K) (By default, VxVM uses 2048 sectors (1024K).)
Contain an s2 slice that represents the full disk ( The s2 slice cannot contain a file system.)
Encapsulation . . .
The partitions are converted to subdisks that are used to create the volumes that replace the Solaris partitions.
Initialization
Eliminating all of the data on a physical disk brought under Volume Manager control is called disk initialization
Any disks that are encapsulated or initialized during installation are placed in the disk group rootdg. If disks are left alone during installation, they can be placed under Volume Manager control later and assigned to disk
Do you want to place the system root disk under Volume Manager control?
Existing /, /usr, and /var partitions are converted to volumes without removing the partitions. Other partitions are converted to volumes, and then partitions are removed. The existing swap area is converted to a volume. If there is insufficient space for the private region on the boot disk, Volume Manager takes sectors from the swap area of the disk, which makes the private region overlap the public region. The swap partition remains the same size, and the swap volume is resized to
Note: Volume Manager preserves a copy of the original VTOC of any disk that is encapsulated in /etc/vx/reconfig.d/disks.d/cx tydz /vtoc, where cxtydz is the SCSI address of the disk.
Typical Initial VxVM Setup
VxVM Licensing
# vxlicense –c license a key
;Adding a
# vxlicense –p ;Viewing installed license keys
VxVM Standard Packages
VRTSvxvm ;Driver & Utilities VRTSlic ;Licensing VRTSvmdoc ;Documentation VRTSvmman ;Manual Pages VRTSvmdev ;Developer kit VRTSvmsa ;Storage Admin (GUI)
Steps to Add VxVM Packages
# # # # # #
pkgadd pkgadd pkgadd pkgadd pkgadd pkgadd
-d -d -d -d -d -d
. . . . . .
VRTSlic VRTSvxvm VRTSvmsa VRTSvmdoc VRTSvmman VRTSvmdev
The vxinstall Program
# vxinstall ;to configure VxVM after package installation
The vxinstall Process
The vxinstall Process. . .
VxVM User Interfaces
Volume Manager Storage Administrator (VMSA)
Command Line Interface (CLI)
Volume Manager Support Operations (vxdiskadm)
Monitoring the Command Log
# tail -f /var/opt/vmsa/logs/command
CLI Directories
/etc/vx/bin
/usr/sbin
/usr/lib/vxvm/bin
Managing Disks
Placing a Disk Under Volume Manager Control
Placing a Disk Under Volume Manager Control. . .
Placing a Disk Under Volume Manager Control . ..
# vxdisksetup -i c1t0d0
The -i option writes a disk header to the disk, making the disk directly usable
# vxdg -g newdg adddisk newdg02=c2t0d0
Evacuating a Disk
Evacuating a disk moves the contents of the volumes on a disk to another disk. The contents of a disk can be evacuated only to disks in the same disk group that have sufficient free space. You must evacuate a disk if you plan to
# vxevac -g datadg datadg02 datadg03
Removing a disk
If a disk is no longer needed in a disk group, you can remove the disk. After you remove a disk from a disk group, the disk cannot be accessed. When removing a disk from a disk group, you have two options: Move the disk to the free disk pool. With this option, the disk remains under Volume Manager control. Send the disk back to an uninitialized state. With this option, the disk is no longer under Volume Manager control. You cannot remove the last disk in a disk group. To remove the last disk in a disk
# vxdg -g newdg rmdisk newdg02
vxdiskunsetup –C c1t0d0
Rename the disk
# vxedit -g datadg rename datadg01 datadg03
Managing Disk Group
A disk group is created when you place at least one disk in the disk group. When you add a disk to a disk group, a disk group entry is added to the private region header of that disk. Because a disk can only have one disk group entry in its private region header, one disk group does not "know about" other disk groups, and therefore disk groups cannot share resources, such as disk drives, plexes, and volumes. A volume with a plex can belong to only one disk group, and subdisks and plexes of a
When you add a disk to a disk group, VxVM assigns the disk media name to the disk and maps this name to the disk access record. In addition, the host name is also recorded in the private region. This information is written to the
Disk media name: A disk media name is the logical disk name assigned to a drive by VxVM. VxVM uses this name to identify the disk for volume operations, such as volume creation and mirroring.
Disk access record: A disk access record is a record of how a disk maps to a physical location and represents the UNIX path to the device. Disk access records are dynamic and can be recreated when vxdctl enable is run.
Once disks are placed under Volume Manager control, storage is managed in terms of the logical configuration. File systems mount to logical volumes, not to physical partitions. Logical names, such as /dev/vx/[r]dsk/diskgroup_name/volum e, replace physical locations, such as /dev/[r]dsk/c0t4d2s5
Whenever the VxVM configuration daemon is started (or vxdctl enable is run), the system reads the private region on every disk and establishes the connections between disk access records and disk media names.
The rootdg Disk Group
The rootdg disk group is a special disk group that is created when you install VxVM during the vxinstall process. VxVM requires that the rootdg disk group exist and that it contain at least one disk. It is recommended that at least two disks are in the rootdg disk group so that the VxVM configuration database can be maintained on at least two disks. If you want your
Disk Groups and High Availability
Creating Disk Group
# vxdg init newdg \ newdg01=c1t1d0s2
Designating a Disk As a Hot-Relocation Spare
vxedit -g diskgroup set spare=on|off datadg02
Making a Disk Group Unavailable
# umount /filesystem1
# umount /filesystem2
# vxdg deport newdg
# vxdg -n newerdg deport newdg
# vxdg -h server1 deport newdg
Importing a Deported Disk Group
# vxdg import newdg
# vxdg -n newerdg import newdg
# vxdg -f import newdg
Moving Disk Groups between Systems
Renaming a Disk Group
# vxdg -n mktdg deport datadg # vxdg import mktdg # vxvol -g mktdg startall (or)
# vxdg deport datadg # vxdg -n mktdg import datadg # vxvol -g mktdg startall
Destroying a Disk Group
# vxdg destroy newdg
Viewing Disk Group Information
Upgrading the Disk Group Version
#vxdg upgrade datadg
Selecting a Volume Layout
Concatenated Layout
Concatenation: Advantages
Removes size restrictions: Concatenation removes the restriction on size of storage devices imposed by physical disk size.
Better utilization of free space: Concatenation enables better utilization of free space on disks by providing for the ordering of available discrete disk space on multiple disks into a single addressable volume.
Simplified administration:
Concatenation: Disadvantages
No protection against disk failure: Concatenation does not protect against disk failure. A single disk failure may result in the failure of the entire volume.
Striped Layout
Striping: Advantages
Parallel data transfer: Striping is useful if you need large amounts of data written to or read from the physical disks quickly by using parallel data transfer to multiple disks.
Load balancing: Striping is also helpful in balancing the I/O load from multiuser applications across multiple disks.
Improved performance: Improved performance is obtained by increasing the effective bandwidth of the I/O path to the data. This may be achieved by a single volume I/O operation spanning across
Striping: Disadvantages
No redundancy: Striping alone offers no redundancy or recovery features.
Disk failure: Striping a volume increases the chance that a disk failure will result in failure of that volume. For example, if you have three volumes striped across two disks, and one of the disks is used by two of the volumes, then if that one disk goes down, both volumes will go down.
Mirrored Layout
Mirroring: Advantages
Improved reliability and availability: With concatenation or striping, failure of any one disk can make the entire plex unusable. With mirroring, data is protected against the failure of any one disk. Mirroring improves the reliability and availability of a striped or concatenated volume.
Improved read performance: Reads benefit from having multiple places from
Mirroring: Disadvantages
Requires more disk space: Mirroring requires twice as much disk space, which can be costly for large configurations. Each mirrored plex requires enough space for a complete copy of the volume's data.
Slightly slower write performance: Writing to volumes is slightly slower, because multiple copies have to be written in parallel. The overall time the write operation takes is determined by the time needed to write to the slowest disk
RAID-5
RAID-5: Advantages
Redundancy through parity: With a RAID-5 volume layout, data can be re-created from remaining data and parity in case of disk failure. Requires less space than mirroring: RAID-5 stores parity information, rather than a complete copy of the data. Improved read performance: RAID-5 provides similar improvements in read performance as in a normal striped layout. Fast recovery through logging: RAID5 logging minimizes recovery time in case
RAID-5: Disadvantages
Slow write performance: The performance overhead for writes can be substantial, because a write can involve much more than simply writing to a data block. A write can involve reading the old data and parity, computing the new
Creating a Concatenated Volume
# vxassist -g datadg make datavol 10g
# vxassist -g datadg make datavol layout=nostripe 10g
# vxassist -g datadg make datavol 10g datadg02 datadg03
Creating a Striped Volume
# vxassist -g acctdg make payvol 20m layout=stripe
# vxassist -g acctdg make expvol 20m layout=stripe ncol=3 stripeunit=64K acctdg01 acctdg02 acctdg03
Creating a RAID-5 Volume
# vxassist -g acctdg make expvol 20m layout=raid5
# vxassist -g acctdg make expvol 20m layout=raid5 stripeunit=32K acctdg01 acctdg02 acctdg03 acctdg04
Creating a Mirrored Volume
# vxassist -g datadg make datavol 5g layout=mirror
# vxassist -g datadg make datavol 5g layout=stripe,mirror
# vxassist -g datadg -b make datavol 5g layout=stripe,mirror nmirror=3
Estimating Volume Size
# vxassist -g datadg maxsize
# vxassist -g datadg maxsize layout=raid5
vxassist -g datadg maxgrow datavol
Displaying Volume information
# # # # # #
vxprint vxprint vxprint vxprint vxprint vxprint
–Aht –g rootdg –dt –st –pt -vt
Removing a Volume
Removing a Volume . . .
# vxassist -g datadg remove volume datavol
# vxedit –g datadg –rf rm datavol
Adding Mirror
Adding Mirror . . .
# vxassist -g datadg mirror datavol
# vxassist -g datadg mirror datavol datadg03
vxmirror -g datadg -a
# /etc/vx/bin/vxmirror -d yes
# vxassist make nomirrorvol 20m nmirror=1
Mirroring across Controllers and Enclosures
# vxassist -g datadg make datavol 10g layout=mirror nmirror=2 mirror=ctlr ctlr:c2 ctlr:c3
# vxassist -g datadg make datavol 10g layout=mirror nmirror=2 mirror=enclr enclr:emc1 enclr:emc2
Removing a Mirror
# vxassist -g datadg remove mirror datavol datadg02 (or)
# vxplex -g datadg dis datavol-02
# vxedit -g datadg -rf rm \ datavol-02
Excluding Storage from Volume Creation
# vxassist -g datadg make datavol 5g !datadg05
# vxassist -g datadg make datavol 5g !ctlr:c2
# vxassist -g datadg make datavol 5g ctlr:c1 !target:c1t5
# vxassist -g datadg maxsize layout=raid5 nlog=2 !datadg07 !datadg08
Adding File System to a Volume
# newfs dev/vx/rdsk/datadg/datavol
# mount /dev/vx/dsk/datadg/datavol /data
Volume Maintenance . . .
Volume Maintenance . . .
Managing Volume Tasks
Managing Volume Tasks . . .
# # # # # #
vxtask -l list ;long format vxtask -h list ; hierarchically vxtask -p list ; paused state vxtask -r list ; running state vxtask -a list ; aborted state vxtask list convertop1 ;specific task tag
Monitoring a Task
This is same as vxtask list command and addition to that we can use additional option like –w intervel
Controlling the tasks
# vxtask pause 198
# vxtask resume 198
# vxtask abort 198
# vxtask abort convertop1
Disk Encapsulation
On a Solaris system VxVM uses the volume table of contents (VTOC) to determine disk size (partition 2), then creates two partitions on the physical disk:
One partition contains the private region. The private region stores VxVM information, such as disk headers, configuration copies, and kernel logs. Tag 15 is always associated with the private region. When a disk is encapsulated, tag 15 is always associated to a slice other than slice 3. The other partition contains the public region. The public region is used for storage space
Root Disk Encapsulation
Data Disk Encapsulation Requirements
At least two partition table entries must be available on the disk.
One partition is used for the public region. One partition is used for the private region.
A minimum of 1024 sectors of unpartitioned free space, rounded up to the nearest cylinder boundary, must be available either at the beginning or at the end of the disk. VxVM uses 2048 sectors
Boot Disk Encapsulation Requirements
Boot disk encapsulation has the same requirements as data disk encapsulation, with one important distinction: when encapsulating the root disk, the private region can be created from the swap area, which reduces the swap area by the size of the private region. The private region is created at the beginning of swap, and the swap partition
Encapsulating Root
Select disk devices to encapsulate: [<pattern-list>,all,list,q,?] c0t0d0 Which disk group [
VTOC: Before Encapsulating Root Disk
VTOC: After Encapsulating Root Disk
As part of the root encapsulation process, the /etc/system file is updated to include information that tells VxVM to boot up on the encapsulated volumes. The following two lines are added to the /etc/system file: rootdev:/pseudo/vxio@0:0 set
/etc/vfstab: Before Root Encapsulation
/etc/vfstab: After Root Encapsulation
Mirroring the Root Disk
The boot disk must be encapsulated by VxVM in order to be mirrored.
To mirror the root disk, you must provide another disk with enough space to contain all of the root partitions (/, /usr, /var, /opt, and swap).
You can only use disks in the rootdg disk group for the boot disk and alternate boot
Why Create an Alternate Boot Disk
# /etc/vx/bin/vxrootmir secrootmir
# vxassist mirror homevol secrootmir
# /etc/vx/bin/vxmirror rootdisk secrootmir
# prtconf -vp | grep bootpath
bootpath: '/sbus@3,0/SUNW,socal@d,100 00/sf@0,0/ ssd@w2100002037590098,0:a'
# prtconf -vp | grep vx vx-disk01: '/sbus@3,0/SUNW,socal@d,10000/ sf@0,0/ ssd@w21000020374fe71f,0:a' vx-rootdisk: '/sbus@3,0/SUNW,socal@d,10000/ sf@0,0/ ssd@w2100002037590098,0:a'
Unencapsulating a Root Disk
# vxunroot To convert the root, swap, usr, var, opt, and home file systems back to being accessible directly through disk partitions instead of through volume devices, you use the vxunroot utility. Other changes that were made to ensure the booting of the system from the root
For vxunroot to work properly, the following conditions must be met: All but one plex of rootvol, swapvol, usr, var, opt, and home must be removed (using vxedit or vxplex). One disk in addition to the root disk must exist in rootdg.
To convert a root volume back to partitions:
Ensure that the rootvol, swapvol, usr, and var volumes have only one associated plex each. The plex must be contiguous, nonstriped, nonspanned, and nonsparse.
# vxprint -ht rootvol swapvol usr var
# vxplex -o rm dis plex_name
# /etc/vx/bin/vxunroot
This command changes the volume entries in /etc/vfstab to the underlying disk partitions for the rootvol, swapvol, usr, and var volumes. The command also modifies /etc/system and prompts for a reboot so that disk partitions are mounted instead of volumes for the root, swap, usr, and var
VxVM Architecture
VxVM is a device driver that is placed between the UNIX operating system and the SCSI device drivers. When VxVM is installed, UNIX invokes the VxVM device drivers instead of the SCSI device drivers. VxVM determines which SCSI drives are involved in the requested I/O and delivers the I/O request to those
VxVM Daemons
vxconfigd When a system is booted, the command vxdctl enable is automatically executed to start the VxVM configuration daemon, vxconfigd. VxVM reads the /etc/vx/volboot file to determine disk ownership, then automatically imports rootdg and all other disk groups owned by this host. vxconfigd reads the kernel log to determine the state of VxVM objects. vxconfigd reads the configuration database on the disks, then uses the kernel log to update the state information
VxVM Daemons . . .
vxiod vxiod—VxVM I/O kernel threads provide extended I/O operations without blocking calling processes. By default, 10 I/O threads are started at boot time, and at least one I/O thread must continue to run at all times.
VxVM Daemons . . .
vxrelocd vxrelocd is the hot relocation daemon that monitors events that affect data redundancy. If redundancy failures are detected, vxrelocd automatically relocates affected data from mirrored or RAID5 subdisks to spare disks or other free space within the disk group. vxrelocd also notifies the system
V M Disks – Private Region
The disk header contains the disk label, disk group information, host ID, and pointers to the private and public regions. You can display disk header information by using vxdisk list diskname. The configuration database contains VxVM object definitions. The size of the configuration database is approximately 70 percent of the private region. Kernel logs contain configuration changes, including information about log
Types of VxVM Disks
A simple disk is a disk that is created dynamically in the kernel and has public and private regions that are contiguous inside a single partition. A sliced disk is a disk that has separate slices for the public and private regions. A NOPRIV disk is a disk that does not contain a private region
VxVM Configuration Database
The VxVM configuration database stores all disk, volume, plex, and subdisk configuration records. The vxconfig device (/dev/vx/config) is the interface through which all changes to the volume driver state are performed. This device can only be opened by one process at a time, and the initial volume configuration is downloaded into the kernel
The configuration database is stored in the private region of a VxVM disk.
The VxVM configuration is replicated within the disk group so that sufficient copies exist to protect against loss of the configuration in case of physical disk failure. VxVM attempts to store at least four copies for each disk group.
Displaying Disk Group Configuration Data
To display the status of the configuration database for a disk group
# vxdg list acctdg
In the output, there are four disks that have configuration databases that are active (online), and there is one disk that does not have an active copy of the data (disabled).
The size of the configuration database for a disk group is the size of the smallest private region in the disk group. In the example, permlen=2630.
Log entries are on all disks that have databases. The log is used by the VxVM kernel to keep the state of the drives
Displaying Disk Configuration Data
# vxdisk –g newdg list newdg01
Term Device devicetag type hostid controlling this
Description Full UNIX device name of disk Device name used by VxVM to reference the physical disk Method of placing disk under VxVM controlDefault is sliced. Name of system that manages the disk groupIf blank, no host is currently
group disk name VM disk media name and internal ID group name Disk group name and internal ID flags Settings that describe status and options for the disk Pubpaths Paths for block and character device files of public region of disk Privpaths Paths for block and character device files of private region of disk version Version number of header format iosize The size of I/O to private region public, private slices Partition (slice) number, offset from beginning of partition, and length of partition (public offset=0, private offset=1) update time Date, time, and sequence number of last update to private region headers Offset to two copies of the private region header configs count Number of config database copies kept in the private region logs count Number of kernel logs kept in the private region Defined regions Location of configuration database(s) and kernel logs in the private regionBecause the database or logs can be split, there could be multiple pieces. Therefore, the offset is the starting location within the private region where this piece of the database begins. Copy represents which copy of the database that this piece is a part of. Note: There are "multiple pieces" of the config database because one is READ-ONLY and the other is READ-WRITE. When you perform tasks in VxVM, only the RW piece's data changes. The other piece contains the control files for the config database. Multipathing information If dynamic multipathing is enabled and there are multiple paths to the disk,
How Does vxconfigd Work?
The VxVM configuration daemon must be running in order for configuration changes to be made to the VxVM configuration database. If vxconfigd is not running, VxVM will operate properly, but configuration changes are not allowed.
vxconfigd Modes
Enabled Enabled is the normal operating mode in which most configuration operations are allowed. Disk groups are imported, and VxVM begins to manage device nodes stored in /dev/vx/dsk and /dev/vx/rdsk.
Disabled In the disabled mode, most operations are not allowed. vxconfigd does not retain configuration information for the imported disk groups and does not maintain the volume and plex device directories. Certain failures, most commonly the loss of all disks or configuration copies in the rootdg disk group, will cause vxconfigd to enter the disabled state automatically.
Booted The booted mode is part of normal system startup, prior to checking the root file system. The booted mode imports the rootdg disk group and waits for a request to enter the enabled mode. Volume device node directories are not maintained, because it may not be possible to write to the root file system.
The vxdctl Utility
vxconfigd is invoked by startup scripts during the boot procedure. To manage some aspects of vxconfigd, you can use the vxdctl utility.
# vxdctl mode mode: enabled This command displays the status of the configuration daemon. If the configuration daemon is not running, it must be started in order to make configuration changes. Disk failures are also configuration changes, but there is another way of tracking them if the daemon is down (kernel logs).
If vxconfigd is running, but not enabled, the following message is displayed: mode: disabled To enable the configuration daemon, you type: # vxdctl enable
mode: not-running
# vxconfigd
Once started, vxconfigd automatically becomes a background process.
# vxdctl stop
To send a kill -9 to vxconfigd:
# vxdctl -k stop
# vxdctl disable
To prevent configuration changes from occurring, but to allow administrative commands to be used, you can disable the daemon
# vxdctl license [init]
To display the list of VxVM features which are currently available based on known licensing information
This command checks /etc/vx/elm files in order to display currently licensed features
By adding the init argument, you can request that vxconfigd reread any persistently stored license information. If licenses have expired, some features may become unavailable. If new
The volboot File
The /etc/vx/volboot file contains a host ID that is used by VxVM to establish ownership of physical disks. This host ID is used to ensure that two or more hosts that can access disks on a shared SCSI bus will not interfere with each other in their use of those disks. This host ID is important in the generation of unique ID strings
The volboot file also contains a list of disks to scan in search of the rootdg disk group. At least one disk in this list must be both readable and a part of the rootdg disk group, or VxVM will not be able to start up correctly.
Never edit the volboot file manually. If you do so, its checksum will be invalidated.
To view the decoded contents of the volboot file
# vxdctl list
Changing the Host ID
If you change your host name in UNIX, you need to change your host ID in the volboot file. To change the host ID in the volboot file and on all disks in disk groups currently imported on the machine # vxdctl hostid hostid # vxdctl enable
If some disks are inaccessible at the time of a hostid operation, it may be necessary to use the vxdisk clearimport operation to clear out the old host ID on those disks when they become reaccessible. Otherwise, you may not be able to re-add those disks to their disk groups.
Caution: Be careful when using this command. If the system crashes before the hostid operation completes, some disk groups may not reimport automatically
Re-Creating the volboot File
To re-create the volboot file because it was removed or invalidated
# vxdctl init [hostid]
If a hostid operand is specified, then this string is used; otherwise, a default host ID is used. The default host ID is the network node name for the host
Introduction to Recovery
What Is Resynchronization ?
Minimizing the Impact of Resynchronization
DRL - Dirty region logging for Mirror Logging b – for RAID 5
DRL
How Does DRL Work? DRL logically divides a volume into a set of consecutive regions and keeps track of the regions to which writes occur. A log is maintained that contains a status bit representing each region of the volume. For any write operation to the volume, the regions being written are marked dirty in the log before the data is written. If a write causes a log region to become dirty when it was previously clean, the log is synchronously written to disk before the write operation can occur. On system restart, VxVM recovers only those regions of the volume that
RAID 5 Logging
RAID-5 volumes use RAID-5 logs to keep a copy of the data and parity currently being written.
Without logging, data not involved in any active writes can be lost or silently corrupted if both a disk in a RAID-5 volume and the system fail.
If this double-failure occurs, there is no way of knowing if the data being written to the data portions of the disks or the parity being written to the parity portions have actually been
RAID-5 logging is used to prevent corruption of data during recovery by immediately recording changes to data and parity to a log area. The new data and parity are then written to disk.
Logs are associated with a RAID-5 volume by being attached as log plexes. More than one log plex can exist for each RAID-5 volume, in which case the
Disk Failure
Permanent disk failure:
When a disk is corrupted and no longer usable, the disk must be logically and physically removed, and then replaced with a new disk. With permanent disk failure, data on the disk is lost. Example: VTOC is damaged.
Temporary disk failure:
Temporary disk failure: When communication to a disk is interrupted, but the disk is not damaged, the disk can be logically removed, then reattached as the replacement disk. With temporary (or intermittent) disk failure, data still exists on the disk. Example: Power is disrupted.
Impact of Disk Failure
VxVM is designed to protect your system from the impact of disk failure through a feature called hot relocation. The hot-relocation feature of VxVM automatically detects disk failures and restores redundancy to failed VxVM objects by moving subdisks from failed disks to other disks.
Note: Hot relocation is only performed for redundant (mirrored or RAID-5) subdisks on a failed disk. Nonredundant subdisks on a failed disk are not relocated, but the system administrator is notified of the failure.
Partial Disk Failure
When a partial disk failure occurs (that is, a failure affecting only some subdisks on a disk), redundant data on the failed portion of the disk is relocated. Existing volumes on the unaffected portions of the disk remain accessible. With partial disk failure, the disk is not removed from VxVM control and is labeled as FAILING, rather than as FAILED. Before removing a FAILING disk for replacement, you must evacuate any remaining volumes on the disk.
How Does Hot Relocation Work?
The vxrelocd daemon starts during system startup and monitors VxVM for failures involving disks, plexes, or RAID-5 subdisks. When a failure occurs, vxrelocd triggers a hotrelocation attempt and notifies the system administrator, through email, of failures and any relocation and recovery actions.
The vxrelocd daemon is started from the S95vxvm-recover file. The argument to vxrelocd is the list of people to e-mail notice of a relocation (default is root). To disable vxrelocd, you can place a "#" in front of the line in the S95vxvm-recover file.
The hot-relocation feature is enabled by default. No system administrator action is needed to start hot relocation when a failure occurs.
A successful hot-relocation process involves
Failure detection: Detecting the failure of a disk, plex, or RAID-5 subdisk Notification: Notifying the system administrator and other designated users and identifying the affected Volume Manager objects Relocation: Determining which subdisks can be relocated, finding space for those subdisks in the disk group, and relocating the subdisks (The system administrator is notified of the success or failure of these actions. Hot relocation does not guarantee the same layout of data or the same performance after relocation.) Recovery: Initiating recovery procedures, if necessary, to restore the volumes and data (Again, the system administrator is notified of
Setting Up Spare Disks
vxedit -g datadg set spare=on datadg03
vxedit -g datadg set spare=off datadg03
vxedit -g datadg set nohotuse=on datadg04
vxedit -g datadg set nohotuse=off datadg04
Including Spare Disks in Space Availability
# vxassist -g mydg -r maxsize layout=stripe ncolumns=3
To include spare disks when determining how much space is available using the maxsize or maxgrow options, you add the -r flag in the vxassist command
Reserving Disks
A spare disk is not the same as a reserved disk. You can reserve a set of disks for special purposes, such as to avoid general use of a particularly slow or a particularly fast disk
# vxedit set reserve=on datadg05
# vxedit set reserve=off datadg05
After you type this command, vxassist does not allocate space from the selected disk unless that disk is specifically mentioned on the vxassist command line.
# vxassist make vol03 20g datadg05
overrides the reservation and creates a 20-MB volume on datadg05. However, the command
# vxassist make vol04 20g
does not use datadg05, even if there is no free space on any other disk.
Disk Replacement Tasks
Replacing a failed or corrupted disk involves two main operations :
1. Disk replacement When a disk fails, you replace the corrupt disk with a new disk. The disk used to replace the failed disk must be either an uninitialized disk or a disk in the free disk pool. The replacement disk cannot already be in a disk group. If you want to use a disk that exists in another disk group, then you must remove the disk from the disk group and place it back into the free disk pool
2. Volume recovery When a disk fails and is removed for replacement, the plex on the failed disk is disabled, until the disk is replaced. Volume recovery involves:
Starting disabled volumes
Note: A volume remains started, and does not need to be restarted, if it has a RAID-5 or mirrored layout, that is, if the volume has one remaining active plex.
Resynchronizing mirrors Resynchronizing RAID-5 parity
After successful recovery, the volume is available for use again. Redundant (mirrored or RAID-5) volumes can be recovered by VxVM. Nonredundant (unmirrored) volumes must be restored from backup.
Adding a New Disk
Before VxVM can use a new disk, you must ensure that Solaris recognizes the disk. When adding a new disk, follow these steps to ensure that the new disk is recognized:
1.Connect the new disk. 2. Get Solaris to recognize the disk: # drvconfig # disks Note: In Solaris 7 and later, use devfsadm, a onecommand replacement for drvconfig and disks. 3. Verify that Solaris recognizes the disk: # prtvtoc /dev/dsk/device_name 4. Get VxVM to recognize that a failed disk is now working again: # vxdctl enable 5. Verify that VxVM recognizes the disk: # vxdisk list After Solaris and VxVM recognize the new disk, you can then use the disk as a replacement disk.
Replace a failed disk
Select a removed or failed disk [
The following devices are available as replacements: c1t0d0s2 c1t1d0s2 Choose a device, or select "none" [<device>,none,q,?] (default: c1t0d0s2)
After you confirm the operation, the following status message is displayed:
Replacement of disk datadg02 in group datadg with disk device c1t0d0s2 completed successfully.
Replacing a Disk: CLI
# vxdg -k -g datadg adddisk datadg01=c1t1d0s2
The -k switch forces VxVM to take the disk media name of the failed disk and assign it to the new disk
For example, if the failed disk datadg01 in the datadg disk group was removed, and you want to add the new device
Note: Exercise caution when using the -k option to vxdg. Attaching the wrong disk with the -k option can cause unpredictable results in VxVM
The vxunreloc Utility
The hot-relocation feature detects I/O failures in a subdisk, relocates the subdisk, and recovers the plex associated with the subdisk.
VxVM also provides a utility that unrelocates a disk, that is, moves relocated subdisks back to their original disk. After hot relocation moves subdisks from a failed disk to other disks, you can
Unrelocation is performed using the vxunreloc utility, which restores the system to the same configuration that existed before a disk failure caused subdisks to be relocated.
Viewing Relocated Subdisks
When a subdisk is hotrelocated, its original disk media name is stored in the orig_dm_name field of the subdisk record files. You can search this field to find all the subdisks that originated from a failed disk using the vxprint command
For example, to display all the subdisks that were hot-relocated from datadg01 within the datadg disk group:
# vxprint -g datadg -se `sd_orig_dmname="datadg01"'
Unrelocating a Disk: vxdiskadm
Recovering a Volume
The vxreattach utility reattaches disks to a disk group and retains the same media name. This command attempts to find the name of the drive in the private region and to match it to a disk media record that is missing a disk access record. This operation may be necessary if a disk has a transient failure, for example, if a drive is turned off and then back on, or if the Volume
vxreattach tries to find a disk in the same disk group with the same disk ID for the disks to be reattached. The reattach operation may fail even after finding the disk with the matching disk ID if the original cause (or some other cause) for the disk failure still exists.
/etc/vx/bin/vxreattach [-bcr] [dm_name]
-b Performs the reattach operation in the background -c Checks to determine if a reattach is possibleNo operation is performed, but the disk group name and the disk media name at which the disk can be reattached are displayed. -r Attempts to recover stale plexes of any
The vxrecover Command
To perform volume recovery operations from the command line, you use the vxrecover command. The vxrecover program performs plex attach, RAID-5 subdisk recovery, and resynchronize operations for specified volumes (volume_name), or for volumes residing on specified disks (dm_name). You can run vxrecover
Note: The vxrecover command will only work on a started volume. A started volume displays an ENABLED state in vxprint -ht.
Recovery operations are started in an order that prevents two concurrent operations from involving the same disk. Operations that involve unrelated disks will run in parallel.
# vxrecover -bs -g datadg datadg01
After replacing the failed disk datadg01 in the datadg disk group, and adding the new disk c1t1d0s2 in its place, you can attempt to recover the volumes.
# vxrecover -b -g datadg datadg01
To recover, in the background, any detached subdisks or plexes that resulted from replacement of the disk datadg01 in the datadg disk group
# vxrecover -v -g datadg datadg01
To monitor the operations during the recovery
Protecting VxVM Configuration
Precautionary Tasks
To protect the VxVM configuration, you can perform two precautionary tasks: 1. Save a copy of the VxVM configuration using the vxprint command.
You can also use the vxprint command to save the VxVM configuration to a file and later use that file to recover removed volumes. By saving the output of the vxprint command to a file, you can then use the vxmake command with the saved file to restore the configuration if needed.
When saving the VxVM database configuration, you use the -m option. This option displays all information about each record in a format that is useful as input to the vxmake utility. To view the saved records, you can use the -D - option. This option reads a configuration from the standard
Saving the Configuration Database
# vxprint -g diskgroup -hmQqr > backup.DGname (or)
# vxprint -g diskgroup -hmvpsQqr > backup.DGname
Displaying a Saved Configuration
# vxprint -D - -rht < backup.DGname
Recovering a Lost Volume
# vxprint -D - -rhtmqQ lostvolume < backup.DGname > restoredvolume
This command creates object definitions for a restored volume out of the object definitions in the lost volume
To implement the object definitions of restoredvolume into a real volume
# vxmake -g diskgroup -d restoredvolume
To start the restored volume, and recover its plexes.
# vxrecover -Es restoredvolume
Related Documents
Vxvm
May 2020 4
Vxvm
November 2019 11
Vxvm Toc
May 2020 1
Vxvm Commands
May 2020 1
Vmmaintenenace(vxvm)
May 2020 2