Logical Volume Manager (LVM) using HP-UX

Glossary

Term
 Definition
Journaled File System (JFS)
 Unix filesystem that uses a journaled log to record changes to the filesystem.  In the event of a failure the log can be used to recover.
Logical Extents
 Logical extents are used to create the logical volume - they are the same size as physical extents
Logical Interchange Format (LIF)
 HP Storage format used for interchange of files between HP systems.
Logical Volumes
 Allocates the space inside a volume group, the logical volume gets the file system
Mirroring
 Mapping a logical extent to more than one physical extent.  Logical volumes are mirrored, not disks.
Physical Extent (PE)
 Unit used for allocating disk space to logical volumes, default size is 4MB.  It is a set of contiguous disk blocks.
Physical Partitions
 The physical volume is organized into blocks called physical partitions (same as physical extent??)
Physical Volume
 Physical disk that is being managed by LVM
Striping
 A logical volume that is configured to write across more than one disk in a stripe.
Volume
 A device used for storage - either an entire disk or a partition.
Volume Group
 Contain one or more physical volumes, a physical volume can only be in one volume group (pool of partitions)

Commands

Command Function
lvcreate
 Command used to create logical volumes
lvdisplay -v /dev/<VG>/<LV> Command used to display information about logical volumes
pvchange
 Changes the attributes of existing physical volumes.
pvcreate /dev/rdsk/cxtxdx
pvcreate -f /dev/rdsk/cxtxdx
 Creates a physical volume by creating the LVM data structures on the physical disk called physical extents.  The -f option will force the creation of the PV - potentially destroying old PV information.
pvdisplay /dev/dsk/cxtxdx
pvdisplay -v /dev/dsk/cxtxdx
 Command used to display information about physical volumes
vgchange
 Command used to change information about the volume group, such as to activate an inactive vg
vgcreate
 Command used to create the volume groups
vgdisplay [-v] [VG_NAME}
 Command used to display information about volume groups, -v is the verbose option which will list lvs and pvs.
vgexport
vgimport


Files

File Description
/dev/dsk/cxtxdx Block device for disk
/dev/rdsk/cxtxdx Character device for disk
/dev/<volumegroup>/group
 Character device file (created with the mknod command) that allows the lvm kernel and the lvm commands to communicate
/etc/lvmrc
 Script that starts each volume group based upon the contents of /etc/lvmtab
/etc/lvmtab This file has the device file associated with each disk in a volume group.  Access it with the strings command.
/etc/vpath.cfg If IBM SDD is being used this file contains the mapping of vpath to the corresponding disk-target-lun.  Access it with the strings command.

Procedures

Filesystem Procedures:

Create a new filesystem:
(A new filesystem will require a new logical volume)
  1. Create the Logical Volume (Procedure Below)
  2. Create the File System
      1. newfs -F <filesystem_type> <path_to_lvol_devicefile>
      2. newfs -F vxfs /dev/vg02/lvol1
  3. Create the Mount Point
    1. mkdir -p <full path to mountpoint>
    2. -p will make any subdirectories in the path that do not already exist
  4. Set the proper ownership and permissions on the mount point
    1. chown owner:group /<mount point>
    2. chmod XXX /<mount point>
  5. Mount the File System
    1. Edit /etc/fstab using vi and add the proper entry
      1. <LV PATH> <MOUNTPOINT PATH> <FILESYSTEM> <OPTIONS> <BACKUP FREQUENCY> <PASS NUMBER>
      2. For example:  "/dev/vgbill/lvbill  /maindir/subdir/share vxfs rw,suid,largefiles,delaylog,datainlog 0 2"
    2. "mount /<mountpoint>" to mount the file system
  6. Test
    1. Do a "bdf <mount point>".    This command will show if the file system is mounted, and it's size.
    2. Do an "ls -ld <mount point>" to verify that the proper permissions are there.
    3. Have the user test the filesystem, and report back with any problems
Extending a VxFS Filesystem (Online JFS)
  1. Verify that there is enough space in the volume group
  2. First lvextend the logical volume - 2 Options for lvextend:
    1. lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME
    2. lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME
  3. fsadm -F vxfs  -b <new size of filesystem in K> <MOUNT POINT>
Extending a VxFS Filesystem (No Online JFS)
  1. Verify that there is enough space in the volume group
  2. First lvextend the logical volume - 2 Options for lvextend:
    1. lvextend -L <New Size in MB> /dev/VG_NAME/LV_NAME
    2. lvextend -l <NEW Size in # of LEs> /dev/VG_NAME/LV_NAME
  3. Unmount the filesystem
  4. extendfs -F vxfs /dev/<VG_Name>/r<LV_Name>


How to determine if you have Online JFS
How to determine the filesystem type

Logical Volume Procedures:

Creating a Logical Volume:
  1. Determine if an existing volume group has enough spare capacity for a new logical volume.
    1. Is mirroring or striping required (if not known, then see if other logical volumes in that volume group use it)?
    1. vgdisplay   <This will list existing volume groups
      1. Spare capacity is indicated by the number of free PEs
      2. Multiply the "Free PE" by "PE Size (MBytes)" to get spare capacity in MB of the volume group
        • Note:  If mirroring or striping is required then there must be enough space available on multiple disks
      3. Is there a volume group with enough spare capacity?
        1. If yes, the continue to "Create the Logical Volume"
        2. If no, then the Volume Group must be extended or a new Volume Group must be created - see procedures below.
  2. Create the logical volume:
      1. -l is size in Logical Extents (same as PE size)
      2. -n is the name of the Logical Volume
      3. -r N disable bad block allocation
      1. lvcreate -L 100 /dev/vg01
        • Logical volume will be 100MB in size
        • Name of the logical volume default name will be lvolx
      2. lvcreate -L 100 -n oracle /dev/vg01
        • This creates a logical volume called oracle of 100 MB
  3. Verify the lv was created properly "vgdisplay -v <volume group>"
Removing a Logical Volume

Troubleshooting (just an idea, not sure if this really works??)
ONCTRA01# lvchange -a n /dev/vg_wte_test/lv_wte_test
Logical volume "/dev/vg_wte_test/lv_wte_test" has been successfully changed.
Volume Group configuration for /dev/vg_wte_test has been saved in /etc/lvmconf/vg_wte_test.conf

Unmirroring a Logical Volume

Volume Group Procedures:

Create a Volume Group
  1. Determine the list of disk devices that will make up the volume group (procedure below)
  2. Create PV disks for each of the disks (procedure below)
  3. Create the volume group directory
    1. mkdir /dev/vg01
  4. Create the group file
    1. Determine the minor number to use
      1. ls -l /dev/*/group | sort +5
      2. Choose the next unused number
    1. mknod /dev/vol_group_directory/group c Major_number Minor_number
      1. mknod /dev/vg01/group c 64 0x010000
      2. The group file is used to allow communications between the lvm commands and the lvm kernel
  5. Create the volume group
    1. vgcreate -l 255 -p 64 -s 16 /dev/<vg id>/dev/dsk/<disk id>  <-- create the volume group with initial disk devices
      1. -l sets the maximum number of logical volumes for the volume group
      2. -p sets the maximum number of physical volumes for the volume group
      3. -s sets the physical extent size
      1. /dev/<vg id> is the volume group directory
      2. /dev/dsk/diskid are the physical volumes to be added to the volume group
      3. vgcreate /dev/vg01 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0
    2. vgextend /dev/vgxx /dev/dsk/diskid  <-- adds additional disk devices to the volume group
Extending a Volume Group
Activating a Volume Group
  1. vgchange -a y /dev/<vg name>
  2. mount -a  ==>  This may indicate that a filesystem check needs to be run (fsck)
  3. fsck -m /dev/<vg name>/<lv name>  ==> this is a sanity check, it is very quick, but may indicate that a complete fsck needs to be run
  4. fsck /dev/<vg name>/<lv name>  ==> this is the full file system check
Deactivating a Volume Group
  1. umount all of the logical volumes in the volume group
  2. vgchange -a n /dev/<vg name>

Adding mirroring to the root vg - vg00
  1. pvcreate -B /dev/rdisk/c2t2d0
  2. mkboot /dev/rdsk/c2t2d0
  3. mkboot -a "hpux-lq (;0)/stand/vmunix" /dev/rdsk/c2t2d0
  4. vgextend /dev/vg00 /dev/dsk/c2t2d0
  5. vgdisplay -v vg00
  6. lvlnboot -v
  7. lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c2t2d0

Removing a Volume Group

To remove a volume group using vgreduce and vgremove:
To remove a volume group using vgexport:

PV Procedures:

How to recognize a newly added LUN:
  1. Check for the new hardware
  2. Create the device file for the hardware path
  3. If using vpaths then create the vpath association
  4. Verify the new devices/vpaths are there:
  5. Create the PV
How to create Physical Volumes:
How to determine available disks to be used in a Volume Group:
  1. "ioscan -funC disk"  will list all of the disk devices
    1. Some of these devices will be allocated, some will not
  2. "vgdisplay -v" will list all of the PVs and their devices for all of the existing volume groups
    1. This is a list of the devices that are in use
  3. Any devices that are on the ioscan, but are NOT on the vgdisplay are available for use
    1. Possible strategy to automate this process using sed, awk and grep
      1. create a file that has all of the disks that can be used (in this example HITACHI)
        1. cat wte_disk_ioscan | sed 1,2d | grep -v -e TOSHIBA -e c2t0d0 | xargs -n 10 | grep HITACHI | grep -vi subsystem > wte_hitachi_disk
          1. cat will output the file that contains the disk ioscan (ioscan -fnC disk) 
          2. sed is used to delete the first 2 header lines of the file.
          3. Next grep is used to print any lines that DO NOT include TOSHIBA or c2t0d0
          4. xargs is used to group the output into groups of 10
          5. Next grep finds all of the lines with HITACHI in them
          6. All of the lines that have HITACHI in them are saved to a file
      1. Refine the ioscan of HITACHI disks to include just the disk devices, sorted - this is a list of all HITACHI disks on the system
        1. awk '{ print $9 }' wte_hitachi_disk | sort -u > wte_hitachi_sorted_u
          1. awk prints just the 9th field of the file
          2. sort - u sorts the file and surpresses any duplicates
          3. This is saved to a sorted file
      2. Print a list of all the disks that are currently being used (a list of PVs)
        1. vgdisplay -v | grep  "PV Name" > wte_pvdisk_used
          1. vgdisplay -v prints a verbose listing of all volume groups
          2. grep only prints lines that contain PV Name
          3. The list of PVs is saved to a file
      3. Refine the list of disks that are being used
        1. awk '{ print $3 }' wte_pvdisk_used | sort -u > wte_pvdisk_sorted_u
          1. awk prints on the 3rd field (the disk device)
          2. sort will sort the list, surpressing any duplicate entries
          3. the results are saved to a file
      4. Compare the 2 files - the list of all Hitachi disks on the system with the list of all disks being used
        1. diff wte_hitachi_sorted_u wte_pvdisk_sorted_u
          1. diff compares the 2 files and prints out any differences.  The difference will be a disk that the system sees, but that is not being used by LVM
How to remove PVs

How to perform an non-destructive test of a disk:
dd if=/dev/rdsk/cxxxxxx of=/dev/null bs=1024k