Using Logical Volume Management (LVM) to Organize Your Disks on SLES 10
Gone are the days when we build a SUSE Enterprise Linux Server (SLES) and simply create one data partition for root (/) and one swap partition. Today’s Linux servers have reached a level of maturity in the enterprise that calls for more file systems that are more versatile, configurable and manageable. Linux Logical Volume Management (LVM), using Volume Groups and individual file systems allows for this flexibility.
In this article, we will walk through configuring your disk devices to use the various utilities included with LVM. We will also show how to control, as administrators, how much disk allocation the various components or applications are allowed to function properly.
- General
- Layout and Design
- Setting up Logical Volume Management
- Creating Disk Partitions Suitable for LVM
- Creating Physical Volumes
- Creating Volume Groups
- Creating Logical Volumes
- Expanding a Volume
- Conclusion
General
I know you’re wondering why you should do something like this, when you’ve done just fine with a root (/) partition. Ever see what happens to a NetWare server when SYS: volume fills up? The same thing happens to a Linux server when root (/) maxes out. Did you store your NetWare Server data on SYS:? Or did you create a VOL1 or similar? LVM gives you a whole lot more flexibility to managing file systems than non-LVM configurations ever did. Some of the pros to LVM over traditional (physical) partitioning are:
- Add physical disks to an Logical Volume without shutting the system down, assuming you use hot-swappable disk systems.
- Take a snapshot of the Logical Volume during normal operation.
- Expand a Logical Volume on the fly.
- Multiple physical volumes can be combined into one or more Logical Volume.
- Up to 256 Logical Volumes can be added, Traditional (physical) partitioning only allows for 4 Primary and 15 Logical partitions.
- There’s one other feature, although not recommended unless you’re deploying RAID 5 or better, Disk Striping.
Layout and Design
Deciding how granular your Logical Volumes and Volume Groups is really up to you and your needs in your organization. I will recommend that you create standards for a base server configuration and enforce it. Leaving enough disk space unused for various applications. If you are using a virtual environment, create a template and use that for your standard.
I have found that for most of what I need generally, including monitoring, server management, OS and two swap files is a 50GB disk. Make sure you adjust your sizes to accommodate your organizational needs. Most of the servers I use are virtual, which makes it easier to reduce the waste of today’s larger disks that come with most physical servers.
For ancillary applications, I add an additional 20GB disk device or larger, depending on the needs of the project. Web servers, Application Servers, etc, will be loaded there. Note that these are for SLES only. No Open Enterprise Server components are put into consideration.
The SLES installation utility will suggest a very basic partitioning scheme. It uses traditional (physical) partitioning by default. LVM is under Expert Configuration.
Although possible, it is not recommended to use LVM for your /boot and SWAP partitions. I wouldn’t recommend it either.
Setting up Logical Volume Management
On our system installation example, we’ll setup LVM as part of the initial SLES installation. As far as naming and sizes, refer to your own organizational needs and/or specifications.
There are basically, four steps to setting up LVM.
Create physical disk partition with type 8e (LVM)
Create physical volumes
Create volume groups
Create logical volumes
Pre-plan and write down what names you want for Physical Volumes, Volume Groups and Logical Volumes. Doing this prior to installation will make your partitioning configuration flow more smoothly without you wondering what labels to place on the various components.
Here is an example of what could be configured using LVM
I will use the two disk devices mentioned earlier:
50GB System and 20GB Application
/dev/sda1 500M /boot
/dev/sda2 2.0G SWAP
/dev/mapper/vgroot-root 3.0G /
/dev/mapper/vgroot-home 9.0G /home
/dev/mapper/vgroot-opt 5.0G /opt
/dev/mapper/vgroot-tmp 1.0G /tmp
/dev/mapper/vgroot-usr 5.0G /usr
/dev/mapper/vgroot-var 5.0G /var
Total = 30.5G
/dev/sdb1 2.0G SWAP
/dev/mapper/vgapp-apache 10G /opt/apache
Total = 12.0G
You can see we still have some room to play, should we need slight adjustments. With LVM, we can add more space on the fly, if needs arise.
In our example, you’ll notice that the /boot and both SWAP partitions are physical disk types. Because LVM uses Device Mappings for Volume Groups, the path of /dev/mapper/ is default. I created two different Volume Groups, “vgroot” and “vgapp.” I used the mount point names as Logical Volumes for each. The system inserts a “-” in the path for each Logical Volume. This makes the logical volumes more manageable when it comes time for backup or maintenance. You may also want to use individual application names that are targeted for that logical volume, such as “vgapp-apache.” If you do not use a centralized log repository, then you might want to consider creating an additional logical volume for /var/log.
In the example above, you may question why I only gave 3.0GB to root (/). I’ve partitioned off /var, /opt, /usr, /tmp and /home onto their own partitions. This eliminates their need for space from the root (/) partition.
Creating Disk Partitions Suitable for LVM
All operations are performed as root.
We have our /boot partition already created. Now we can create our additional partitions for LVM. I prefer the command line, you can use the YaST utility, the end results are the same.
In our example, we have /dev/sda and /dev/sdb disk devices.
Run fdisk to partition the remainder of /dev/sda for the OS and our root partitions.
[ /root ] root@myserver1 # fdisk /dev/sda Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 2610. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help):
In fdisk, there are several options once you load it. If you forget your commands, type ‘m’ and it will display the menu.
Command (m for help): m Command action a toggle a bootable flag b edit bsd disklabel c toggle the dos compatibility flag d delete a partition l list known partition types m print this menu n add a new partition o create a new empty DOS partition table p print the partition table q quit without saving changes s create a new empty Sun disklabel t change a partition's system id u change display/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only) Command (m for help):
Let’s print (list) the existing partition table for /dev/sda
Command (m for help): p Disk /dev/sda: 51.4 GB, 52474836480 bytes 255 heads, 63 sectors/track, 6379 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 1 62 59898+ 83 Linux Command (m for help):
We can see /boot and we’ve set it to 500MB as defined earlier. Now we’re going to add a 2GB SWAP.
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (63-6379, default 63): Using default value 63 Last cylinder or +size or +sizeM or +sizeK (63-6379, default 6379): +2G Command (m for help): t Selected partition 2 Hex code (type L to list codes): L 0 Empty 1e Hidden W95 FAT1 80 Old Minix be Solaris boot 1 FAT12 24 NEC DOS 81 Minix / old Lin bf Solaris 2 XENIX root 39 Plan 9 82 Linux swap / So c1 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 83 Linux c4 DRDOS/sec (FAT- 4 FAT16 <32M 40 Venix 80286 84 OS/2 hidden C: c6 DRDOS/sec (FAT- 5 Extended 41 PPC PReP Boot 85 Linux extended c7 Syrinx 6 FAT16 42 SFS 86 NTFS volume set da Non-FS data 7 HPFS/NTFS 4d QNX4.x 87 NTFS volume set db CP/M / CTOS / . 8 AIX 4e QNX4.x 2nd part 88 Linux plaintext de Dell Utility 9 AIX bootable 4f QNX4.x 3rd part 8e Linux LVM df BootIt a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e1 DOS access b W95 FAT32 51 OnTrack DM6 Aux 94 Amoeba BBT e3 DOS R/O c W95 FAT32 (LBA) 52 CP/M 9f BSD/OS e4 SpeedStor e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi eb BeOS fs f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ee EFI GPT 10 OPUS 55 EZ-Drive a6 OpenBSD ef EFI (FAT-12/16/ 11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f0 Linux/PA-RISC b 12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f1 SpeedStor 14 Hidden FAT16 <3 61 SpeedStor a9 NetBSD f4 SpeedStor 16 Hidden FAT16 63 GNU HURD or Sys ab Darwin boot f2 DOS secondary 17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fd Linux raid auto 18 AST SmartSleep 65 Novell Netware b8 BSDI swap fe LANstep 1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid ff BBT 1c Hidden W95 FAT3 75 PC/IX Hex code (type L to list codes): 82 Changed system type of partition 2 to 82 (Linux swap / Solaris) Command (m for help):
We’ll use the remaining space on this disk for LVM.
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 3 First cylinder (307-6379, default 307): Using default value 307 Last cylinder or +size or +sizeM or +sizeK (307-6379, default 6379): Using default value 6379 Command (m for help): t Partition number (1-4): 3 Hex code (type L to list codes): 8e Changed system type of partition 3 to 8e (Linux LVM)
Now let’s write it to the partition table.
Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [ /root ] root@myserver1 #
Let’s list the partitions to see the table.
[ /root ] root@myserver1 # fdisk -l /dev/sda Disk /dev/sda: 51.4 GB, 52474836480 bytes 255 heads, 63 sectors/track, 6379 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sda1 1 62 59898 83 Linux /dev/sda2 63 306 1958325 82 Linux swap / Solaris /dev/sdb2 307 6379 19004895+ 8e Linux LVM [ /root ] root@myserver1 #
Now let’s create the second disk’s partitions
[ /root ] root@myserver1 # fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 2610. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help):
Print the part table just to be sure that there’s nothing on it. Can never be too sure.
Command (m for help): p Disk /dev/sdb: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System
First, create another 2GB SWAP
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-2610, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-2610, default 2610): +2G Command (m for help): t Selected partition 1 Hex code (type L to list codes): 82 Changed system type of partition 1 to 82 (Linux swap / Solaris)
Add the remaining for LVM
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (245-2610, default 245): Using default value 245 Last cylinder or +size or +sizeM or +sizeK (245-2610, default 2610): Using default value 2610 Command (m for help): t Partition number (1-4): 2 Hex code (type L to list codes): 8e Changed system type of partition 2 to 8e (Linux LVM)
Write it to the partition table
Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [ /root ] root@myserver1 #
Let’s verify that /dev/sdb was written correctly.
[ /root ] root@myserver1 # fdisk -l /dev/sdb Disk /dev/sdb: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/sdb1 1 244 1959898+ 82 Linux swap / Solaris /dev/sdb2 245 2610 19004895 8e Linux LVM [ /root ] root@myserver1 #
Create our swap files on each disk
[ /root ] root@myserver1 # mkswap /dev/sda2 Setting up swapspace version 1, size = 2006929 kB [ /root ] root@myserver1 # mkswap /dev/sdb1 Setting up swapspace version 1, size = 2006929 kB [ /root ] root@myserver1 #
And use swapon to tell the system to use the new swap files.
[ /root ] root@myserver1 # swapon /dev/sda2 [ /root ] root@myserver1 # swapon /dev/sdb1
Add your swap files to /etc/fstab
file: /etc/fstab
... /dev/sda2 swap swap defaults 0 0 /dev/sdb1 swap swap defaults 0 0 ...
After you add the swap partitions to /etc/fstab, we can activate them using the “-a” option.
[ /root ] root@myserver1 # swapon -a
Use the “-s” option to see the swap files and usage.
[ /root ] root@myserver1 # swapon -s Filename Type Size Used Priority /dev/sda2 partition 1959888 0 -1 /dev/sdb1 partition 1959888 0 -2 [ /root ] root@myserver1 #
Now that we have our swap partitions up and swapping, let’s set up LVM.
Physical Volumes
When we create our physical volume (PV), we use the pvcreate utility. pvcreate initializes the partition(s) for later use by the LVM. Each PV can be a disk partition, whole disk, meta device, or loopback file. You can specify multiple partitions using the pvcreate command.
Since we want to keep the OS from other disk systems, we’ll make a separate PV.
[ /root ] root@myserver1 # pvcreate /dev/sda3 Physical volume "/dev/sda3" successfully created [ /root ] root@myserver1 # pvcreate /dev/sdb2 Physical volume "/dev/sdb2" successfully created
Volume Groups
We use vgcreate to create the volume groups (VG). Vgcreate creates a new volume group called VolumeGroupName using the block special device PhysicalVolumePath previously configured for LVM with pvcreate.
You can add multiple Physical Volumes to your Volume Group.
[ /root ] root@myserver1 # vgcreate /dev/vgroot /dev/sda3 Volume group "/dev/vgroot" successfully created [ /root ] root@myserver1 # vgcreate /dev/vgapp /dev/sdb2 Volume group "/dev/vgapp" successfully created
Logical Volumes
lvcreate creates a new logical volume (LV) in a volume group. I separate logical volumes and name them according to their mount point I want. Since our example has multiple mount points for the OS, we’ll a few to make. Notice that the Volume Group remains the same for the different logical volumes.
[ /root ] root@myserver1 # lvcreate --name root --size 3G /dev/vgroot Logical volume "root" created [ /root ] root@myserver1 # lvcreate --name home --size 9G /dev/vgroot Logical volume "home" created [ /root ] root@myserver1 # lvcreate --name opt --size 5G /dev/vgroot Logical volume "opt" created [ /root ] root@myserver1 # lvcreate --name tmp --size 1G /dev/vgroot Logical volume "tmp" created [ /root ] root@myserver1 # lvcreate --name usr --size 5G /dev/vgroot Logical volume "usr" created [ /root ] root@myserver1 # lvcreate --name var --size 5G /dev/vgroot Logical volume "var" created [ /root ] root@myserver1 #
This last one is for our application Apache.
[ /root ] root@myserver1 # lvcreate --name apache --size 10G /dev/vgapp Logical volume "apache" created [ /root ] root@myserver1 #
Now all of the newly created logical volumes need to be formatted with the file system you choose. For this example, we’re using ReiserFS for the format.
[ /root ]
root@myserver1 # echo y | mkreiserfs /dev/vgapp/apache
...
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..ok
ReiserFS is successfully created on /dev/vgapp/apache.
[ /root ]
root@myserver1 #
Format the remaining logical volumes.
Edit /etc/fstab and put your mount points for each newly created file system.
/dev/mapper/vgapp-apache /opt/apache reiserfs acl,user_xattr 1 2
Use the Dump and fsck options (the last two numbers) at your discretion. Use the man pages for fstab to view the options.
Once you’ve added your /etc/fstab entries, mount everything with mount’s -a option.
[ /root ] root@myserver1 # mount -a [ /root ] root@myserver1 #
Use df to view your mounted file systems. We’ll just look at one of the mount points to save time.
[ /root ]
root@myserver1 # df -k /opt/apache
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vgapp-apache
5242716 32840 5208976 1% /opt/apache
[ /root ]
root@myserver1 #
Extending a Volume
We’ll use two examples of extending a volume (adding more space) to show a simple case and a more involved example of adding an additional disk.
Example 1: Adding 5GB to a Logical Volume.
Using the setup for our /opt/apache application mount point, we will add 5GB to it, as there is 12GB available space on the /dev/sdb2 PV.
[ /root ] root@myserver1 # lvextend -L +5G /dev/vgapp/apache Extending logical volume apache to 15.1 GB Logical volume apache successfully resized [ /root ] root@myserver1 #
Now the file system needs to be extended also. We can use resize-reiserfs command for this, since our examples are of type reiserfs. For ext3, use resizefs for online file system expansion.
[ /root ] root@myserver1 # resize_reiserfs /dev/vgapp/apache resize_reiserfs: On-line resizing /dev/vgapp/apache. ... resize_reiserfs: Resizing finished successfully. [ /root ] root@myserver1 #
If you are not comfortable with online expansion, both commands work with unmounted file systems also.
Example 2: Adding an additional disk to /opt/apache
We’ll assume you’ve already added the disk or virtual disk to your server and booted. In this example, we’ll use /dev/sdc for our disk device name.
You can use the fdisk example above to fdisk the new device. The entire disk will be used for LVM. We’ll start at that point.
Start by creating a new Physical Volume for the new disk.
[ /root ] root@myserver1 # pvcreate /dev/sdc1 Physical volume "/dev/sdc1" successfully created
Extend the Volume Group vgapp to include the new physical volume.
[ /root ] root@myserver1 # vgextend vgapp /dev/sdc1 Volume group "vgapp" successfully extended [ /root ] root@myserver1 #
Use vgdisplay the volume group information.
[ /root ] root@myserver1 # vgdisplay vgapp --- Volume group --- VG Name vgapp System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 6 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 2 Act PV 2 VG Size 121.74 GB PE Size 4.00 MB Total PE 53438 Alloc PE / Size 35840 / 120.00 GB Free PE / Size 17598 / 100.74 GB VG UUID GNrQo7-FFmg-ohzP-wMLY-wGps-I8IK-6ZEeD5 [ /root ] root@myserver1 #
We can see that there are 2 Physical Volumes (Cur PV) in our Volume Group now.
Now we want to add half of the 100GB of free space to our /dev/vgapp/apache logical volume.
[ /root ] root@myserver1 # lvextend --extents +50%FREE /dev/vgapp/apache Extending logical volume apache to 61.1 GB Logical volume apache successfully resized [ /root ] root@myserver1 #
Resize the file system.
[ /root ] root@myserver1 # resize_reiserfs /dev/vgapp/apache resize_reiserfs: On-line resizing /dev/vgapp/apache. ... resize_reiserfs: Resizing finished successfully. [ /root ] root@myserver1 #
Check the new size using df
[ /root ]
root@myserver1 # df -k /opt/apache
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vgapp-apache
15242716 32840 15208976 1% /opt/apache
[ /root ]
root@myserver1 #
We can see the additional space has been added.
Conclusion
Even though this is a somewhat radical example of LVM, there are useful, everyday, procedures for using LVM in your environment. As usual, set up a lab environment and learn what you can and can’t do with LVM and its associated tools.
Enjoy!
No comments yet