SUSE Conversations


Using Logical Volume Management (LVM) to Organize Your Disks on SLES 10

mfaris01

By: mfaris01

May 5, 2010 10:23 am

Reads:11604

Comments:0

Rating:5.0

Gone are the days when we build a SUSE Enterprise Linux Server (SLES) and simply create one data partition for root (/) and one swap partition. Today’s Linux servers have reached a level of maturity in the enterprise that calls for more file systems that are more versatile, configurable and manageable. Linux Logical Volume Management (LVM), using Volume Groups and individual file systems allows for this flexibility.

In this article, we will walk through configuring your disk devices to use the various utilities included with LVM. We will also show how to control, as administrators, how much disk allocation the various components or applications are allowed to function properly.

NOTE: Be aware that the examples in this document were performed using a bootable USB with SLES 10 SP2. The disks are blank and the installation of Linux was performed manually. Refer to Novell’s SUSE Linux Enterprise Server 10 documentation for creating and configuring LVM during installation. The methods used in this example can be used for additional disk devices not configured during the YaST installation process. This is truly for the Linux Expert and should not be attempted by beginners, but to use as a reference guide as to the true flexibility of Linux.

General

I know you’re wondering why you should do something like this, when you’ve done just fine with a root (/) partition. Ever see what happens to a NetWare server when SYS: volume fills up? The same thing happens to a Linux server when root (/) maxes out. Did you store your NetWare Server data on SYS:? Or did you create a VOL1 or similar? LVM gives you a whole lot more flexibility to managing file systems than non-LVM configurations ever did. Some of the pros to LVM over traditional (physical) partitioning are:

  • Add physical disks to an Logical Volume without shutting the system down, assuming you use hot-swappable disk systems.
  • Take a snapshot of the Logical Volume during normal operation.
  • Expand a Logical Volume on the fly.
  • Multiple physical volumes can be combined into one or more Logical Volume.
  • Up to 256 Logical Volumes can be added, Traditional (physical) partitioning only allows for 4 Primary and 15 Logical partitions.
  • There’s one other feature, although not recommended unless you’re deploying RAID 5 or better, Disk Striping.

Layout and Design

Deciding how granular your Logical Volumes and Volume Groups is really up to you and your needs in your organization. I will recommend that you create standards for a base server configuration and enforce it. Leaving enough disk space unused for various applications. If you are using a virtual environment, create a template and use that for your standard.

I have found that for most of what I need generally, including monitoring, server management, OS and two swap files is a 50GB disk. Make sure you adjust your sizes to accommodate your organizational needs. Most of the servers I use are virtual, which makes it easier to reduce the waste of today’s larger disks that come with most physical servers.

For ancillary applications, I add an additional 20GB disk device or larger, depending on the needs of the project. Web servers, Application Servers, etc, will be loaded there. Note that these are for SLES only. No Open Enterprise Server components are put into consideration.

The SLES installation utility will suggest a very basic partitioning scheme. It uses traditional (physical) partitioning by default. LVM is under Expert Configuration.

Although possible, it is not recommended to use LVM for your /boot and SWAP partitions. I wouldn’t recommend it either.

Setting up Logical Volume Management

On our system installation example, we’ll setup LVM as part of the initial SLES installation. As far as naming and sizes, refer to your own organizational needs and/or specifications.

There are basically, four steps to setting up LVM.

Create physical disk partition with type 8e (LVM)
Create physical volumes
Create volume groups
Create logical volumes

Pre-plan and write down what names you want for Physical Volumes, Volume Groups and Logical Volumes. Doing this prior to installation will make your partitioning configuration flow more smoothly without you wondering what labels to place on the various components.

Here is an example of what could be configured using LVM

I will use the two disk devices mentioned earlier:

50GB System and 20GB Application
/dev/sda1 500M /boot
/dev/sda2 2.0G SWAP
/dev/mapper/vgroot-root 3.0G /
/dev/mapper/vgroot-home 9.0G /home
/dev/mapper/vgroot-opt 5.0G /opt
/dev/mapper/vgroot-tmp 1.0G /tmp
/dev/mapper/vgroot-usr 5.0G /usr
/dev/mapper/vgroot-var 5.0G /var
Total = 30.5G

/dev/sdb1 2.0G SWAP
/dev/mapper/vgapp-apache 10G /opt/apache
Total = 12.0G

You can see we still have some room to play, should we need slight adjustments. With LVM, we can add more space on the fly, if needs arise.

In our example, you’ll notice that the /boot and both SWAP partitions are physical disk types. Because LVM uses Device Mappings for Volume Groups, the path of /dev/mapper/ is default. I created two different Volume Groups, “vgroot” and “vgapp.” I used the mount point names as Logical Volumes for each. The system inserts a “-” in the path for each Logical Volume. This makes the logical volumes more manageable when it comes time for backup or maintenance. You may also want to use individual application names that are targeted for that logical volume, such as “vgapp-apache.” If you do not use a centralized log repository, then you might want to consider creating an additional logical volume for /var/log.

In the example above, you may question why I only gave 3.0GB to root (/). I’ve partitioned off /var, /opt, /usr, /tmp and /home onto their own partitions. This eliminates their need for space from the root (/) partition.

Creating Disk Partitions Suitable for LVM

All operations are performed as root.

We have our /boot partition already created. Now we can create our additional partitions for LVM. I prefer the command line, you can use the YaST utility, the end results are the same.

In our example, we have /dev/sda and /dev/sdb disk devices.

Run fdisk to partition the remainder of /dev/sda for the OS and our root partitions.

[ /root ]
root@myserver1 # fdisk /dev/sda
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help):

In fdisk, there are several options once you load it. If you forget your commands, type ‘m’ and it will display the menu.

Command (m for help): m

Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help):

Let’s print (list) the existing partition table for /dev/sda

Command (m for help): p

Disk /dev/sda: 51.4 GB, 52474836480 bytes
255 heads, 63 sectors/track, 6379 cylinders 
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         62       59898+   83  Linux

Command (m for help):

We can see /boot and we’ve set it to 500MB as defined earlier. Now we’re going to add a 2GB SWAP.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (63-6379, default 63):
Using default value 63
Last cylinder or +size or +sizeM or +sizeK (63-6379, default 6379): +2G

Command (m for help): t
Selected partition 2
Hex code (type L to list codes): L

 0  Empty           1e  Hidden W95 FAT1 80  Old Minix       be  Solaris boot
 1  FAT12           24  NEC DOS         81  Minix / old Lin bf  Solaris
 2  XENIX root      39  Plan 9          82  Linux swap / So c1  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  83  Linux           c4  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 5  Extended        41  PPC PReP Boot   85  Linux extended  c7  Syrinx
 6  FAT16           42  SFS             86  NTFS volume set da  Non-FS data
 7  HPFS/NTFS       4d  QNX4.x          87  NTFS volume set db  CP/M / CTOS / .
 8  AIX             4e  QNX4.x 2nd part 88  Linux plaintext de  Dell Utility
 9  AIX bootable    4f  QNX4.x 3rd part 8e  Linux LVM       df  BootIt
 a  OS/2 Boot Manag 50  OnTrack DM      93  Amoeba          e1  DOS access
 b  W95 FAT32       51  OnTrack DM6 Aux 94  Amoeba BBT      e3  DOS R/O
 c  W95 FAT32 (LBA) 52  CP/M            9f  BSD/OS          e4  SpeedStor
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a0  IBM Thinkpad hi eb  BeOS fs
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a5  FreeBSD         ee  EFI GPT
10  OPUS            55  EZ-Drive        a6  OpenBSD         ef  EFI (FAT-12/16/
11  Hidden FAT12    56  Golden Bow      a7  NeXTSTEP        f0  Linux/PA-RISC b
12  Compaq diagnost 5c  Priam Edisk     a8  Darwin UFS      f1  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       a9  NetBSD          f4  SpeedStor
16  Hidden FAT16    63  GNU HURD or Sys ab  Darwin boot     f2  DOS secondary
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fd  Linux raid auto
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fe  LANstep
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid ff  BBT
1c  Hidden W95 FAT3 75  PC/IX
Hex code (type L to list codes): 82
Changed system type of partition 2 to 82 (Linux swap / Solaris)

Command (m for help):

We’ll use the remaining space on this disk for LVM.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (307-6379, default 307):
Using default value 307
Last cylinder or +size or +sizeM or +sizeK (307-6379, default 6379):
Using default value 6379

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)

Now let’s write it to the partition table.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[ /root ]
root@myserver1 #

Let’s list the partitions to see the table.

[ /root ]
root@myserver1 # fdisk -l  /dev/sda

Disk /dev/sda: 51.4 GB, 52474836480 bytes
255 heads, 63 sectors/track, 6379 cylinders 
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1          62       59898   83  Linux
/dev/sda2	       63         306     1958325   82  Linux swap / Solaris
/dev/sdb2             307        6379    19004895+  8e  Linux LVM

[ /root ]
root@myserver1 #

Now let’s create the second disk’s partitions

[ /root ]
root@myserver1 # fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): 

Print the part table just to be sure that there’s nothing on it. Can never be too sure.

Command (m for help): p

Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System

First, create another 2GB SWAP

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-2610, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-2610, default 2610): +2G

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 82

Changed system type of partition 1 to 82 (Linux swap / Solaris)

Add the remaining for LVM

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (245-2610, default 245):
Using default value 245
Last cylinder or +size or +sizeM or +sizeK (245-2610, default 2610):
Using default value 2610

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

Write it to the partition table

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[ /root ]
root@myserver1 #

Let’s verify that /dev/sdb was written correctly.

[ /root ]
root@myserver1 # fdisk -l  /dev/sdb

Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         244     1959898+  82  Linux swap / Solaris
/dev/sdb2             245        2610    19004895   8e  Linux LVM

[ /root ]
root@myserver1 #

Create our swap files on each disk

[ /root ]
root@myserver1 # mkswap /dev/sda2
Setting up swapspace version 1, size = 2006929 kB

[ /root ]
root@myserver1 # mkswap /dev/sdb1
Setting up swapspace version 1, size = 2006929 kB

[ /root ]
root@myserver1 #

And use swapon to tell the system to use the new swap files.

[ /root ]
root@myserver1 # swapon /dev/sda2

[ /root ]
root@myserver1 # swapon /dev/sdb1

Add your swap files to /etc/fstab

file: /etc/fstab

...

/dev/sda2            swap                 swap       defaults              0 0
/dev/sdb1            swap                 swap       defaults              0 0

...

After you add the swap partitions to /etc/fstab, we can activate them using the “-a” option.

[ /root ]
root@myserver1 # swapon -a

Use the “-s” option to see the swap files and usage.

[ /root ]
root@myserver1 # swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda2                               partition       1959888 0       -1
/dev/sdb1                               partition       1959888 0       -2

[ /root ]
root@myserver1 #

Now that we have our swap partitions up and swapping, let’s set up LVM.

Physical Volumes

When we create our physical volume (PV), we use the pvcreate utility. pvcreate initializes the partition(s) for later use by the LVM. Each PV can be a disk partition, whole disk, meta device, or loopback file. You can specify multiple partitions using the pvcreate command.

Since we want to keep the OS from other disk systems, we’ll make a separate PV.

[ /root ]
root@myserver1 # pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created

[ /root ]
root@myserver1 # pvcreate /dev/sdb2
Physical volume "/dev/sdb2" successfully created

Volume Groups

We use vgcreate to create the volume groups (VG). Vgcreate creates a new volume group called VolumeGroupName using the block special device PhysicalVolumePath previously configured for LVM with pvcreate.

You can add multiple Physical Volumes to your Volume Group.

[ /root ]
root@myserver1 # vgcreate /dev/vgroot /dev/sda3
   Volume group "/dev/vgroot" successfully created

[ /root ]
root@myserver1 # vgcreate /dev/vgapp /dev/sdb2
   Volume group "/dev/vgapp" successfully created

Logical Volumes

lvcreate creates a new logical volume (LV) in a volume group. I separate logical volumes and name them according to their mount point I want. Since our example has multiple mount points for the OS, we’ll a few to make. Notice that the Volume Group remains the same for the different logical volumes.

[ /root ]
root@myserver1 # lvcreate --name root --size 3G /dev/vgroot
   Logical volume "root" created

[ /root ]
root@myserver1 # lvcreate --name home --size 9G /dev/vgroot
   Logical volume "home" created

[ /root ]
root@myserver1 # lvcreate --name opt --size 5G /dev/vgroot
   Logical volume "opt" created

[ /root ]
root@myserver1 # lvcreate --name tmp --size 1G /dev/vgroot
   Logical volume "tmp" created

[ /root ]
root@myserver1 # lvcreate --name usr --size 5G /dev/vgroot
   Logical volume "usr" created

[ /root ]
root@myserver1 # lvcreate --name var --size 5G /dev/vgroot
   Logical volume "var" created

[ /root ]
root@myserver1 #

This last one is for our application Apache.

[ /root ]
root@myserver1 # lvcreate --name apache --size 10G /dev/vgapp
   Logical volume "apache" created

[ /root ]
root@myserver1 #

Now all of the newly created logical volumes need to be formatted with the file system you choose. For this example, we’re using ReiserFS for the format.

[ /root ]
root@myserver1 # echo y | mkreiserfs /dev/vgapp/apache
      ...
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..ok
ReiserFS is successfully created on /dev/vgapp/apache.

[ /root ]
root@myserver1 # 

Format the remaining logical volumes.

Edit /etc/fstab and put your mount points for each newly created file system.

/dev/mapper/vgapp-apache    /opt/apache       reiserfs    acl,user_xattr    1 2

Use the Dump and fsck options (the last two numbers) at your discretion. Use the man pages for fstab to view the options.

Once you’ve added your /etc/fstab entries, mount everything with mount’s -a option.

[ /root ]
root@myserver1 # mount -a

[ /root ]
root@myserver1 #

Use df to view your mounted file systems. We’ll just look at one of the mount points to save time.

[ /root ]
root@myserver1 # df -k /opt/apache
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vgapp-apache
                       5242716     32840   5208976   1% /opt/apache

[ /root ]
root@myserver1 #

Extending a Volume

We’ll use two examples of extending a volume (adding more space) to show a simple case and a more involved example of adding an additional disk.

Example 1: Adding 5GB to a Logical Volume.

Using the setup for our /opt/apache application mount point, we will add 5GB to it, as there is 12GB available space on the /dev/sdb2 PV.

[ /root ]
root@myserver1 # lvextend -L +5G /dev/vgapp/apache
Extending logical volume apache to 15.1 GB
  Logical volume apache successfully resized

[ /root ]
root@myserver1 #

Now the file system needs to be extended also. We can use resize-reiserfs command for this, since our examples are of type reiserfs. For ext3, use resizefs for online file system expansion.

[ /root ]
root@myserver1 # resize_reiserfs /dev/vgapp/apache
resize_reiserfs: On-line resizing /dev/vgapp/apache.
...
resize_reiserfs: Resizing finished successfully.

[ /root ]
root@myserver1 #

If you are not comfortable with online expansion, both commands work with unmounted file systems also.

Note: Expanding specific OS file systems, /home, /var, /usr, /tmp and /opt require booting into single-user mode first.

Example 2: Adding an additional disk to /opt/apache

We’ll assume you’ve already added the disk or virtual disk to your server and booted. In this example, we’ll use /dev/sdc for our disk device name.

You can use the fdisk example above to fdisk the new device. The entire disk will be used for LVM. We’ll start at that point.

Start by creating a new Physical Volume for the new disk.

[ /root ]
root@myserver1 # pvcreate /dev/sdc1
Physical volume "/dev/sdc1" successfully created

Extend the Volume Group vgapp to include the new physical volume.

[ /root ]
root@myserver1 # vgextend vgapp /dev/sdc1
Volume group "vgapp" successfully extended

[ /root ]
root@myserver1 # 

Use vgdisplay the volume group information.

[ /root ]
root@myserver1 # vgdisplay vgapp
--- Volume group ---
VG Name                           vgapp
System ID
Format                                lvm2
Metadata Areas                1
Metadata Sequence No  6
VG Access                         read/write
VG Status                           resizable
MAX LV                               0
Cur LV                                1
Open LV                             1
Max PV                                0
Cur PV                                 2
Act PV                                  2
VG Size                               121.74 GB
PE Size                               4.00 MB
Total PE                              53438
Alloc PE / Size                    35840 / 120.00 GB
Free  PE / Size                   17598 / 100.74 GB
VG UUID               GNrQo7-FFmg-ohzP-wMLY-wGps-I8IK-6ZEeD5

[ /root ]
root@myserver1 #

We can see that there are 2 Physical Volumes (Cur PV) in our Volume Group now.

Now we want to add half of the 100GB of free space to our /dev/vgapp/apache logical volume.

[ /root ]
root@myserver1 # lvextend --extents +50%FREE /dev/vgapp/apache
Extending logical volume apache to 61.1 GB
  Logical volume apache successfully resized

[ /root ]
root@myserver1 # 

Resize the file system.

[ /root ]
root@myserver1 # resize_reiserfs /dev/vgapp/apache
resize_reiserfs: On-line resizing /dev/vgapp/apache.
...
resize_reiserfs: Resizing finished successfully.

[ /root ]
root@myserver1 #

Check the new size using df

[ /root ]
root@myserver1 # df -k /opt/apache
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/vgapp-apache
                      15242716     32840  15208976   1% /opt/apache

[ /root ]
root@myserver1 #

We can see the additional space has been added.

Conclusion

Even though this is a somewhat radical example of LVM, there are useful, everyday, procedures for using LVM in your environment. As usual, set up a lab environment and learn what you can and can’t do with LVM and its associated tools.

Enjoy!

VN:D [1.9.22_1171]
Rating: 5.0/5 (2 votes cast)
VN:F [1.9.22_1171]
Rating: +2 (from 2 votes)
Using Logical Volume Management (LVM) to Organize Your Disks on SLES 10, 5.0 out of 5 based on 2 ratings

Tags:
Categories: SUSE Linux Enterprise Server, Technical Solutions

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

Comment

RSS