5.4 Configuring the System for Multipathing

5.4.1 Preparing SAN Devices for Multipathing

Before configuring multipath I/O for your SAN devices, prepare the SAN devices, as necessary, by doing the following:

  • Configure and zone the SAN with the vendor’s tools.

  • Configure permissions for host LUNs on the storage arrays with the vendor’s tools.

  • Install the Linux HBA driver module. Upon module installation, the driver automatically scans the HBA to discover any SAN devices that have permissions for the host. It presents them to the host for further configuration.

    NOTE:Ensure that the HBA driver you are using does not have native multipathing enabled.

    See the vendor’s specific instructions for more details.

  • After the driver module is loaded, discover the device nodes assigned to specific array LUNs or partitions.

  • If the SAN device will be used as the root device on the server, modify the timeout settings for the device as described in Section 5.2.6, SAN Timeout Settings When the Root Device Is Multipathed.

If the LUNs are not seen by the HBA driver, lsscsi can be used to check whether the SCSI devices are seen correctly by the operating system. When the LUNs are not seen by the HBA driver, check the zoning setup of the SAN. In particular, check whether LUN masking is active and whether the LUNs are correctly assigned to the server.

If the LUNs are seen by the HBA driver, but there are no corresponding block devices, additional kernel parameters are needed to change the SCSI device scanning behavior, such as to indicate that LUNs are not numbered consecutively. For information, see Options for SCSI Device Scanning in the Novell Support Knowledgebase.

5.4.2 Partitioning Multipathed Devices

Partitioning devices that have multiple paths is not recommended, but it is supported.

SUSE Linux Enterprise Server 10

In SUSE Linux Enterprise Server 10, you can use the kpartx tool to create partitions on multipathed devices without rebooting. You can also partition the device before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool.

SUSE Linux Enterprise Server 9

In SUSE Linux Enterprise Server 9, if you want to partition the device, you should configure its partitions before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool. This is necessary because partitioning an existing multipathed device is not supported. Partitioning operations on multipathed devices fail if attempted.

If you configure partitions for a device, DM-MP automatically recognizes the partitions and indicates them by appending p1 to pn to the device’s ID, such as

/dev/disk/by-id/26353900f02796769p1

To partition multipathed devices, you must disable the DM-MP service, partition the normal device node (such as /dev/sdc), then reboot to allow the DM-MP service to see the new partitions.

5.4.3 Configuring the Server for Multipathing

The system must be manually configured to automatically load the device drivers for the controllers to which the multipath I/O devices are connected within the INITRD. Therefore add the needed driver module to the variable INITRD_MODULES in the file /etc/sysconfig/kernel.

For example, if your system contains a RAID controller accessed by the cciss driver and multipathed devices connected to a QLogic controller accessed by the driver qla2xxx, this entry would look like:

   INITRD_MODULES="cciss"

Because the QLogic driver is not automatically loaded on start-up, add it here:

   INITRD_MODULES="cciss qla23xx"

After having changed /etc/sysconfig/kernel, you must re-create the INITRD on your system with the command mkinitrd, then reboot in order for the changes to take effect.

When you are using LILO as a boot manager, reinstall it with the command /sbin/lilo. No further action is required if you are using GRUB.

5.4.4 Adding multipathd to the Boot Sequence

Use either of the methods in this section to add multipath I/O services (multipathd) to the boot sequence.

YaST

  1. In YaST, click System > System Services (Runlevel) > Simple Mode.

  2. Select multipathd, then click Enable.

  3. Click OK to acknowledge the service startup message.

  4. Click Finish, then click Yes.

    The changes do not take affect until the server is restarted.

Command Line

  1. Open a terminal console, then log in as the root user or equivalent.

  2. At the terminal console prompt, enter

    insserv multipathd
    

5.4.5 Creating and Configuring the /etc/multipath.conf File

The /etc/multipath.conf file does not exist unless you create it. The /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file contains a sample /etc/multipath.conf file that you can use as a guide for multipath settings. See /usr/share/doc/packages/multipath-tools/multipath.conf.annotated for a template with extensive comments for each of the attributes and their options.

Creating the multipath.conf File

If the /etc/multipath.conf file does not exist, copy the example to create the file:

  1. In a terminal console, log in as the root user.

  2. Enter the following command (all on one line, of course) to copy the template:

    cp /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic /etc/multipath.conf
    
  3. Use the /usr/share/doc/packages/multipath-tools/multipath.conf.annotated file as a reference to determine how to configure multipathing for your system.

  4. Ensure that there is an appropriate device entry for your SAN. Most vendors provide documentation on the proper setup of the device section.

    The /etc/multipath.conf file requires a different device section for different SANs. If you are using a storage subsystem that is automatically detected (see Tested Storage Arrays for Multipathing Support), the default entry for that device can be used; no further configuration of the /etc/multipath.conf file is required.

  5. Save the file.

Verifying the Setup in the etc/multipath.conf File

After setting up the configuration, you can perform a dry run by entering

multipath -v2 -d

This command scans the devices, then displays what the setup would look like. The output is similar to the following:

26353900f02796769
[size=127 GB]
[features="0"]
[hwhandler="1    emc"] 
\_ round-robin 0 [first]
  \_ 1:0:1:2 sdav 66:240  [ready ]
  \_ 0:0:1:2 sdr  65:16   [ready ]
\_ round-robin 0 
  \_ 1:0:0:2 sdag 66:0    [ready ]
  \_ 0:0:0:2 sdc   8:32   [ready ] 

Paths are grouped into priority groups. Only one priority group is in active use at a time. To model an active/active configuration, all paths end in the same group. To model active/passive configuration, the paths that should not be active in parallel are placed in several distinct priority groups. This normally happens automatically on device discovery.

The output shows the order, the scheduling policy used to balance I/O within the group, and the paths for each priority group. For each path, its physical address (host:bus:target:lun), device node name, major:minor number, and state is shown.

By using a verbosity level of -v3 in the dry run, you can see all detected paths, multipaths, and device maps. Both wwid and device node blacklisted devices are displayed.

multipath -v3 d

The following is an example of -v3 output on a 64-bit SLES server with two Qlogic HBA connected to a Xiotech Magnitude 3000 SAN. Some multiple entries have been omitted to shorten the example.

dm-22: device node name blacklisted
< content omitted >
loop7: device node name blacklisted
< content omitted >
md0: device node name blacklisted
< content omitted >
dm-0: device node name blacklisted
sdf: not found in pathvec
sdf: mask = 0x1f
sdf: dev_t = 8:80
sdf: size = 105005056
sdf: subsystem = scsi
sdf: vendor = XIOtech
sdf: product = Magnitude 3D
sdf: rev = 3.00
sdf: h:b:t:l = 1:0:0:2
sdf: tgt_node_name = 0x202100d0b2028da
sdf: serial = 000028DA0014
sdf: getuid = /sbin/scsi_id -g -u -s /block/%n (config file default)
sdf: uid = 200d0b2da28001400 (callout)
sdf: prio = const (config file default)
sdf: const prio = 1
< content omitted >
ram15: device node name blacklisted
< content omitted >
===== paths list =====
uuid              hcil    dev dev_t pri dm_st  chk_st  vend/prod/rev
200d0b2da28001400 1:0:0:2 sdf 8:80  1   [undef][undef] XIOtech,Magnitude 3D
200d0b2da28005400 1:0:0:1 sde 8:64  1   [undef][undef] XIOtech,Magnitude 3D
200d0b2da28004d00 1:0:0:0 sdd 8:48  1   [undef][undef] XIOtech,Magnitude 3D
200d0b2da28001400 0:0:0:2 sdc 8:32  1   [undef][undef] XIOtech,Magnitude 3D
200d0b2da28005400 0:0:0:1 sdb 8:16  1   [undef][undef] XIOtech,Magnitude 3D
200d0b2da28004d00 0:0:0:0 sda 8:0   1   [undef][undef] XIOtech,Magnitude 3D
params = 0 0 2 1 round-robin 0 1 1 8:80 1000 round-robin 0 1 1 8:32 1000
status = 2 0 0 0 2 1 A 0 1 0 8:80 A 0 E 0 1 0 8:32 A 0
sdf: mask = 0x4
sdf: path checker = directio (config file default)
directio: starting new request
directio: async io getevents returns 1 (errno=Success)
directio: io finished 4096/0
sdf: state = 2
< content omitted >

Configuring User-Friendly Names or Alias Names in /etc/multipath.conf

A multipath device can be identified by its WWID, by a user-friendly name, or by an alias that you assign for it. Table 5-4 describes the types of device names that can be used for a device in the /etc/multipath.conf file.

Table 5-4 Comparison of Multipath Device Name Types

Name Types

Description

WWID (default)

The WWID (Worldwide Identifier) is an identifier for the multipath device that is guaranteed to be globally unique and unchanging. The default name used in multipathing is the ID of the logical unit as found in the /dev/disk/by-id directory. Because device node names in the form of /dev/sdn and /dev/dm-n can change on reboot, referring to multipath devices by their ID is preferred.

User-friendly

The Device Mapper Multipath device names in the /dev/mapper directory also reference the ID of the logical unit. These multipath device names are user-friendly names in the form of /dev/mapper/mpath<n>, such as /dev/mapper/mpath0. The names are unique and persistent because they use the /var/lib/multipath/bindings file to track the association between the UUID and user-friendly names.

Alias

An alias name is a globally unique name that the administrator provides for a multipath device. Alias names override the WWID and the user-friendly /dev/mapper/mpathN names.

The global multipath user_friendly_names option in the /etc/multipath.conf file is used to enable or disable the use of user-friendly names for multipath devices. If it is set to “no” (the default), multipath uses the WWID as the name of the device. If it is set to “yes”, multipath uses the /var/lib/multipath/bindings file to assign a persistent and unique name to the device in the form of mpath<n>. The bindings_file option in the /etc/multipath.conf file can be used to specify an alternate location for the bindings file.

The global multipath alias option in the /etc/multipath.conf file is used to explicitly assign a name to the device. If an alias name is set up for a multipath device, the alias is used instead of the WWID or the user-friendly name.

Using the user_friendly_names option can be problematic in the following situations:

  • Root Device Is Using Multipath: If the system root device is using multipath and you use the user_friendly_names option, the user-friendly settings in the /var/lib/multipath/bindings file are included in the initrd. If you later change the storage setup, such as by adding or removing devices, there is a mismatch between the bindings setting inside the initrd and the bindings settings in /var/lib/multipath/bindings.

    WARNING:A bindings mismatch between initrd and /var/lib/multipath/bindings can lead to a wrong assignment of mount points to devices, which can result in file system corruption and data loss.

    To avoid this problem, we recommend that you use the default WWID settings for the system root device. You can also use the alias option to override the user_friendly_names option for the system root device in the /etc/multipath.conf file.

    For example:

    multipaths {
           multipath {
                   wwid           36006048000028350131253594d303030
                   alias             mpatha
           }
           multipath {
                   wwid           36006048000028350131253594d303041
                   alias             mpathb
           }
           multipath {
                   wwid           36006048000028350131253594d303145
                   alias             mpathc
           }
           multipath {
                   wwid           36006048000028350131253594d303334
                   alias             mpathd
           }
    }
    

    IMPORTANT:We recommend that you do not use aliases for the system root device, because the ability to seamlessly switch off multipathing via the kernel command line is lost because the device name differs.

  • Mounting /var from Another Partition: The default location of the user_friendly_names configuration file is /var/lib/multipath/bindings. If the /var data is not located on the system root device but mounted from another partition, the bindings file is not available when setting up multipathing.

    Ensure that the /var/lib/multipath/bindings file is available on the system root device and multipath can find it. For example, this can be done as follows:

    1. Move the /var/lib/multipath/bindings file to /etc/multipath/bindings.

    2. Set the bindings_file option in the defaults section of /etc/multipath.conf to this new location. For example:

      defaults {
                     user_friendly_names yes
                     bindings_file "/etc/multipath/bindings"
      }
      
  • Multipath Is in the initrd: Even if the system root device is not on multipath, it is possible for multipath to be included in the initrd. For example, this can happen of the system root device is on LVM. If you use the user_friendly_names option and multipath is in the initrd, you should boot with the parameter multipath=off to avoid problems.

    This disables multipath only in the initrd during system boots. After the system boots, the boot.multipath and multipathd boot scripts are able to activate multipathing.

For an example of multipath.conf settings, see the /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file.

To enable user-friendly names or to specify aliases:

  1. In a terminal console, log in as the root user.

  2. Open the /etc/multipath.conf file in a text editor.

  3. (Optional) Modify the location of the /var/lib/multipath/bindings file.

    The alternate path must be available on the system root device where multipath can find it.

    1. Move the /var/lib/multipath/bindings file to /etc/multipath/bindings.

    2. Set the bindings_file option in the defaults section of /etc/multipath.conf to this new location. For example:

      defaults {
                     user_friendly_names yes
                     bindings_file "/etc/multipath/bindings"
      }
      
  4. (Optional) Enable user-friendly names:

    1. Uncomment the defaults section and its ending bracket.

    2. Uncomment the user_friendly_names option, then change its value from No to Yes.

      For example:

      ## Use user friendly names, instead of using WWIDs as names.
      defaults {
        user_friendly_names yes
      }
      
  5. (Optional) Specify your own names for devices by using the alias option in the multipath section.

    For example:

    ## Use alias names, instead of using WWIDs as names.
    multipaths {
           multipath {
                   wwid           36006048000028350131253594d303030
                   alias             mpatha
           }
           multipath {
                   wwid           36006048000028350131253594d303041
                   alias             mpathb
           }
           multipath {
                   wwid           36006048000028350131253594d303145
                   alias             mpathc
           }
           multipath {
                   wwid           36006048000028350131253594d303334
                   alias             mpathd
           }
    }
    
  6. Save your changes, then close the file.

Blacklisting Non-Multipathed Devices in /etc/multipath.conf

The /etc/multipath.conf file should contain a blacklist section where all non-multipathed devices should be listed. For example, local IDE hard drives and floppy drives are not normally multipathed. If you have single-path devices that multipath is trying to manage and you want multipath to ignore them, put them in the blacklist section to resolve the problem.

NOTE:Beginning in SLES 10 SP3, the keyword devnode_blacklist has been deprecated and replaced with the keyword blacklist.

For example, to blacklist local devices and all arrays from the cciss driver from being managed by multipath, the blacklist section looks like this:

blacklist {
      wwid 26353900f02796769
      devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*"
      devnode "^hd[a-z][0-9]*"
      devnode "^cciss!c[0-9]d[0-9].*"
}

You can also blacklist only the partitions from a driver instead of the entire array. For example, using the following regular expression would blacklist only partitions from the cciss driver and not the entire array:

^cciss!c[0-9]d[0-9]*[p[0-9]*]

After you modify the /etc/multipath.conf file, you must run mkinitrd to re-create the INITRD on your system, then reboot in order for the changes to take effect.

Afterwards, the local devices should no longer be listed in the multipath maps when you issue the multipath -ll command.

Configuring Default Multipath Behavior in /etc/multipath.conf

The /etc/multipath.conf file should contain a defaults section where you can specify default behaviors. If the field is not otherwise specified in a device section, the default setting is applied for that SAN configuration.

The following defaults section specifies a simple failover policy:

defaults {
       multipath_tool  "/sbin/multipath -v0"
       udev_dir        /dev
       polling_interval 10
       default_selector        "round-robin 0"
       default_path_grouping_policy    failover
       default_getuid "/sbin/scsi_id -g -u -s /block/%n"
       default_prio_callout    "/bin/true"
       default_features        "0"
       rr_min_io               100
       failback                immediate

NOTE:In the default_getuid command line, use the path /sbin/scsi_id as shown in the above example instead of the sample path of /lib/udev/scsi_id that is found in the sample file /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file (and in the default and annotated sample files).

Applying Changes Made to the /etc/multipath.conf File

Changes to the /etc/multipath.conf file cannot take effect when multipathd is running. After you make changes, save and close the file, then do the following to apply the changes:

  1. Stop the multipathd service.

  2. Clear old multipath bindings by entering

    /sbin/multipath -F
    
  3. Create new multipath bindings by entering

    /sbin/multipath -v2 -l
    
  4. Start the multipathd service.

  5. Run mkinitrd to re-create the INITRD on your system, then reboot in order for the changes to take effect.