Before configuring multipath I/O for your SAN devices, prepare the SAN devices, as necessary, by doing the following:
Configure and zone the SAN with the vendor’s tools.
Configure permissions for host LUNs on the storage arrays with the vendor’s tools.
Install the Linux HBA driver module. Upon module installation, the driver automatically scans the HBA to discover any SAN devices that have permissions for the host. It presents them to the host for further configuration.
NOTE:Ensure that the HBA driver you are using does not have native multipathing enabled.
See the vendor’s specific instructions for more details.
After the driver module is loaded, discover the device nodes assigned to specific array LUNs or partitions.
If the SAN device will be used as the root device on the server, modify the timeout settings for the device as described in Section 5.2.6, SAN Timeout Settings When the Root Device Is Multipathed.
If the LUNs are not seen by the HBA driver, lsscsi can be used to check whether the SCSI devices are seen correctly by the operating system. When the LUNs are not seen by the HBA driver, check the zoning setup of the SAN. In particular, check whether LUN masking is active and whether the LUNs are correctly assigned to the server.
If the LUNs are seen by the HBA driver, but there are no corresponding block devices, additional kernel parameters are needed to change the SCSI device scanning behavior, such as to indicate that LUNs are not numbered consecutively. For information, see Options for SCSI Device Scanning in the Novell Support Knowledgebase.
Partitioning devices that have multiple paths is not recommended, but it is supported.
In SUSE Linux Enterprise Server 10, you can use the kpartx tool to create partitions on multipathed devices without rebooting. You can also partition the device before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool.
In SUSE Linux Enterprise Server 9, if you want to partition the device, you should configure its partitions before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool. This is necessary because partitioning an existing multipathed device is not supported. Partitioning operations on multipathed devices fail if attempted.
If you configure partitions for a device, DM-MP automatically recognizes the partitions and indicates them by appending p1 to pn to the device’s ID, such as
/dev/disk/by-id/26353900f02796769p1
To partition multipathed devices, you must disable the DM-MP service, partition the normal device node (such as /dev/sdc), then reboot to allow the DM-MP service to see the new partitions.
The system must be manually configured to automatically load the device drivers for the controllers to which the multipath I/O devices are connected within the INITRD. Therefore add the needed driver module to the variable INITRD_MODULES in the file /etc/sysconfig/kernel.
For example, if your system contains a RAID controller accessed by the cciss driver and multipathed devices connected to a QLogic controller accessed by the driver qla2xxx, this entry would look like:
INITRD_MODULES="cciss"
Because the QLogic driver is not automatically loaded on start-up, add it here:
INITRD_MODULES="cciss qla23xx"
After having changed /etc/sysconfig/kernel, you must re-create the INITRD on your system with the command mkinitrd, then reboot in order for the changes to take effect.
When you are using LILO as a boot manager, reinstall it with the command /sbin/lilo. No further action is required if you are using GRUB.
Use either of the methods in this section to add multipath I/O services (multipathd) to the boot sequence.
In YaST, click > > .
Select , then click .
Click to acknowledge the service startup message.
Click , then click .
The changes do not take affect until the server is restarted.
Open a terminal console, then log in as the root user or equivalent.
At the terminal console prompt, enter
insserv multipathd
The /etc/multipath.conf file does not exist unless you create it. The /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file contains a sample /etc/multipath.conf file that you can use as a guide for multipath settings. See /usr/share/doc/packages/multipath-tools/multipath.conf.annotated for a template with extensive comments for each of the attributes and their options.
If the /etc/multipath.conf file does not exist, copy the example to create the file:
In a terminal console, log in as the root user.
Enter the following command (all on one line, of course) to copy the template:
cp /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic /etc/multipath.conf
Use the /usr/share/doc/packages/multipath-tools/multipath.conf.annotated file as a reference to determine how to configure multipathing for your system.
Ensure that there is an appropriate device entry for your SAN. Most vendors provide documentation on the proper setup of the device section.
The /etc/multipath.conf file requires a different device section for different SANs. If you are using a storage subsystem that is automatically detected (see Tested Storage Arrays for Multipathing Support), the default entry for that device can be used; no further configuration of the /etc/multipath.conf file is required.
Save the file.
After setting up the configuration, you can perform a dry run
by entering
multipath -v2 -d
This command scans the devices, then displays what the setup would look like. The output is similar to the following:
26353900f02796769 [size=127 GB] [features="0"] [hwhandler="1 emc"]
\_ round-robin 0 [first] \_ 1:0:1:2 sdav 66:240 [ready ] \_ 0:0:1:2 sdr 65:16 [ready ]
\_ round-robin 0 \_ 1:0:0:2 sdag 66:0 [ready ] \_ 0:0:0:2 sdc 8:32 [ready ]
Paths are grouped into priority groups. Only one priority group is in active use at a time. To model an active/active configuration, all paths end in the same group. To model active/passive configuration, the paths that should not be active in parallel are placed in several distinct priority groups. This normally happens automatically on device discovery.
The output shows the order, the scheduling policy used to balance I/O within the group, and the paths for each priority group. For each path, its physical address (host:bus:target:lun), device node name, major:minor number, and state is shown.
By using a verbosity level of -v3 in the dry run, you can see all detected paths, multipaths, and device maps. Both wwid and device node blacklisted devices are displayed.
multipath -v3 d
The following is an example of -v3 output on a 64-bit SLES server with two Qlogic HBA connected to a Xiotech Magnitude 3000 SAN. Some multiple entries have been omitted to shorten the example.
dm-22: device node name blacklisted < content omitted > loop7: device node name blacklisted < content omitted > md0: device node name blacklisted < content omitted > dm-0: device node name blacklisted sdf: not found in pathvec sdf: mask = 0x1f sdf: dev_t = 8:80 sdf: size = 105005056 sdf: subsystem = scsi sdf: vendor = XIOtech sdf: product = Magnitude 3D sdf: rev = 3.00 sdf: h:b:t:l = 1:0:0:2 sdf: tgt_node_name = 0x202100d0b2028da sdf: serial = 000028DA0014 sdf: getuid = /sbin/scsi_id -g -u -s /block/%n (config file default) sdf: uid = 200d0b2da28001400 (callout) sdf: prio = const (config file default) sdf: const prio = 1 < content omitted > ram15: device node name blacklisted < content omitted > ===== paths list ===== uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev 200d0b2da28001400 1:0:0:2 sdf 8:80 1 [undef][undef] XIOtech,Magnitude 3D 200d0b2da28005400 1:0:0:1 sde 8:64 1 [undef][undef] XIOtech,Magnitude 3D 200d0b2da28004d00 1:0:0:0 sdd 8:48 1 [undef][undef] XIOtech,Magnitude 3D 200d0b2da28001400 0:0:0:2 sdc 8:32 1 [undef][undef] XIOtech,Magnitude 3D 200d0b2da28005400 0:0:0:1 sdb 8:16 1 [undef][undef] XIOtech,Magnitude 3D 200d0b2da28004d00 0:0:0:0 sda 8:0 1 [undef][undef] XIOtech,Magnitude 3D params = 0 0 2 1 round-robin 0 1 1 8:80 1000 round-robin 0 1 1 8:32 1000 status = 2 0 0 0 2 1 A 0 1 0 8:80 A 0 E 0 1 0 8:32 A 0 sdf: mask = 0x4 sdf: path checker = directio (config file default) directio: starting new request directio: async io getevents returns 1 (errno=Success) directio: io finished 4096/0 sdf: state = 2 < content omitted >
A multipath device can be identified by its WWID, by a user-friendly name, or by an alias that you assign for it. Table 5-4 describes the types of device names that can be used for a device in the /etc/multipath.conf file.
Table 5-4 Comparison of Multipath Device Name Types
|
Name Types |
Description |
|---|---|
|
WWID (default) |
The WWID (Worldwide Identifier) is an identifier for the multipath device that is guaranteed to be globally unique and unchanging. The default name used in multipathing is the ID of the logical unit as found in the /dev/disk/by-id directory. Because device node names in the form of /dev/sdn and /dev/dm-n can change on reboot, referring to multipath devices by their ID is preferred. |
|
User-friendly |
The Device Mapper Multipath device names in the /dev/mapper directory also reference the ID of the logical unit. These multipath device names are user-friendly names in the form of /dev/mapper/mpath<n>, such as /dev/mapper/mpath0. The names are unique and persistent because they use the /var/lib/multipath/bindings file to track the association between the UUID and user-friendly names. |
|
Alias |
An alias name is a globally unique name that the administrator provides for a multipath device. Alias names override the WWID and the user-friendly /dev/mapper/mpathN names. |
The global multipath user_friendly_names option in the /etc/multipath.conf file is used to enable or disable the use of user-friendly names for multipath devices. If it is set to “no” (the default), multipath uses the WWID as the name of the device. If it is set to “yes”, multipath uses the /var/lib/multipath/bindings file to assign a persistent and unique name to the device in the form of mpath<n>. The bindings_file option in the /etc/multipath.conf file can be used to specify an alternate location for the bindings file.
The global multipath alias option in the /etc/multipath.conf file is used to explicitly assign a name to the device. If an alias name is set up for a multipath device, the alias is used instead of the WWID or the user-friendly name.
Using the user_friendly_names option can be problematic in the following situations:
Root Device Is Using Multipath: If the system root device is using multipath and you use the user_friendly_names option, the user-friendly settings in the /var/lib/multipath/bindings file are included in the initrd. If you later change the storage setup, such as by adding or removing devices, there is a mismatch between the bindings setting inside the initrd and the bindings settings in /var/lib/multipath/bindings.
WARNING:A bindings mismatch between initrd and /var/lib/multipath/bindings can lead to a wrong assignment of mount points to devices, which can result in file system corruption and data loss.
To avoid this problem, we recommend that you use the default WWID settings for the system root device. You can also use the alias option to override the user_friendly_names option for the system root device in the /etc/multipath.conf file.
For example:
multipaths {
multipath {
wwid 36006048000028350131253594d303030
alias mpatha
}
multipath {
wwid 36006048000028350131253594d303041
alias mpathb
}
multipath {
wwid 36006048000028350131253594d303145
alias mpathc
}
multipath {
wwid 36006048000028350131253594d303334
alias mpathd
}
}
IMPORTANT:We recommend that you do not use aliases for the system root device, because the ability to seamlessly switch off multipathing via the kernel command line is lost because the device name differs.
Mounting /var from Another Partition: The default location of the user_friendly_names configuration file is /var/lib/multipath/bindings. If the /var data is not located on the system root device but mounted from another partition, the bindings file is not available when setting up multipathing.
Ensure that the /var/lib/multipath/bindings file is available on the system root device and multipath can find it. For example, this can be done as follows:
Move the /var/lib/multipath/bindings file to /etc/multipath/bindings.
Set the bindings_file option in the defaults section of /etc/multipath.conf to this new location. For example:
defaults {
user_friendly_names yes
bindings_file "/etc/multipath/bindings"
}
Multipath Is in the initrd: Even if the system root device is not on multipath, it is possible for multipath to be included in the initrd. For example, this can happen of the system root device is on LVM. If you use the user_friendly_names option and multipath is in the initrd, you should boot with the parameter multipath=off to avoid problems.
This disables multipath only in the initrd during system boots. After the system boots, the boot.multipath and multipathd boot scripts are able to activate multipathing.
For an example of multipath.conf settings, see the /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file.
To enable user-friendly names or to specify aliases:
In a terminal console, log in as the root user.
Open the /etc/multipath.conf file in a text editor.
(Optional) Modify the location of the /var/lib/multipath/bindings file.
The alternate path must be available on the system root device where multipath can find it.
Move the /var/lib/multipath/bindings file to /etc/multipath/bindings.
Set the bindings_file option in the defaults section of /etc/multipath.conf to this new location. For example:
defaults {
user_friendly_names yes
bindings_file "/etc/multipath/bindings"
}
(Optional) Enable user-friendly names:
Uncomment the defaults section and its ending bracket.
Uncomment the user_friendly_names option, then change its value from No to Yes.
For example:
## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
}
(Optional) Specify your own names for devices by using the alias option in the multipath section.
For example:
## Use alias names, instead of using WWIDs as names.
multipaths {
multipath {
wwid 36006048000028350131253594d303030
alias mpatha
}
multipath {
wwid 36006048000028350131253594d303041
alias mpathb
}
multipath {
wwid 36006048000028350131253594d303145
alias mpathc
}
multipath {
wwid 36006048000028350131253594d303334
alias mpathd
}
}
Save your changes, then close the file.
The /etc/multipath.conf file should contain a blacklist section where all non-multipathed devices should be listed. For example, local IDE hard drives and floppy drives are not normally multipathed. If you have single-path devices that multipath is trying to manage and you want multipath to ignore them, put them in the blacklist section to resolve the problem.
NOTE:Beginning in SLES 10 SP3, the keyword devnode_blacklist has been deprecated and replaced with the keyword blacklist.
For example, to blacklist local devices and all arrays from the cciss driver from being managed by multipath, the blacklist section looks like this:
blacklist {
wwid 26353900f02796769
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*"
devnode "^hd[a-z][0-9]*"
devnode "^cciss!c[0-9]d[0-9].*"
}
You can also blacklist only the partitions from a driver instead of the entire array. For example, using the following regular expression would blacklist only partitions from the cciss driver and not the entire array:
^cciss!c[0-9]d[0-9]*[p[0-9]*]
After you modify the /etc/multipath.conf file, you must run mkinitrd to re-create the INITRD on your system, then reboot in order for the changes to take effect.
Afterwards, the local devices should no longer be listed in the multipath maps when you issue the multipath -ll command.
The /etc/multipath.conf file should contain a defaults section where you can specify default behaviors. If the field is not otherwise specified in a device section, the default setting is applied for that SAN configuration.
The following defaults section specifies a simple failover policy:
defaults {
multipath_tool "/sbin/multipath -v0"
udev_dir /dev
polling_interval 10
default_selector "round-robin 0"
default_path_grouping_policy failover
default_getuid "/sbin/scsi_id -g -u -s /block/%n"
default_prio_callout "/bin/true"
default_features "0"
rr_min_io 100
failback immediate
NOTE:In the default_getuid command line, use the path /sbin/scsi_id as shown in the above example instead of the sample path of /lib/udev/scsi_id that is found in the sample file /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic file (and in the default and annotated sample files).
Changes to the /etc/multipath.conf file cannot take effect when multipathd is running. After you make changes, save and close the file, then do the following to apply the changes:
Stop the multipathd service.
Clear old multipath bindings by entering
/sbin/multipath -F
Create new multipath bindings by entering
/sbin/multipath -v2 -l
Start the multipathd service.
Run mkinitrd to re-create the INITRD on your system, then reboot in order for the changes to take effect.