There is more than one way to skin a cat
© 2010-2015 Dennis Leeuw dleeuw at made-it dot com
License: GPLv2 or later
To be able to support failover connections to a device one needs multiple paths to that device. If we have a iSCSI disk connected to an ethernet network and we want to make sure that we can reach the disk from a host even when we disconnect one ethernet cable we need two network cables running from the host to the disk and we need a piece of software that makes sure when one link failes the other one is used. This last piece of software is called multipath on GNU/Linux systems. multipath makes use of the device mapper as discussed in a previous article.
User Space | dmsetup | multipath |
libdevmapper | ||
Kernel | Device Mapper |
To make a full redundant system we need two switches and two paths from every point to every point:
When a HBA driver loads or when the iSCSI subsystem is started, the SCSI midlayer initiates a scan of its bus and detects all assigned storage LUNs through every available path, this means that our Host, in the on the left hand side provided image, is able to see a single disk in the Hardware Enclosure through four different paths. With multipath enabled this means that 4 different disks will be registered by the system:
Block device | Path | Mode | Multipath device |
/dev/sdb | Green, Blue | active | /dev/dm-1 |
/dev/sdc | Green, Roze | passive | |
/dev/sdd | Grey, Purple | active | |
/dev/sde | Grey, Red | passive |
This means that /proc/partitions contains four more disks.
This over comes the following failures:
The DM Multipath system consists of a kernel module called dm-multipath.ko or just multipath.ko, the multipath commandline tool to configure multipath devices and multipathd the watchdog that monitors the paths and takes action when paths fail or come back. The multipathd is configured through the /etc/multipath.conf configuration file.
Userspace Applications | |||||
---|---|---|---|---|---|
ioctl interface | filesystem interface | ||||
control interface | block interface | ||||
core device-mapper | |||||
mapping / target interface | |||||
linear | mirror | snapshot | multipath | ||
path selectors | hardware handlers |
There are two different ways in which multipaths can be used. The first option, symmetric, is to use every path to the device to provide for higher through put, the other option is to use one path for access and the other for hot-standby.
Asymmetric arrays devide paths to the same LUN in active/passive groups
Asymmetric arrays assign a LUN to a one of the two storage processors in the system. Only when a problem occurs the LUN is assigned to the secondary processor. This process is known as LUN trespassing. LUN trespassing is achieved by sending a device-dependent trespassing command to the storage array.
The choice for symmetric of asymmetric is dependend on the the provider of the storage array(?).
In an ideal world multipathd would start and finds everything on its own and handles everything automatically, but this is not an ideal world. So sometimes we need to help multipathd a bit by providing settings in the /etc/multipath.conf file. Experience has tought me to do as little as possible and have the daemon figure out as much on its own as is possible.
Per default the multipath system will create a dm-X device (where X is a number) as a single device that reroutes I/O to the underlying disks. If you set the user_friendly_names option in the /etc/multipath.conf file an alias is created in /dev/mapper/. The /dev/dm-X device should never be used since it is as volatile as the other device names. The name is handed out on a first come first serve manner and can thus change after a reboot. The /dev/mapper/ device is bound to the WWID of the multipath device and is stable over reboots. Use this name to configure and setup your multipath system.
If you set the user_friendly_names option to no the link in /dev/mapper/ is the WWID of the multipath-device. This is the default.
If you set the user_friendly_names option to yes a /dev/mapper/mpath-N link is created, where N is a number. The mapping between the WWID and the user friendly name is maintained in the /var/lib/multipath/bindings file. If you want your bindings to be identical on all your hosts distribute this file to all your systems. You can even set the bindings_file option in /etc/multipath.conf if you want this file to be placed in a different place then the default one e.g. /etc/multipath.d/bindings.
Our advise is to use the default, since it is the easiest and most uniform way of handling multipath devices.
We start our configuration file with some defaults. The provided values are the default values for these options, so nothing fancy. This section is only provided to give you an idea of how it works. For more details use man 5 multipath.conf:
defaults { udev_dir /dev user_friendly_names no }
To prevent any device from being recognized by the multipath daemon we are going to tell the system that all devices are blacklisted. This is easiest done by the following configuration in your /etc/multipath.conf:
defaults { udev_dir /dev user_friendly_names no } blacklist { wwid "*" } blacklist_exceptions { }Make sure nothing else is present in the file.
To start the multipathd use on Red Hat based systems:
service multipathd startand on Debian based systems:
service multipath-tools start
Now we check to see if the multipath daemon is alive:
echo "help" | multipathd -kthe "help" command does not exist, but it will tell you which commands are available anyway. The first line that the daemon returns also tells you the version of the multipath-tools you are running.
To stop the multipath daemon use the before mentioned service command and replace start by stop.
Your multipathd comes already with a lot of supported devices without you knowing about it. And that will change with using the following command:
echo "show config" | multipathd -kThis will provide you with a long list of already supported devices and also a complete overview of all the different sections that can occur in the multipath.conf configuration file. The available sections are:
As you might have seen in the output of the "show config" command, your multipathd knows about some systems. We are doing our tests with a Dell MD3200i system, so we will use that as an example. The configuration shows a configuration for the Dell MD32xxi that looks something like this:
device { vendor "DELL" product "MD32xx|MD32xxi" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac checker rdac features "2 pg_init_retries 50" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" prio rdac failback immediate rr_weight uniform no_path_retry 15 rr_min_io 1000 }
So apparantly the system can detect that this device is present, but how does it do that? Well, multipathd is started after OpenISCSI which means that the iscsid has already made sure the iSCSI devices are available to the system as block devices. If we do some investigation we find the following information in our system:
# cat /sys/block/sdb/device/vendor DELL # cat /sys/block/sdb/device/model MD32xxiThat is exactly what is matched with the vendor and product lines from our device definition. Except that our product line has an or (|) and matches thus for both the MD32xxi (iSCSI) and the MD32xx (locally attached) devices.
Since we have 4 paths to our device we have 4 devices:
# lsscsi | grep MD32xxi [9:0:0:0] disk DELL MD32xxi 0784 /dev/sdc [10:0:0:0] disk DELL MD32xxi 0784 /dev/sdb [11:0:0:0] disk DELL MD32xxi 0784 /dev/sdd [12:0:0:0] disk DELL MD32xxi 0784 /dev/sde
The trick for multipathd is to figure out which device is actually the same device, but using a different path. The easiest way to figure this out is by comparing the WWID, the world wide unique identifier. The getuid_callout line tells how we should get that information, where %n is replace by the device name. Luckily for us the different distributions use different commands to figure this out. So for Debian-based systems use:
getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"and for Red Hat-based systems use:
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"Running one of the commands for each iSCSI device shows us exactly what is happening:
# /sbin/scsi_id -g -u -s /block/sdb 36842b2b0006b95b20000047f4d533f9c # /sbin/scsi_id -g -u -s /block/sdc 36842b2b0006b95b20000047f4d533f9c # /sbin/scsi_id -g -u -s /block/sdd 36842b2b0006b95b20000047f4d533f9c # /sbin/scsi_id -g -u -s /block/sde 36842b2b0006b95b20000047f4d533f9cEvery device reports the same WWID, which means that each device is actually one and the same device but reached by a different path.
And the smart thing multipath does is tying this knowledge to a single device. Using the commandline tool multipath one can list how multipathd configured the system for us:
# multipath -l mpath0 (36842b2b0006b95b20000047f4d533f9c) dm-2 DELL,MD32xxi [size=16T][features=3 queue_if_no_path pg_init_retries 50][hwhandler=1 rdac][rw] \_ round-robin 0 [prio=0][active] \_ 10:0:0:0 sdb 8:16 [active][ready] \_ 11:0:0:0 sdd 8:48 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 9:0:0:0 sdc 8:32 [active][ghost] \_ 12:0:0:0 sde 8:64 [active][ghost]Multipath deals with two files: dm-2, our device mapper file, and mpath0, our user friendly name. The last name is maintained in a simple database file stored as /var/lib/multipath/bindings which looks like this:
# Multipath bindings, Version : 1.0 # NOTE: this file is automatically maintained by the multipath program. # You should not need to edit this file in normal circumstances. # # Format: # alias wwid # mpath0 36842b2b0006b95b20000047f4d533f9cThis explains how the system figured out how all the 4 devices are actually one and the same device.
In the defaults section you can set defaults for all settings used in the devices and the multipaths sections, plus a couple of generic items that can only be set in the defaults section. Configuration options that we would like to mention here are:
Example:
defaults { udev_dir /dev user_friendly_names yes }
There are 3 valid options to use in both the blacklist and blacklist_exceptions sections:
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
wwid 46832b2b3796b95b2aad0007885150e5f1
device { vendor "IBM" product "S/390.*" }
Example:
blacklist { wwid "*" } blacklist_exceptions { # Make an exception for our device(s) wwid "46832b2b3796b95b2aad0007885150e5f1" }
With multipath -l you can get an overview of which disks belong to which multipath device:
mpath3 (36a4badb00018e1c50000051c4c450a6b) dm-3 DELL,MD3000i [size=4.5T][features=0][hwhandler=1 rdac][rw] \_ round-robin 0 [prio=200][active] \_ 8:0:0:2 sdd 8:48 [active][ready] \_ 11:0:0:2 sde 65:0 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 9:0:0:2 sdf 8:80 [active][ghost] \_ 10:0:0:2 sdg 8:240 [active][ghost]
With multipathd -k one can directly talk to the multipath daemon. It opens an interactive console from which you can escape by pressing CTRL-D
On the console prompt type help to get an overview of the available commands. The command that might be the most useful is reconfigure, which makes multipathd reread the /etc/multipath.conf configuration file.
One can also talk to multipathd directly from the commandline to e.g. look at the available paths. On newer versions you could use the "show status" command (see below), but that is not availble in older versions, what always works is:
# echo "show paths" | multipathd -kA system with multipaths present should report something like this:
hcil dev dev_t pri dm_st chk_st next_check 10:0:0:0 sdb 8:16 100 [active][ready] X......... 2/20 11:0:0:0 sdd 8:48 100 [active][ready] XX........ 5/20 9:0:0:0 sdc 8:32 0 [active][ghost] XXX....... 7/20 12:0:0:0 sde 8:64 0 [active][ghost] XXXXXXXX.. 17/20If you get no output there are no paths. As mentioned before the newer versions (0.4.9) can present you with a status.
# echo "show status" | multipathd -kThe output should tell you if there is a multipath present:
multipathd> show status path checker states: paths: 0 busy: FalseAs you can see the system I ran this on has no device with multipaths.
To find out which devices are built-in to your multipath system (Red Hat), use
echo "show config" | multipathd -kDebian based system do not seem to have default built-in configurations. If someone knows more about this please contact me.
We discussed the devices section in the multipath.conf file before and promised to come back to it. Now is the time. It happens that new devices come to market, which are not yet supported by the installed version of multipath. To overcome this hurdle we can define our own device in the devices section. Before we can do this we need to collect some information from the our system:
A multipath map is defined as: start length multipath #features [feature1 ... featureN] #dsm [dsm1 ... dsmN] #pathgroups next_pathgroup pathgroupargs1 ... pathgroupargsN
Example, the output of the table command is wrapped for readability, of course it is presented as one long line:
[prompt]# dmsetup ls --tree -o device data-part1 (253:10) |_mpath3 (253:3) |- (8:240) |- (8:80) |- (65:0) |_ (8:48) [prompt]# dmsetup table /dev/mapper/mpath3 0 9663676416 multipath 0 1 rdac 2 1 round-robin 0 2 1 8:48 1000 65:0 1000 round-robin 0 2 1 8:80 1000 8:240 1000 [prompt]# ls -l /dev | grep -E '^b.+[[:space:]]8,.*[[:space:]]240[[:space:]]' brw-r----- 1 root disk 8, 240 Feb 22 18:39 sdp
devices { device { vendor "APPLE*" product "Xserve RAID " path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "3PARdata" product "VV" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "DEC" product "HSG80" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker hp_sw features "1 queue_if_no_path" hardware_handler "1 hp-sw" prio_callout "/sbin/mpath_prio_hp_sw /dev/%n" rr_weight uniform rr_min_io 1000 } device { vendor "COMPAQ" product "(MSA|HSV)1.0.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker hp_sw features "1 queue_if_no_path" hardware_handler "1 hp-sw" prio_callout "/sbin/mpath_prio_hp_sw /dev/%n" rr_weight uniform no_path_retry 12 rr_min_io 100 } device { vendor "(COMPAQ|HP)" product "HSV1[01]1|HSV2[01]0|HSV3[046]0|HSV4[05]0" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "(COMPAQ|HP)" product "MSA VOLUME" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "MSA2[02]12fc|MSA2012i" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "MSA2012sa|MSA2312sa|MSA2324sa" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -n -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "MSA2312fc|MSA2324fc|MSA2312i|MSA2324i|MSA2000s VOLUME" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "P2000 G3 FC" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "HSVX700" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "1 alua" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "HSVX740" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "HP" product "A6189A" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "HP" product "LOGICAL VOLUME.*" path_grouping_policy multibus getuid_callout "/sbin/cciss_id /dev/%n" path_selector "round-robin 0" path_checker cciss_tur features "0" hardware_handler "0" rr_weight uniform no_path_retry 12 rr_min_io 1000 } device { vendor "HP" product "P2000 G3*|P2000G3 FC/iSCSI" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 100 } device { vendor "DDN" product "SAN DataDirector" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "EMC" product "SYMMETRIX" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -ppre-spc3-83 -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" rr_weight uniform no_path_retry 6 rr_min_io 1000 } device { vendor "DGC" product ".*" product_blacklist LUNZ path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker emc_clariion features "1 queue_if_no_path" hardware_handler "1 emc" prio_callout "/sbin/mpath_prio_emc /dev/%n" failback immediate rr_weight uniform no_path_retry 60 rr_min_io 1000 } device { vendor "FSC" product "CentricStor" path_grouping_policy group_by_serial getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "(HITACHI|HP)" product "OPEN-.*" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" failback immediate rr_weight uniform no_path_retry 18 rr_min_io 1000 } device { vendor "HITACHI" product "DF.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_hds_modular /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "EMC" product "Invista" product_blacklist LUNZ path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" rr_weight uniform no_path_retry 5 rr_min_io 1000 } device { vendor "IBM" product "ProFibre 4000R" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "1722-600" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "1 queue_if_no_path" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 300 rr_min_io 1000 } device { vendor "IBM" product "1724" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "1 queue_if_no_path" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 300 rr_min_io 1000 } device { vendor "IBM" product "1726" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "1 queue_if_no_path" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 300 rr_min_io 1000 } device { vendor "IBM" product "1742" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "1814" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "IBM" product "1745|1746" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "2 pg_init_retries 50" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 15 rr_min_io 1000 } device { vendor "IBM" product "1815" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "IBM" product "1818" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "IBM" product "3526" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "3542" path_grouping_policy group_by_serial getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "2105(800|F20)" path_grouping_policy group_by_serial getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "1 queue_if_no_path" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "1750500" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "1 queue_if_no_path" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "2107900" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "1 queue_if_no_path" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "2145" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "1 queue_if_no_path" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "S/390 DASD ECKD" product_blacklist S/390.* path_grouping_policy multibus getuid_callout "/sbin/dasd_id /dev/%n" path_selector "round-robin 0" path_checker directio features "1 queue_if_no_path" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "S/390 DASD FBA" product_blacklist S/390.* path_grouping_policy multibus getuid_callout "/sbin/dasd_id /dev/%n" path_selector "round-robin 0" path_checker directio features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "NETAPP" product "LUN.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker directio features "1 queue_if_no_path" hardware_handler "0" prio_callout "/sbin/mpath_prio_ontap /dev/%n" failback immediate rr_weight uniform rr_min_io 128 } device { vendor "IBM" product "Nseries.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "1 queue_if_no_path" hardware_handler "0" prio_callout "/sbin/mpath_prio_ontap /dev/%n" failback immediate rr_weight uniform rr_min_io 128 } device { vendor "Pillar" product "Axiom [35]00" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" prio_callout "/sbin/mpath_prio_alua /dev/%n" rr_weight uniform rr_min_io 1000 } device { vendor "IBM" product "3303 NVDISK" path_grouping_policy failover getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" failback immediate rr_weight uniform no_path_retry 60 rr_min_io 1000 } device { vendor "AIX" product "VDASD" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker directio features "0" hardware_handler "0" failback immediate rr_weight uniform no_path_retry 60 rr_min_io 1000 } device { vendor "AIX" product "NVDISK" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "1 alua" prio_callout "/sbin/mpath_prio_alua /dev/%n" failback immediate rr_weight uniform no_path_retry 60 rr_min_io 1000 } device { vendor "SGI" product "TP9[13]00" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "SGI" product "TP9[45]00" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "SGI" product "IS.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "STK" product "OPENstorage D280" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform rr_min_io 1000 } device { vendor "STK" product "FLEXLINE 380" product_blacklist Universal Xport path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "SUN" product "(StorEdge 3510|T4)" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "PIVOT3" product "RAIGE VOLUME" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -p 0x80 -g -u -d /dev/%n" path_selector "round-robin 0" path_checker tur features "1 queue_if_no_path" hardware_handler "0" rr_weight uniform rr_min_io 100 } device { vendor "SUN" product "CSM200_R" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "SUN" product "LCSM100_F" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "0" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "(LSI|ENGENIO)" product "INF.*" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "2 pg_init_retries 50" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 15 rr_min_io 1000 } device { vendor "DELL" product "MD3000|MD3000i" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "2 pg_init_retries 50" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 15 rr_min_io 1000 } device { vendor "DELL" product "MD32xx|MD32xxi" path_grouping_policy group_by_prio getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker rdac features "2 pg_init_retries 50" hardware_handler "1 rdac" prio_callout "/sbin/mpath_prio_rdac /dev/%n" failback immediate rr_weight uniform no_path_retry 15 rr_min_io 1000 } device { vendor "COMPELNT" product "Compellent Vol" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -g -u -s /block/%n" path_selector "round-robin 0" path_checker tur features "0" hardware_handler "0" failback immediate rr_weight uniform no_path_retry queue rr_min_io 1000 } device { vendor "GNBD" product "GNBD" path_grouping_policy multibus getuid_callout "/sbin/gnbd_import -q -U /block/%n" path_selector "round-robin 0" path_checker directio features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } device { vendor "EUROLOGC" product "FC2502" path_grouping_policy multibus getuid_callout "/sbin/scsi_id -p 0x80 -g -u -s /block/%n" path_selector "round-robin 0" path_checker readsector0 features "0" hardware_handler "0" rr_weight uniform rr_min_io 1000 } }