Configure multiple iSCSI ifaces and use multipathing | SUSE Communities

Configure multiple iSCSI ifaces and use multipathing

Share
Share

Objective: To provide a quick overview of the steps needed to bind different NICS to iSCSI ifaces and use those ifaces to setup multipathing.

SCENARIO

SLES 11 SP3 server with 5 NICS (1 on public network, 3 on private network dedicated to iscsi).

eth0 – 151.155.55.22

eth1 – 10.1.10.1

eth2 – 10.1.10.2

eth3 – 10.1.10.3

iscsi-target – 10.1.10.100 (iqn.my-target.com)

BASIC PREP-WORK

  1. Install open-iscsi

    zypper install open-iscsi

  2. Run “yast iscsi-client”, do nothing but let yast configure the initiator name.

    Yast has specific parameters regarding the syntax of the initiator name. Rather than manually configuring the initiator name and run into problems with Yast later, let Yast set the initiator name up front and save time later on.

  3. Check the network setup and verify that all IP addresses and subnet masks are correct.

    yast lan

  4. Check network connectivity by pinging from the different interfaces.

    syntax: ping -I ethX IP.of.target

    example: ping -I eth1 10.1.1.100

  5. Set /proc/sys/net/ipv4/conf/rp_filter to 0

    The rp_filter setting impacts packet filtering/routing and can cause problems with iSCSI connections on a multi-homed system.

SETUP ISCSI IFACE FILES

  1. Configure iface devices

    syntax: iscsiadm -m iface -I iface# –op=new (dash dash op=new)

    example: iscsiadm -m iface -I iface1 –op=new (dash dash op=new)

    Run this command to create an iface file for each desired interface. As of today, 08/22/2014, Yast cannot create iface files thus this step must be done manually.

  2. Modify iface file to uniquely identify one NIC. There are several different parameters that can be used (iface.hwaddress, iface.net_ifacename, iface.ipaddress, etc) see the README for a more complete list of parameters.

    syntax: iscsiadm -m iface -I iface# –op=update -n iface.hwaddress -v MAC:of:NIC (dash dash op=update)

    example: iscsiadm -m iface -I iface1 –op=update -n iface.hwaddress -v 00:0c:29:d9:6f:75 (dash dash op=update)

    A word of caution: use the minimum number of parameters to correlate the iface and the NIC. Adding more parameters increases the likely-hood of typos or other mistakes that can cause iscsi not to work as expected.

  3. Check the contents of the iface files to ensure accuracy. Either cat the file or use iscsiadm to view the files

    Option 1

    syntax: cat /etc/iscsi/ifaces/iface#

    example: cat /etc/iscsi/ifaces/iface1

    Option 2

    syntax: iscsiadm -m iface -I iface#

    example: iscsiadm -m iface -I iface1

  4. Set the initiator name.

    If not done earlier set the initiator name. As noted above Yast has some specific syntax rule for the initiator names. Run “yast iscsi-client” and let Yast set the initiator name and then exit before doing any discovery.

DISCOVER TARGETS

  1. The easiest method to have iscsid scan the target device and then use the /etc/iscsi/ifaces files create records for each target and iface combination is:

    syntax: iscsiadm -m discoverydb -t st -p IP.of.target.nic:3260 –discover (dash dash discover)

    example: iscsiadm -m discoverydb -t st -p 10.1.10.100:3260 –discover (dash dash discover)

    It is also possible to just discover a target on one iface.

    syntax: iscsiadm -m discovery -t st -p IP.of.target.nic:3260 –interface=iface# –discover (dash dash discover)

    example: iscsiadm -m discovery -t st -p 10.1.10.100:3260 –interface=iface1 –discover (dash dash discover)

ISCSI LOGIN

Once iscsid has built the send_targets database a login must be initiated to access the target device. Yast or the cli can be used to manage the iscsi connections and login to each node.

Using YaST

With the iface files created in /etc/iscsi/ifaces and the target discovered YaST will read the send_targets database and display each connection between the iface and the target and will list them in the Discovered Targets tab. From there log-in options are available via the log-in button. If authentication is needed YaST may be an easier option over the cli.

Using CLI

With the targets discovered for the desired ifaces the initiator now needs to log into the target.

syntax: iscsiadm -m node -l

example: iscsiadm -m node -l

This will log the initiator into all of the nodes that are in the database. Please note this example assumes no authentication is needed. If authentication is needed use YaST or consult the documentation. More granular options, such as logging into only one node, can be found in the documentation.

NODE SETTINGS

There are many settings relating to each iscsi node. The startup method and authentication methods to name a few. Several settings are available in YaST or manually manipulating the file is an option; however, misconfiguration of the node settings may cause problems with log-in and operation.

Once everything is configured as desired verify that chkconfig open-iscsi is set to on.

MPIO

  1. Start multipathing

    rcmultipathd start

  2. Check mpio paths

    multipath -ll

    If only one path is listed under multipath -ll double check the value of /proc/sys/net/ipv4/conf/all/rp_filter.

For help with multipath configuration please see http://support.novell.com/techcenter/sdb/en/2005/04/sles_multipathing.html

 

The README for iscsi can be found at /usr/share/doc/packages/open-iscsi/README

Share
(Visited 1 times, 1 visits today)

Comments

  • Avatar photo deadpoint says:

    You’ll need to make a few additional modifications or you’re going to hit some issue

    1. You’ll want to set the MTU=9000 to enable jumbo frame for your iscsi NIC’s

    2. You need to disable the FW_KERNEL_SECURITY firewall setting or it will override rp_filter. It’s easier to reboot after making this change since it modifies lots of kernel setting.

    sysconf_addword /etc/sysconfig/SuSEfirewall2 FW_KERNEL_SECURITY no

    3. Preventing ARP Flux. Without this only a single interface will be able to communicate with the SAN due to arp flux. This needs to be done in addition to setting net.ipv4.conf.all.rp_filter=0.

    net.ipv4.conf.eth1.arp_ignore=1
    net.ipv4.conf.eth1.arp_announce=2
    net.ipv4.conf.eth2.arp_ignore=1
    net.ipv4.conf.eth2.arp_announce=2
    net.ipv4.conf.eth3.arp_ignore=1
    net.ipv4.conf.eth3.arp_announce=2

  • Avatar photo patelsb says:

    I have two iscsi interfaces and my system was only using one interface. iscsi now uses both interfaces and using multipath after i followed deadpoint’s recommendations under step 3.

  • Avatar photo todorojo says:

    AUTO LOGIN AFTER REBOOT?

    Thanks to this article and the comment by deadpoint about ARP flux I was able to get this to work on my system.

    HOWEVER, after the client reboots the iscsi sessions are gone. I then have to manually login again to the iscsi volume(s) after the system comes back up.

    Is there a way to have open-iscsi auto login during startup?

  • Leave a Reply

    Your email address will not be published. Required fields are marked *

    Avatar photo
    41,611 views