SUSE Conversations


PXE Booting NFS or iSCSI Roots for a Diskless Environment



By: utlemming

April 1, 2009 7:14 am

Reads:3609

Comments:3

Rating:0

Abstract

SLES 10 and SLES 11 both support interesting features — that is the idea of diskless booting. There are a couple of really good guides on how to do it, but very few take you step by step through the process. By following this guide, you should be able to configure a PXE Boot Server that will enable booting of iSCSI and NFS root servers. This guide is also written for the poor-man, i.e. those of us that can’t afford those nice, new fangled QLogic cards with iSCSI initiators and those who have VMware environments.

Supportability

The individual components in this guide are supported by Novell. However, the implementation of these several technologies together span several disciplines, and as such are hard to support. For example, DHCP in and of itself can be rather complex, much less booting via PXE and the concerns that diskless booting have. As such, you are strongly advised to consider the implications and to thoroughly test any configuration before you decide to employ it.

iSCSI booting is only supported by Novell when using an iSCSI firmware (i.e. QLogic QLA3xxx, QLA4xxx and similar cards) that is supported by Novell. While this guide helps you get around that restrictions, please understand that Novell support will be unable to assist if there is a problem booting from the iSCSI disk. This guide shows how to get around the firmware restriction, so you are essentially on your own if and when you have boot problems. If you are looking for a firmware for you card, you may want to look at the Etherboot project.

The reason for writing this guide is because of community interest. For some, this guide might help solve idle curiosity. For others who think that this solution might work in production environment, PLEASE TEST AND CONSIDER the design. There are several different ways to accomplish the same thing and you are advised to consider all the possibilities. Linux is Linux and as such, there are some very good guides across the internet that apply to SuSE and other Linux variants.

Design Flaw

One serious design flaw to using PXE for booting a server environment is that you now have implemented a single source of failure. If your PXE server goes down, becomes unavailable, then you’re entire boot environment could be affected. For networks requiring high availability you may consider copying all the files and configuration to another machine. In the event of a down-server situation you could then put the IP address on the secondary server. If you’re feeling adventurous, you could implement OCFS2 or DRBD inside of Linux HA to provide full-fault tolerance.

Components

In order to get this working, you need:

  • A tftp server
  • A DHCP server
  • An iSCSI target or NFS server

The list looks pretty straight forward, however, each step can be rather difficult to implement if you do it wrong

tftp server

The tftp server is needed in order to get the boot files. The tftp server will serve out the kernel and initrd that is built at install time. With out a tftp server, you’ll have to come up with another way to boot the system. Fortunately, SLE comes with all the components that are needed.

To get the tftp installed, type:

yast -i tftp yast2-tftp-server

Then run “yast2 tftp-server”. Set the directory you want to use. I suggest using /srv/tftp.

 

DHCP service

For this guide, we will be using DHCP in order to serve out IP address and point the booting servers to the TFTP server.

While it is possible to do some fancy diskless booting without a physical disk or DHCP server, the configuration usually entails using a Etherboot ROM and overwriting the ROM image on your NIC. If you are interested in voiding your warranty on your NIC cards, you can see the Etherboot project here LINK. Alternatively you can use EtherBoot to boot off of a USB drive or CD-ROM. In fact you can create some very elegant boot situations using EtherBoot (i.e. the identity of a server is held on a USB stick. If the server needs to have maintenance done, you can simply move the USB stick to another server).

To install the DHCP service, type:

yast -i dhcp-server yast2-dhcpd-server

After it is installed, run “yast2 dhcp-server”

Select the card that you want to use

Populate the options that you or want to use. You don’t have to populate all of them, and you can even populate none of them (i.e. a close network).

Put in the pool range. The pool range can be empty (for example on a network where you only have statically assigned IP addresses)

Click on “expert options” and then “host management”. Populate each host that you want to use there.

You should now have a working DHCP server

Modify DHCP for PXE Linux

PXELinux is part of the SYSLinux package. Many people are tempted to use Grub NetBoot. However, the number and type of cards supported by Grub is very limited. Some cards like VMware PCNet32 cards on 64-bit hosts and Broadcom BNX2 cards are not supported. SYSLinux is able to use a much wider range of cards and it works on just about every host I have tried.

To get started, you need to create a directory structure in /srv/tftp to support PXELinux.

  1. Make sure “syslinux” is installed, it should be there by default.
  2. Create directory /srv/tftp/pxelinux.cfg
  3. Create directory /srv/tftp/images
  4. Copy /usr/share/packages/syslinux/pxelinux.0 to /srv/tftp

PXELinux relies on the pxelinux.cfg directory to be in the root of the tftp directory.

In /etc/dhcpd.conf, you should have a file similar to this:

	option domain-name "example.com";
	option domain-name-servers 172.16.100.1, 172.16.100.2;
	option routers 172.16.100.1;
	default-lease-time 14400;
	ddns-update-style none;

	subnet 172.16.100.0 netmask 255.255.255.0 {
	  range dynamic-bootp 172.16.100.10 172.16.100.250;
	  default-lease-time 14400;
	  max-lease-time 172800;

		group {
			host sles10iscsi {
				hardware ethernet 00:0C:29:51:2D:DB;
				fixed-address 172.16.100.10;			
			}

			host sles10nfs {
				hardware ethernet 00:0C:29:51:2D:DC;
				fixed-address 172.16.100.11;			
			}
		}
	}

Unfortunately, you will need to modify it by hand and add in the PXE options. Below is an example of one that supports the PXE options:

	option domain-name "example.com";
	option domain-name-servers 172.16.100.1, 172.16.100.2;
	option routers 172.16.100.1;
	default-lease-time 14400;
	ddns-update-style none;

	#####ADDED
	  option pxelinux.magic      code 208 = string;
	  option pxelinux.configfile code 209 = text;
	  option pxelinux.pathprefix code 210 = text;
	  option pxelinux.reboottime code 211 = unsigned integer 32;
	#####END ADD

	subnet 172.16.100.0 netmask 255.255.255.0 {
	  range dynamic-bootp 172.16.100.10 172.16.100.250;
	  default-lease-time 14400;
	  max-lease-time 172800;

        ######ADDED
	  site-option-space             "pxelinux";
  	  option pxelinux.magic         f1:00:74:7e;
	  option pxelinux.reboottime    30;
	  filename "pxelinux.0";
	######END ADD

		group {
			host sles10iscsi {
				hardware ethernet 00:0C:29:51:2D:DB;
				fixed-address 172.16.100.10;			
			}

			host sles10nfs {
				hardware ethernet 00:0C:29:51:2D:DC;
				fixed-address 172.16.100.11;			
			}
		}
	}

You’ll now need to restart “dhcpd” in order to use the new option (i.e. “rcdhcpd restart”). If there are any problems, then it will let you know.

NFS shares

Updates to the kernel can cause some serious problems. By default, all updates are install to /boot. However, on both a NFS and an iSCSI server, the problem of kernel updates can be tricky. The easiest solution is to put the boot files on a NFS share that is accessible via tftp.

For example, during the install, go into the expert partitioner. There you can define a mount point for a NFS share. YaST will detect that /boot is on a NFS share and will not install Grub.

IMPORTANT: this guide assumes that /srv/tftp/boot is shared out via NFS. The NFS share for /boot for each machine should be accessible via tftp. If the NFS share is not read-writable by the machine, then the PXE configuration will most likely fail.

Define the PXELinux configuration

Before starting the installation, you need to make sure that there is a PXE configuration for the machine. If you don’t have a configuration for the machine that you install, the reboot will fail.

In the DHCP configuration, each host was given an IP address. PXELinux allows you to define per-host configuration file. The filename is based on the IP Address of the host. To calculate the filename you run “gethostip”. The output is a hex representation of the IP address.

For example, to get hex for 172.16.100.10:

	#gethostip 172.16.100.10
	172.16.100.10 172.16.100.10 AC10640A

The “AC10640A” is the name of the file that you would use.

Once you get the filename, you need to create a file in /srv/tftp/pxelinux.cfg with that file name.

You will populate the contents with something similar to:

iSCSI

	default sles10

	label sles10
	  kernel boot/sles10iscsi/vmlinuz
	  append initrd=boot/sles10iscsi/initrd ip=172.16.100.11:172.16.100.1:172.16.100.1:255.255.255.0:slesiscsi:eth0:none TargetAddress=172.16.100.1 root=/dev/sda2

NFS

	default sles10

	label sles10
	  kernel boot/sles10iscsi/vmlinuz	
	  append initrd=boot/sles10iscsi/initrd ip=172.16.100.11:172.16.100.1:172.16.100.1:255.255.255.0:slesiscsi:eth0:none

The “ip=” directive is important. The use of “ip” is a robust setting; use of hostip and network can be overridden. However, the “ip” kernel parameter won’t be. In fact if you use another setting it might be overridden when the HAL daemon starts. The syntax is, ADDRESS:GATEWAY:NAMESERVER:NETMASK:HOSTNAME:DEVICE:AUTOCONF. The last parameter AUTOCONF should be none for most setups. The parameter “ip” used to be “nfsaddrs”, but since it works independently of the NFS stack, it was renamed.

The “kernel” and “initrd=” paths are relative to the tftp server and also should be the NFS share for the /boot directory. When you start the installation, these directories should be empty. After the installation, then it should look very similar to a native disk installation.

Install the OS

Boot the installation system based on your preference.

During the installation, you want to go into the expert partitioner and define the NFS root. You will need to boot with USB stick to confuse the installer into allowing you to install.

Install the OS iSCSI

In order to install to an iSCSI target, you need to boot with the kernel parameter “withiscsi=1″. The parameter will instruct YaST to start up the iSCSI initiator. The iBFT tab will not be enabled unless you have an iSCSI card (which if you do, then you don’t need to worry about this guide). After connecting to the initiator, you can proceed to install normally. However, make sure that /boot is on the NFS share.

Test it out

At this point, you are safe to test things out. If everything has been successful, then you should be able to boot up your diskless version of SLE.

Use cases

There are a variety of use cases for diskless servers and workstations. For example, NFS is commonly used for booting up server farms were the physical server identity isn’t that important. Some libraries use NFS as root to boot the end terminals. The use case is especially useful when the function of the computer is more important than the identity and where the function can be run on many computers.

Tin Foil Hat Warnings: Planning for a bad kernel update

After you get the system installed, MAKE A BACKUP COPY OF THE INITRD and THE KERNEL. In the event that something goes wrong, like a kernel update, you are absolutely going to need the INITRD the KERNEL. Locate the backup initrd and the kernel on the TFTP server. I recommend creating a menu to chose the kernel. For example:

	default sles10

	label sles10
	  kernel boot/sles10iscsi/vmlinuz	
	  append initrd=boot/sles10iscsi/initrd ip=172.16.100.11:172.16.100.1:172.16.100.1:255.255.255.0:slesiscsi:eth0:none TargetAddress=172.16.100.1 root=/dev/sda2

Could be modified to:

	menu title iscsi host PXE Menu
	menu tabmsgrow 22
	menu cmdlinerow 22
	menu endrow 24

	prompt 0
	implicit 1
	options 1
	timeout 100

	default sles10

	label sles10
	  kernel boot/sles10iscsi/vmlinuz	
	  append initrd=boot/sles10iscsi/initrd ip=172.16.100.11:172.16.100.1:172.16.100.1:255.255.255.0:slesiscsi:eth0:none TargetAddress=172.16.100.1 root=/dev/sda2     

	label sles10-install
	  kernel boot/sles10iscsi/vmlinuz-install	
	  append initrd=boot/sles10iscsi/initrd-install ip=172.16.100.11:172.16.100.1:172.16.100.1:255.255.255.0:slesiscsi:eth0:none TargetAddress=172.16.100.1 root=/dev/sda2

The reason why I strongly suggest that you do this is that in the event something goes wrong you have a backup. By saving the install kernel and initrd and creating a way to boot it, you can prevent a whole hosts of problems. If a kernel update goes wrong, then you have something to fall back on and fix the problem.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags: , , ,
Categories: SUSE Linux Enterprise Desktop, SUSE Linux Enterprise Server, Technical Solutions

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

3 Comments

  1. By:cyberorg

    Here is a related guide:

    http://en.opensuse.org/LTSP/Fatclient

    Using KIWI’s AoE root feature.

  2. By:jhonnypolak

    Hi mate,

    Great guide, however when I try to install to an NFS mount i am able to define an NFS mount point and setup partitions but then I get an error “Failed to initialize directory” and the installation aborts. I can see some files have been copied to the NFS mount as it’s being hosted from another SLES server.

    Any ideas ?

  3. By:jhonnypolak

    Hi Mate,

    I’m having a problem with your guide. When I try to install SLES10 on my NFS share presented by another SLES10 VM i get an error: “Initializing the target directory failed”. When I press cancel or continue the installation fails. I can see in the background that the process fails when the installer is trying to:

    “Adding entry for mount point / to /etc/fstab”

    Do you have any tips on getting around this ?

    I can see that there are some files that have been mounted on my NFS share folder so there’s no problems for the installer writing to the NFS share.

Comment

RSS