SUSE Conversations


Solving Network Woes After Cloning A SLES 10 Virtual Machine From Template On VMware Virtual Infrastructure / ESX 3.5 Server



By: kryptikos

March 24, 2008 10:05 am

Reads:823

Comments:4

Rating:0

THE SKINNY

So here I am reading all the latest tech magazines and boards watching how virtualization has quickly become the latest fad in the IT sector. Although IBM has been successfully implementing virtual machines since the 1960s, it has only recently become popular and advantageous to utilize virtualization on Intel/AMD based servers with the advent of VMware’s ESX server.

The company I currently work for decided to jump in the fad-mobile and plug VMware’s Virtual Infrastructure 3 virtualization suite into our infrastructure. With the deployment of VMware came many new concepts and techniques…not to mention learning curves…that I as an engineer have had to learn. I’ve noticed there are many tricks, configuration methods and nuances to running this suite.

Standing a Linux virtual server up is a no-brainer. The virtual machine does not know it is virtual and thus your configuration is straightforward just as if you had racked a 1U pizza box and installed the OS via CD-ROM or the network. In VMware when you want to deploy multiple servers you can clone virtual machines from a template instead of building one box individually at a time. Where things got interesting for me was when I built a SLES 10 SP1 template that had been hardened for security and customized for our environment.

I found that when I would clone a new Linux virtual machine it would lose network connectivity. I noticed that two NIC cards had appeared and neither one would establish link. I scoured the Internet and found a few helpful posts, but nothing that specifically addressed my problem. So to help anyone out there who has felt the same pain and had the same woes, I drafted up a mini-HOWTO for correcting the NIC issue when cloning a Linux server on VMware ESX server. So let’s get to it…

THE ENVIRONMENT

The environment I was working in included VMware Virtual Infrastructure 3 (VI), ESX Server version 3.5 and Intel Xeon based hardware. The specific Linux distribution being cloned was SuSE Linux Enterprise Server SP10.

THE HOWTO

Confirm MAC Address

The first step in setting up your Linux clone is to confirm the network settings for the new virtual machine (vm). VMware creates unique MAC addresses for each vm and requires that you assign the NIC card to a network connection. These MAC addresses will start with a 00:50. Confirm both that you have a unique MAC address for your clone as well as assigning it the appropriate network connection name. There are two ways you can do this.

Method 1) From the shell on the ESX server: look inside the vmx file associated with your new vm. Not sure where your vmx is located? From the prompt issue:

#find / -type f -name *.vmx

Once it has returned the results, ‘cd’ into that directory and either cat or less the vmx file. You will see a section that addresses ethernet0. Confirm that ethernet0.generatedAddress MAC value is not the same from your template and that ethernet0.networkName has the correct value (see figure 1 for an example screenshot).

Figure 1 – VMX ethernet0 configuration.

Click to view.

Method 2) From the VI client select your virtual machine and edit the settings. Double-check the section MAC Address and the label in Network Connection (see the red dots in figure 2).

Figure 2 – Confirm in VI Client the MAC address and Network Connection.

Click to view.

The Nitty Gritty

Now that we have confirmed the MAC address and the network label we can get to the actual clean up process to re-establish our clone’s network connectivity. Follow these steps in order and you will be up in no time.

Figure 3 shows that the initial assignment of your network going to eth1. This is due to the fact the template is created with eth0 still containing the ip address assigned during the original Linux server build out. You will notice that PCnet32 is used by VMware as its NIC driver.

Figure 3 – Incorrect assignment of ethernet NIC.

Click to view.

Step 1: Delete the two persistent rules in ‘/etc/udev/rules.d/30-net_persistent_names.rules’.

There are two rules that have been created in this file. You will notice that they are aptly named eth0 and eth1 (see figure 4). If you are like me, you like to understand what you are doing instead of just mindlessly bumbling along and following some HOWTO. Linux used to have a very frustrating and archaic method to handle kernel interaction with devices. It was administratively intensive. After several generations we now have udev. This file-system is responsible for detecting, interacting with and creating nodes in the /dev directory. Each time the server is booted udev takes information passed by sysfs and filters it through rules set up by the administrator in /etc/udev/rules.d. This is how devices that either exist at boot up or are hot pluggable, such as a USB drive, are loaded and named. By deleting these rules and manipulating the NIC settings a bit later is how we will reset our server to have the correct NIC assignment.

Figure 4 – Delete ethernet NIC rules.

Click to view.

Fire up your favorite editor and delete the two entries (entire line) for eth0 and eth1.

Step 2: Start Yast or Yast2 and enter the Network Card configuration module.

You are presented with the current set up. A quick scan shows two NIC cards and if you look more closely you’ll notice that the primary eth0 card is the template server’s MAC and ip address (figure 5).

Figure 5 – Two NICs are installed at the moment.

Click to view.

Delete the first NIC that has the configuration from the template server. Edit the second AMD PCnet card adjusting the ip address, hostname, and default gateway (figures 6 through 9). There are two elements you want to be and check during this process. In the address set up screen confirm the Configuration Name id lists your assigned MAC address as discovered earlier in your vmx file (indicated by the red dot in figure 6). The other location you check the id is under Advanced -> Hardware. Confirm the Configuration Name id listed is the same as well.

Figure 6 – Configure ip address on remaining NIC. Confirm id name.

Click to view.

Figure 7 – Configure hostname, name server etc.

Click to view.

Figure 8 – Configure default gateway.

Click to view.

Figure 9 – Confirm in the hardware that the configuration name is the same MAC.

Click to view.

When you have finished the configuration of your new NIC and click on the “Next” button Yast will write the values out and return you to the main network card configuration overview menu. You will notice that the only card listed is your newly minted eth0. Click “Finish” and let Yast complete the write out.

Step 3: Reboot the virtual machine.

Now some folks may say that you can restart network services using rcnetwork or /etc/init.d/network restart, however the reason I suggest rebooting is two fold. First I’d like to see the kernel load eth0 up correctly during the start up (figure 10) and record it appropriately in dmesg.

Figure 10 – Post configuration reboot looks good. Eth0 has correct assignment now.

Click to view.

The second reason is I want to confirm that udev writes the correct values back into its network persistent rule set cleanly upon booting. After the reboot I can perform some quick status checks to make sure my cloned virtual machine is up and has established network connectivity. Note the red dot in figure 10. If you do not see the hardware assignment on the first line when you issue ifstatus eth0 you have an issue.

Figure 11 – Post configuration connectivity tests. Positive ping and status responses.

Click to view.

If you less out /etc/udev/rules.d/30-net_persistent_names.rules you will see only one entry labeled eth0 with the correct MAC address.

Wrap Up

That’s it! Your Linux virtual machine cloned from your Linux base template is good to go. If you have any questions please feel free to contact me and I will be happy to help in any way I can. Linux is clearly the best operating system out there. Adding in the functionality of virtualization and VMware yields greater and if I might say a sweeter functionality. Combining the power of Linux and ESX (hey, it’s a modified Linux environment anyways) gives you as an administrator true high availability and disaster recovery capabilities never seen before. That’s one powerful penguin.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Tags: , ,
Categories: SUSE Linux Enterprise Server, Technical Solutions, Virtualization

Disclaimer: As with everything else at SUSE Conversations, this content is definitely not supported by SUSE (so don't even think of calling Support if you try something and it blows up).  It was contributed by a community member and is published "as is." It seems to have worked for at least one person, and might work for you. But please be sure to test, test, test before you do anything drastic with it.

4 Comments

  1. By:paulnorthrup

    Hi,

    What about the other settings that are established on the initial setup?
    Our environment: VMWare ESX 3.5 on AMD multicore blade server.

    - ssh keys
    - network (covered in this article)
    - File system

    Is there a “Sysprep” tool for SuSE?

  2. By:kryptikos

    Hi, To answer your question there is not a “sysprep” tool. Linux doesn’t use SID or registration etc etc that would require or even need that.

    There is not a need for adjustments to the file system when you clone as you have already put on the software and necessary base configurations you wanted when you built your template server. Hostname and network settings will be changed per server regardless.

    As far as items you may wish to change…yes, ssh keys will be the same as the template. So if you have a security requirements you can change those:
    # rm /etc/ssh/ssh_host_*sa.key
    # ssh-keygen -l -f ssh_host_rsa.key (wash, rinse, repeat for a dsa key if needed)
    Delete your entries in ~/.ssh/known_hosts if you desire to rebuild it as you ssh to various devices.

    Have not tested to confirm, but if you delete the ssh_keys from your template just before you shutdown they should regenerate when you build the new vm and it boots for the first time. But confirm that if you do try it.

    Also, if you are running SAMBA you may have to be concerned with an SID there.

    Hope that helps.
    Cheers

  3. By:jcschad

    Hi
    but my second server can’ t be registered to be update.
    I think they have the same system id and i can’t acive it in “customer center”

    How can i resolve this problem ?

  4. By:twohlford

    Just migrated my SLES 10.3 server to ESXi 4.1 using VMware vCenter Converter Standalone Client.

    Same solution!

    Just booted to the point where I could edit 30-net_persistent_names.rules, then rebooted the server. Since XWindows was confused, I had to use YAST to configure both the video as well as the NIC after the reboot.

    Rebooted again, and all is well!

    Well, almost — had to run DSRepair to get the replica working (could ping from master replica, but no replication). Repair server address, and within a couple of minutes the server was listed as “online” again in replica operations.

Comment

RSS