XEN: Common problems with network bridges
This document (7001989) is provided subject to the disclaimer at the end of this document.
Novell SUSE Linux Enterprise Server 10 Service Pack 1
- intermittentant connectivity on bonded bridges
- routing issues within Dom0
- DomU's lack network access because of switch security
- DomU's have no network connectivity after the network subsystem has been restarted
- Bridge performance
- DomU's do not see all network traffic
- DomU's connect to the wrong bridge when started
- Dom0 does not have a network bridge.
- Changes using YaST break Xen Networking
- Interfaces on multiple networks
intermittent connectivity on bonded bridges
- DomU's may not have any connectivity
- Dom0 may not have any connectivity to the network
- DomU's and Dom0 are connected
solution: Change to "active-backup" or look change the arp settings on the switch. If the swtich supports "802.11q" or "Dynamic Link Aggregation," mode 4 would also work.
routing: dom0 has network connectivity, but domU's do not
test: To test, type "ip route show", for example:
172.16.142.0/24 dev br1 proto kernel scope link src 172.16.142.1
127.0.0.0/8 dev lo scope link
default via 192.168.0.1 dev br0
solution: Alter your scripts to set the default route. If you are using a static bridge configuration, in /etc/sysconfig/network/routes, define the bridge. Setting this value will, however, break networking when booting into a non-Xen kernel. For example:
default 10.0.0.1 - br0.
solution: Disable port security on the switch or the individual port. Alternatively, change the port to a "bridge" or a "switch" port. Consult with your switch vendor documentation or support in order to establish whether or not this a match.
network restart: domU's have no connectivity after a network restart
solution: See TID 700986 Xen: reattaching network devices after the network bridge is restarted
bridge does not forward all traffic through the bridge
solution: You will need to set up forwarding rules in "ip tables" to forward all traffic through the bridge. Unfortunately, there are too many variables for this document to detail how to do it. If you need to implement this solution, you may need to contact a Novell Linux partner.
solution: An alternative solution is to do PCI pass through, which is well documented in the Xen documentation. The caveaut, however, is that it is only currently available for para-virtual Domians at this time. Newer chips and motherboards which support the Intel-VTd technology will allow you to use PCI pass through with fully virtual domains.
utilization: slow bridges
solution: reduce load for total utilization (Dom0 + DomU's) to be less than 85%. Alternatively, you could do CPU pinning to reduce the load and pin at least two CPU's to Dom0 exclusively.
network topography: slow bridges or slow lookup times
solution: Generally speaking this is not an issue. If, however, there is time sensative communication, use a smaller subnet than /22.
domu's connect to the wrong bridge
solution: Xen DomU's can be configured to attach to a specific bridge inferface. See TID 7000175, Changing network bridges for DomU's
network bridges are not present
solution: Disable network-manager. See TID 35882110 Xen boot problems when the network manager is enabled
dom0 services such as DHCP or LinuxHA do not work after xend starts
solution: There is no "easy" solution to this problem. In general, Novell advises against using Dom0 to host network services. Some notable exceptions are services that are required to run the DomU's or light weight services that are used to support the DomU's. Some services like DHCP and LinuxHA use simple device names like "eth0" to start their services. If you use a different device name, like "br0" or "xenbr0" you need to configure the services to use the bridge name. Further, for those using the default Xen configuration, booting into the Xen kernel and the none-Xen kernel will create two different networking environments. If this is a concern, then it is recommended that you create static bridges that are present in both the Xen and non-Xen environment.
modifications to bonding or network configuration in non-Xen kernel results in broken networking in Xen kernel
solution: Rename an "ifcfg-bus-id" files to be "ifcfg-eth-id-" (followed by the MAC address of the card) or "ifcfg-ethX" (replace X with the pseudo number, for example ifcfg-eth0).
multiple networks on the same interface: domU's have no connectivity
cause: This issue is commonly seen in networks where IP space is limited. In order to add more IP space, system administrators will simply add another IP network on top of an existing network. While this technically possible for IP networks, experience has shown that this does not work on bridges where the bridge interface has an IP address assigned.
solution: Make sure that IP addresses assigned to the interfaces on Dom0 and DomU are on the same subnet.
debug mode for network scripts
In /etc/sysconfig/network/config, set "DEBUG=YES". This will display debug information when the scripts are run.
The following list shows useful bridge commands
- brctl show: shows all bridges on the system
- brctl showstp <bridge>: shows the state information and the path costs
- brctl addif <bridge> <interface>: adds an interface to a bridge
- brctl delif <bridge> <interface>: removes an interface from a bridge
Sometimes it is useful to manipulate and see the arp table on a box. The following is a list of useful ARP/MAC address related commands:
- arping <IP Address>: Sends out an arp request for the MAC address associated with the IP Address. Useful in establishing layer 2 connectivity.
- ip neigh show: Shows ARP table and state for MAC addresses
- ip neigh help: Shows help about how to manipulate the ARP table on a box. Use with caution
There are several ways to manipulate and show the routing table on Linux. The following is a list of useful related commands for basic manipulation:
- ip route show: shows the routing table in the new format
- route -n: shows the routing table in traditional format as known to the to the kernel
- route: route with out options will hang if there are routing problems
- ip route replace default via <gateway> dev <ethernet device>: Replace the default route with a new route
- ip route del <route>: deletes default route
- ip route add default via <gateway> dev <ethernet device>: Adds default route
xen common network
Dom0 is reachable, domU's are intermittently not reachable
Customers have reported this issue can be fixed by disabling redirects to prevent intermittent network interruptions with entrys in /etc/sysctl.conf:
This Support Knowledgebase provides a valuable tool for NetIQ/Novell/SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.
- Document ID:7001989
- Creation Date:21-NOV-08
- Modified Date:11-AUG-14
- SUSESUSE Linux Enterprise Server