RHEL 5 includes the now ancient Xen 3.0 hypervisior. A lot has been improved since then, especially in the current 3.3 release. Additionally, RedHat now owns the company behind KVM, so it is unlikely they will spend much time backporting Xen stuff for RHEL 5.3 or the likes.
Why Xen?
Xen is a proven hypervisor. It works well on lots of hardware, including servers without hardware virtualization and older 64-bit Opterons that wont run 64-bit guests in the likes of VMWare. Since the OS is usually paravirtualized, performance is top notch. By making an OS aware of the environment it is running in, you can optimize it for virtualization. KVM is playing catchup here, realizing that paravirtualization is still ideal for many things.
How..
Okay, so we are using or want to use Xen. Others have already built the packages we need, thankfully!
Head over to http://www.gitco.de/repo/ and grab the repo for your arch. (Most likely wget http://www.gitco.de/repo/CentOS5-GITCO_x86_64.repo in /etc/yum.repos.d/ for the uninitiated).
If you already have Xen installed, you may need to remove and readd it.
yum groupremove Virtualization yum groupinstall Virtualization
You’ll also get some updated tools like Virtual Machine Monitor 0.6.0 that make it easier to install newer guests such as Fedora 10 or Ubuntu. Sweet!
Double check /etc/sysconfig/kernel. It should be set to kernel-xen. Likewise, check /boot/grub.conf and make sure that the Xen kernel is the default if the aforementioned was not done beforehand.
Reboot!
Xen 3.3 and Link Bonding
See my previous post for general information, but it gets harder.
This one is a nightmare. In my previous post, I detailed how to get Xen to work with link aggregation with Xen 3.0. Well, it doesn’t work in 3.3. Xen decides that it still owns eth0 and completely destroys your bond0 setup.
Like these people, I’ve come to the conclusion that the integrated network scripts suck. This is alarming since you’d think link bonded setups would be the norm for Xen setups.
The quick fix is to let the OS handle networking. We do that like so: add a br0 interface and tell the bond to bridge with it.
File /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0 ONBOOT=yes BOOTPROTO=none IPADDR=10.0.6.201 NETMASK=255.255.255.0 GATEWAY=10.0.6.1 NO_ALIASROUTING=yes TYPE=Bridge
Then, edit your /etc/sysconfig/network-scripts/ifcfg-bond0 and add “BRIDGE=br0″ and comment out any IP related information (since you are now defining that in the bridge. Head over to /etc/sysctl.conf and add:
net.ipv4.ip_forward = 1
Now, edit your Xen VMs in /etc/xen/ or /etc/xen/auto and change xenbr0 to br0:
vif = [ ‘mac=ee:cc:aa:88:66:44, bridge=br0′, ]
Okay, now disable the Xen networking garbage. Open /etc/xen/xend-config.sxp and comment out anything that looks like (network-script ….).
Almost done, but wait! RHEL 5.2 has a bug that prevents the bridge coming up on a bonded interface. Hopefully this will make the 5.3 cut or be pushed to 5.2, but until then go here. Download the new patch into /etc/sysconfig/network-scripts/ and run patch -p0 < ifup-eth.patch for instance.
Finish
Reboot. You now have Xen 3.3 goodness on a big Ethernet channel! Post a comment if you have any trouble or questions.
Related posts:
- Link Bonding Craziness in RHEL/Centos 5 I just went through hell in a handbasket trying to...
- Xen 3.4.1 on RHEL/CentOS 5.4 I’m happy to report that the updated Gitco Xen 3.4.1...
- I dream of pervasive virtualization… I dream of a day where virtualization is pervasive. Instead...
- FS-Cache merged in Kernel 2.6.30 FS-Cache has been merged into the upcoming kernel 2.6.30. This...
- DRBD merged with kernel 2.6.33 DRBD has been a long standing external patch in many...
I think I have a simpler way to solve this…
As indicated here http://article.gmane.org/gmane.comp.emulators.xen.user/47039 the bond interface is taked down when network-bridge script is starting the bridge. The red-hat network-bridge script should solve the problem. But xen 3.3 doesn’t exist in redhat.
You can simply patch you network-bridge script with this patch
— network-bridge 2009-07-02 20:11:23.000000000 +0200
+++ network-bridge 2009-07-03 09:03:46.000000000 +0200
@@ -56,6 +56,10 @@
findCommand “$@”
evalVariables “$@”
+is_bonding() {
+ [ -f "/sys/class/net/$1/bonding/slaves" ]
+}
+
is_network_root () {
local rootfs=$(awk ‘{ if ($1 !~ /^[ \t]*#/ && $2 == “/”) { print $3; }}’ /etc/mtab)
local rootopts=$(awk ‘{ if ($1 !~ /^[ \t]*#/ && $2 == “/”) { print $4; }}’ /etc/mtab)
@@ -223,7 +227,8 @@
preiftransfer ${netdev}
transfer_addrs ${netdev} ${tdev}
- if ! ifdown ${netdev}; then
+ if is_bonding ${netdev} || ! ifdown ${netdev}; then
+ # do not shutdown interface if it is a bonding
# If ifdown fails, remember the IP details.
get_ip_info ${netdev}
ip link set ${netdev} down
and
cd /etc/xen/scripts
patch -N < file.patch
–
thibault ketterer.
Just a note, RHEL and CentOS 5.4 have resolved this issue.