Category Archives: Redhat

FS-Cache merged in Kernel 2.6.30

Posted on by
2

FS-Cache has been merged into the upcoming kernel 2.6.30.  This allows for a generic caching interface in the kernel for other file systems.  For example, you can use local hard disks to cache data accessed via NFS, AFS, or CD-Rom.  Since these tend to be high-latency while the disks are low latency, it should provide for a nice speedup.

Of particular interest to me, I contacted maintainer David Howells who is a Redhat employee.  I asked whether this infrastructure would help with large disk image files stored on NFS — a common though not particularly efficient case for VMWare, Xen, KVM, etc.  His exact response was “Quite feasible.  As long as you have a local disk on which to cache the files.”

I am quite happy as I run this setup at work for some production VMs since it allows for easy migration and backup without the complexity and cost of a SAN or cluster FS.  I look forward to testing when 2.6.30 hits the stable tree.

I hate Ubuntu

Posted on by
100

I hate Ubuntu.  I immediately lose respect for anyone who runs it, and especially those who advocate it.   Here’s why:

Name 20 features, release-for-release or year-for-year that have not come from Redhat.  Redhat basically runs the show when it comes to Linux.  This includes things like NetworkManager, Gnome, Xorg, GCC, glibc, LVM, KVM, kernel, file systems et al.  Redhat has developers making significant contributions to the entire FOSS software stack upstream.

Ubuntu on the other hand pulls most of the heavy weight packaging from Debian with each release.  They then perform minor patching and testing.  It generally lags behind Fedora by a release or two in parts of the software stack.  I never see @ubuntu or @cannocial email addresses in upstream changelogs.

So tell me again, how exactly does Ubuntu innovate?  They even struggle to release a new theme with each release, and artwork is about the only original thing in Ubuntu.

Yes, Ubuntu is stable because they are standing on the shoulders of giants.  Most of the hard work is hashed out before they ever import software into their repositories.  This is fine, and what FOSS is all about, but I prefer to be in with the leaders rather than the followers.

What really irks me and what has really brewed my hatred are Ubuntu users.  They seem to think Ubuntu is responsible for all that is good in the FOSS world.  I have just proven how false this is.  In my experience, Ubuntu support mechanisms (IRC, mailing lists, forums) are much less helpful than the alternative.

If you want a nice desktop distro, run Fedora or OpenSUSE.  If you like control, run Gentoo.  If you need stability, run RHEL/CentOS or Debian.  But please, don’t feed the idiot magnet that is Ubuntu!

Xen 3.3 in RHEL/CentOS 5 and more Link Aggregation Fun

Posted on by
2

RHEL 5 includes the now ancient Xen 3.0 hypervisior.  A lot has been improved since then, especially in the current 3.3 release.  Additionally, RedHat now owns the company behind KVM, so it is unlikely they will spend much time backporting Xen stuff for RHEL 5.3 or the likes.

Why Xen?

Xen is a proven hypervisor.  It works well on lots of hardware, including servers without hardware virtualization and older 64-bit Opterons that wont run 64-bit guests in the likes of VMWare.  Since the OS is usually paravirtualized, performance is top notch.  By making an OS aware of the environment it is running in, you can optimize it for virtualization.  KVM is playing catchup here, realizing that paravirtualization is still ideal for many things.

How..

Okay, so we are using or want to use Xen. Others have already built the packages we need, thankfully!

Head over to http://www.gitco.de/repo/ and grab the repo for your arch.  (Most likely wget http://www.gitco.de/repo/CentOS5-GITCO_x86_64.repo in /etc/yum.repos.d/ for the uninitiated).

If you already have Xen installed, you may need to remove and readd it.

yum groupremove Virtualization
yum groupinstall Virtualization

You’ll also get some updated tools like Virtual Machine Monitor 0.6.0 that make it easier to install newer guests such as Fedora 10 or Ubuntu.  Sweet!

Double check /etc/sysconfig/kernel.  It should be set to kernel-xen.  Likewise, check /boot/grub.conf and make sure that the Xen kernel is the default if the aforementioned was not done beforehand.

Reboot!

Xen 3.3 and Link Bonding

See my previous post for general information, but it gets harder.

This one is a nightmare.  In my previous post, I detailed how to get Xen to work with link aggregation with Xen 3.0.  Well, it doesn’t work in 3.3.  Xen decides that it still owns eth0 and completely destroys your bond0 setup.

Like these people, I’ve come to the conclusion that the integrated network scripts suck.  This is alarming since you’d think link bonded setups would be the norm for Xen setups.

The quick fix is to let the OS handle networking.  We do that like so: add a br0 interface and tell the bond to bridge with it.

File /etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
ONBOOT=yes
BOOTPROTO=none
IPADDR=10.0.6.201
NETMASK=255.255.255.0
GATEWAY=10.0.6.1
NO_ALIASROUTING=yes
TYPE=Bridge

Then, edit your /etc/sysconfig/network-scripts/ifcfg-bond0 and add “BRIDGE=br0″ and comment out any IP related information (since you are now defining that in the bridge.  Head over to /etc/sysctl.conf and add:

net.ipv4.ip_forward = 1

Now, edit your Xen VMs in /etc/xen/ or /etc/xen/auto and change xenbr0 to br0:

vif = [ ‘mac=ee:cc:aa:88:66:44, bridge=br0′, ]

Okay, now disable the Xen networking garbage.  Open /etc/xen/xend-config.sxp and comment out anything  that looks like (network-script ….).

Almost done, but wait!  RHEL 5.2 has a bug that prevents the bridge coming up on a bonded interface.  Hopefully this will make the 5.3 cut or be pushed to 5.2, but until then go here.  Download the new patch into /etc/sysconfig/network-scripts/ and run patch -p0 < ifup-eth.patch for instance.

Finish

Reboot.  You now have Xen 3.3 goodness on a big Ethernet channel!  Post a comment if you have any trouble or questions.

Link Bonding Craziness in RHEL/Centos 5

Posted on by
6

I just went through hell in a handbasket trying to get 802.3ad Link Aggregation set up on a Centos 5.2 Xen box.  Setting up link aggregation itself isn’t that bad – http://wiki.centos.org/TipsAndTricks/BondingInterfaces for a simple guide (after your managed switch is configd) – but what ever I did, I was unable to get both interfaces simultaneously active.

About the only useful debugging info I got was that the MAC was in use.  I was puzzled because as far as I know, link agg takes over the primary MAC and sets that up for both NICs.  Furthermore, the same exact hardware was working great on Fedora 10.

bonding: bond0: Warning: the permanent HWaddr of eth0
 - [MAC ADDR]- is still in use by bond0. Set the HWaddr of eth0
to a different address to avoid conflicts.
bonding: bond0: releasing active interface eth0
bonding: bond0: making interface eth1 the new active one.
bonding: bond0: Removing slave eth1
bonding: bond0: releasing active interface eth1
ADDRCONF(NETDEV_UP): bond0: link is not ready
bonding: bond0: Adding slave eth0.

Luckily, I stumbled across this bug report.  If you scroll down to the last comment, this appears to be a Xen specific issue.  By default Xen tries to set its bridge up on eth0, and I assume this prevents the kernel bonding driver from taking over the NIC.  By opening up /etc/xen/xend-config.sxp and adding:

(network-script 'network-bridge netdev=bond0')

Xen will bridge to the bond0 interface, and everything will work as expected.

Another trick I had to do was add a start delay to the networking scripts.  This is useful if your hardware is crap (cough Broadcom), you need a dhcp lease and it fails, or you are running STP, link aggr., etc.  On Fedora, RHEL, and derivitives this is accomplished by adding the NETWORKDELAY directive to /etc/sysconfig/network:

NETWORKING=yes
NETWORKDELAY=31

If you need more granularity, you can set delays to specific adapters in the /etc/sysconfig/ifcfg-{x} files with the LINKDELAY directive.

Just a couple of hard lesssons from the trenches, hopefully this will save someone else some time.