My debian/vmware host blades have vlans trunked in, such that host01 would have eth0.2, eth0.3, eth0.4 and so forth. Then in /etc/vmware/locations I set:

answer VNET_0_INTERFACE eth0
answer VNET_2_INTERFACE eth0.2
answer VNET_4_INTERFACE eth0.4

So that when I create a vmware guest I can choose eth0/eth1 to be bridges to the /dev/vmnet2 or so forth, and know that specific interface will be on a particular vlan. It’s pretty simple, and I like it that way.

I’m trying to convert from debian/vmware server to ubuntu/kvm, for a bunch of reasons. Mostly I don’t like the VMware server UI, although I admit I haven’t tried 2.0 yet. I know that VMware Virtual Infrastructure and friends have more advanced UIs, but of course, they cost money. I’m not crazy about the backdoor interfaces that I use to get data between a host and a guest (right now I have a host tell the guest what it’s hostname is, so the guest can report to iclassify what it’s host is, so I have this information automatically so one doesn’t have to track it elsewhere). I want hot-add memory and migration, without going to ESX and whatnot as well.

Lately I’ve been playing with kvm, libvirt, and ubuntu-vm-builder. The interesting thing is that it appears very user oriented. I guess that is to be expected, most of ubuntu users are individuals and not enterprises. I gave a talk last month where someone asked me if my coworkers care about puppet running on my servers, to which I replied with, “What coworkers?” So I’m in a great position where I can mess about with useful, new technology without having to put together a slide deck and give a meeting to managers about why it’s a good idea. People that I work with trust that I have good ideas, and that’s enough for all of us.

Because of all this user focus, libvirt networking leaves a little to be desired. It’s defined in vi /etc/libvirt/qemu/networks/, with the default network in default.xml. There’s a subdirectory of autostart where you can link back to other networks, which brings these up on startup. You can see them in virsh with ‘net-list –all’. I shut it down with ‘net-destroy default’, and saw the ‘vnet0’ interface dissappear from ‘ifconfig -a’. The format of the libvirt network xml file is partially documented.

Your node/guest/domain configuration is in /etc/libvirt/qemu/domain.xml. There’s a section like this:

<interface type='network'>
<mac address='52:54:00:2a:26:25'/>
<source network='default'/>
</interface>

Which creates a NAT’d interface based on the networks/default.xml configuration. I tried changing the interface type to bridge and the source network to eth0, and got: “libvir: QEMU error : Failed to add tap interface ‘vnet%d’ to bridge ‘eth0’ : Operation not supported”.

Changing interface type to bridge and source to “bridge’br0′”, running ‘define domain.xml’ in virsh then ‘start domain’ produced the expected “libvir: QEMU error : Failed to add tap interface ‘vnet%d’ to bridge ‘br0’ : No such device”.

Then running ‘brctl addbr br0 ; brctl addif br0 eth0’ to produce the bridge and starting the domain again allows life to continue as expected.

Most of the HOWTO’s out there for doing briding have you create a bridge interface, bridge it to eth0, and move your ip address to it. You can create subinterfaces, like br0.4 to map to eth0.4.

You may see that if you start a second guest with the same configuration, libvirt is going to create a vnetN interface for each additional domain which it adds to the bridge. There are some notes here, but really, install the libvirt-doc package and take a look at ‘/usr/share/doc/libvirt-doc/format.html’ as it’s the best technical reference I’ve found so far.

# The primary network interface
auto eth0
iface eth0 inet manual
	up ifconfig $IFACE up

auto eth0.2
iface eth0.2 inet manual
        pre-up /sbin/vconfig add eth0 2
	up ifconfig $IFACE up

auto br0.2
iface br0.2 inet manual
	bridge_ports eth0.2
	bridge_stp off
	bridge_maxwait 0
	bridge_fd 0
	bridge_hello 0

Edit: You can bridge multiple interfaces by duplicating your efforts. libvirt/kvm will add an extra vnet interface for each interface element in the domain xml file, and bridge it to the bridge interface you specify. Be sure to run ‘virsh define /path/to/domain.xml’ to read in the changes.

  <devices>
    <interface type='bridge'>
      <source bridge='br0.4'/>
    </interface>
    <interface type='bridge'>
      <source bridge='br0.11'/>
    </interface>
  </devices>