Category Archives: pax

Enterprise networking with kvm and libvirt

My debian/vmware host blades have vlans trunked in, such that host01 would have eth0.2, eth0.3, eth0.4 and so forth. Then in /etc/vmware/locations I set:

answer VNET_0_INTERFACE eth0
answer VNET_2_INTERFACE eth0.2
answer VNET_4_INTERFACE eth0.4

So that when I create a vmware guest I can choose eth0/eth1 to be bridges to the /dev/vmnet2 or so forth, and know that specific interface will be on a particular vlan. It’s pretty simple, and I like it that way.

I’m trying to convert from debian/vmware server to ubuntu/kvm, for a bunch of reasons. Mostly I don’t like the VMware server UI, although I admit I haven’t tried 2.0 yet. I know that VMware Virtual Infrastructure and friends have more advanced UIs, but of course, they cost money. I’m not crazy about the backdoor interfaces that I use to get data between a host and a guest (right now I have a host tell the guest what it’s hostname is, so the guest can report to iclassify what it’s host is, so I have this information automatically so one doesn’t have to track it elsewhere). I want hot-add memory and migration, without going to ESX and whatnot as well.

Lately I’ve been playing with kvm, libvirt, and ubuntu-vm-builder. The interesting thing is that it appears very user oriented. I guess that is to be expected, most of ubuntu users are individuals and not enterprises. I gave a talk last month where someone asked me if my coworkers care about puppet running on my servers, to which I replied with, “What coworkers?” So I’m in a great position where I can mess about with useful, new technology without having to put together a slide deck and give a meeting to managers about why it’s a good idea. People that I work with trust that I have good ideas, and that’s enough for all of us.

Because of all this user focus, libvirt networking leaves a little to be desired. It’s defined in vi /etc/libvirt/qemu/networks/, with the default network in default.xml. There’s a subdirectory of autostart where you can link back to other networks, which brings these up on startup. You can see them in virsh with ‘net-list –all’. I shut it down with ‘net-destroy default’, and saw the ‘vnet0’ interface dissappear from ‘ifconfig -a’. The format of the libvirt network xml file is partially documented.

Your node/guest/domain configuration is in /etc/libvirt/qemu/domain.xml. There’s a section like this:

<interface type='network'>
<mac address='52:54:00:2a:26:25'/>
<source network='default'/>

Which creates a NAT’d interface based on the networks/default.xml configuration. I tried changing the interface type to bridge and the source network to eth0, and got: “libvir: QEMU error : Failed to add tap interface ‘vnet%d’ to bridge ‘eth0’ : Operation not supported”.

Changing interface type to bridge and source to “bridge’br0′”, running ‘define domain.xml’ in virsh then ‘start domain’ produced the expected “libvir: QEMU error : Failed to add tap interface ‘vnet%d’ to bridge ‘br0’ : No such device”.

Then running ‘brctl addbr br0 ; brctl addif br0 eth0’ to produce the bridge and starting the domain again allows life to continue as expected.

Most of the HOWTO’s out there for doing briding have you create a bridge interface, bridge it to eth0, and move your ip address to it. You can create subinterfaces, like br0.4 to map to eth0.4.

You may see that if you start a second guest with the same configuration, libvirt is going to create a vnetN interface for each additional domain which it adds to the bridge. There are some notes here, but really, install the libvirt-doc package and take a look at ‘/usr/share/doc/libvirt-doc/format.html’ as it’s the best technical reference I’ve found so far.

# The primary network interface
auto eth0
iface eth0 inet manual
	up ifconfig $IFACE up

auto eth0.2
iface eth0.2 inet manual
        pre-up /sbin/vconfig add eth0 2
	up ifconfig $IFACE up

auto br0.2
iface br0.2 inet manual
	bridge_ports eth0.2
	bridge_stp off
	bridge_maxwait 0
	bridge_fd 0
	bridge_hello 0

Edit: You can bridge multiple interfaces by duplicating your efforts. libvirt/kvm will add an extra vnet interface for each interface element in the domain xml file, and bridge it to the bridge interface you specify. Be sure to run ‘virsh define /path/to/domain.xml’ to read in the changes.

    <interface type='bridge'>
      <source bridge='br0.4'/>
    <interface type='bridge'>
      <source bridge='br0.11'/>

PAX 2007 – A convention for the gamer community

It’s the day after my first trip to PAX and I have to say I enjoyed it completely. This being the fourth PAX, and the first at the Washington State Convention Center in Seattle (previous events were held at the Meydenbauer center in Bellevue, but it was outgrown). When I go to computer conventions I prefer the community ones. I learn more, meet more good people, and always enjoy it. The same could be said for PAX. Tabletop, PC or console gaming; it was all to be found here.

Attendance was 19,323 at PAX 2006 and pre-registration this year topped 22,000. I haven’t been able to find a final number but over 30,000 seems to be the common figure. It was huge. There are some venue threads over at the pa forums and everyone seems to be responding with, ‘at least it wasnt Meydenbauer’. I personally thought the venue was alright (but of course loved the show). As Adam stated, the layout at the WSCC blows. There were two large spaces that were connectable that were used as the expo and main theatre. The PC area was pretty large, and nearby with a large hallway between it. Everything else though, was all over the place.

I got into the PC Freeplay area at once point and loved the idea. It was great to try out some games, but got kicked out due to an enforcer being sent to clear out our row. Unfortunately he didn’t know why, just to tell us all that our time was up. This kind of sucked being the last row filled, we were the first ones out, but understandable. The tourneys had no spectactor setup though, which I felt was a big failing. PAX had lots to do, but there were too many people for all of it. Having a couple projectors running spectator in some of the matches, or even just someone with a live camera standing around the players and an area for us to stand and watch some/all of the omegathon tourneys would have been great.

The Console freeplay I never got to try but thought it was a wonderful idea. I don’t know if the line was so slow because there weren’t enough console/space, or weren’t enough volunteers. Again, amazing idea, I really wanted to try out games/consoles to convince myself to buy them, but couldn’t. This is a huge note to the vendors, more consoles for people to play! Even in the expo halls there were often lines wrapping around booths for a chance to play demos and I’m sure there were many like me that wanted to try them, but didn’t want to stand in the line for 30+ minutes for five or ten minutes of play time. The console space was fine, it was basically smaller meeting rooms on it’s own floor. But it was far away from the rest of the conference and walking over to check the line got tiresome. You had to walk through a hallway that felt like a service hallway to get between the two areas. It was tight, and even had service elevators. This isn’t PAX’s fault, I just consider WSCC a poorly designed venue.

The closest theater to the expo was down the hall in a corner, rather than having it’s own room. This theater needed video of the talks on a projection screen to the sides with speakers on the pillar towards the back. Because of the corner, every time I came over here I realized I’d have to fight to get close enough to hear. I’m sure they weren’t using a PA system very loud because it was in fact in a hall, but solutions were needed and I think a couple of satellite screens and speakers for those perhaps not asking questions would have been great.

I only went to one of the other theaters and it was pretty much full. Not nearly as bad as others though. For instance when waiting to get into PAX we had to line up in two giant rooms filled with people. I didn’t really understand the line. We were all getting in, it was just were they stuck us to wait for everyone to get ready. I ended up leaving the line and wandering around. I unfortunately didn’t go to any of the concerts because the bracelet system wasn’t clear as to where I get them and how to know if they ran out, and if I should bother coming if I didn’t get one of the wristbands in the morning.

The lines for the main theater were absurd. Wrapping down a service hallway into one of the aforementioned giant rooms. No other great solution I guess. I really would have liked to see these huge rooms used for other things though. Move the PC and console freeplay areas over there and make room for the spectators. If you can split the old PC area in two (i didn’t notice if there was a partition) turn it into two theaters. This does risk turning the area in the middle into a line catastrophe, and without knowing the deal with the venue and fire codes I can’t give a great solution, but consideration of the above comments would be a good thing.

It looks like clearwire provided wireless in different areas by hooking up a linksys wireless router to a clearwire modem in different locations around the convention. First, these really should have been in the line rooms. We needed something to do in the huge rooms while waiting for events. I saw ‘pipe cleaners’ given out to one row of folks to play with, but that didn’t last long. Then ball throwing ensued. Mostly everyone played portables, but being a geek I wanted net access. I’ve helped at shmoocon doing this sort of thing, and I’ve done it for less geeky conferences in the past, including outdoor fairs with wireless and voip. So I understand why they’d want to keep it simple, but I really hope they go with a full network next year. If you’re from PAX and you’re not sure how to go about this, email me, I’d be happy to volunteer to coordinate it.

All in all, it was amazing. Props to penny arcade and all the enforcers/volunteers/sponsors. I can’t wait until next year.