Comments on: KVM Virtio network performance http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/ Tue, 07 Sep 2010 11:37:30 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: btm http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-2867 btm Sat, 20 Feb 2010 23:17:03 +0000 http://blog.loftninjas.org/?p=256#comment-2867 Right, well, there are a lot of things one could test. I didn't switch to KVM because of the virtio speed. Primarily the change was because it is open source. While testing kvm with and without virtio, it seemed prudent to try some vmware and debian combinations since I still had them lying around. I believe at the time I couldn't find any numbers, so my goal was to produce some data more than write a complete article covering virtualized networking on many different platforms. Right, well, there are a lot of things one could test. I didn’t switch to KVM because of the virtio speed. Primarily the change was because it is open source. While testing kvm with and without virtio, it seemed prudent to try some vmware and debian combinations since I still had them lying around.

I believe at the time I couldn’t find any numbers, so my goal was to produce some data more than write a complete article covering virtualized networking on many different platforms.

]]> By: gr8linux http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-2866 gr8linux Sat, 20 Feb 2010 20:46:08 +0000 http://blog.loftninjas.org/?p=256#comment-2866 The realistic test would be between virtio and VMXNET because both of them using paravirtualized NIC and you should change the guest OS The realistic test would be between virtio and VMXNET because both of them using paravirtualized NIC and you should change the guest OS

]]> By: Osis http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-2018 Osis Mon, 28 Sep 2009 10:16:29 +0000 http://blog.loftninjas.org/?p=256#comment-2018 Yes, kvm networking should be used with virtio, other options is if you have some broken system that you have to feed "real" interface with "real" driver. Yes, kvm networking should be used with virtio, other options is if you have some broken system that you have to feed “real” interface with “real” driver.

]]> By: Scott http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-912 Scott Thu, 23 Oct 2008 20:20:15 +0000 http://blog.loftninjas.org/?p=256#comment-912 Ya, the moral is that full virt I/O of any sort is slow, HVM Xen suffers the same problems. virtio and para-virt drivers solve this, I imagine kvm uses virtio for block i/o as well, you should see similar improvements there. Ya, the moral is that full virt I/O of any sort is slow, HVM Xen suffers the same problems. virtio and para-virt drivers solve this, I imagine kvm uses virtio for block i/o as well, you should see similar improvements there.

]]> By: btm http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-911 btm Thu, 23 Oct 2008 19:38:36 +0000 http://blog.loftninjas.org/?p=256#comment-911 Between two debian etch hosts (running vmware) on the same switch: <pre> bryanm@vmware02:~$ iperf -c vmware01 ------------------------------------------------------------ Client connecting to vmware01, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.162 port 46332 connected with 10.0.1.161 port 5001 [ 3] 0.0-10.0 sec 1.09 GBytes 940 Mbits/sec bryanm@vmware02:~$ uname -a Linux vmware02 2.6.18-6-amd64 #1 SMP Sun Feb 10 17:50:19 UTC 2008 x86_64 GNU/Linux bryanm@vmware02:~$ lsb_release -a LSB Version: core-2.0-noarch:core-3.0-noarch:core-3.1-noarch:core-2.0-amd64:core-3.0-amd64:core-3.1-amd64 Distributor ID: Debian Description: Debian GNU/Linux 4.0 (etch) Release: 4.0 Codename: etch </pre> From said debian/vmware host, to a debian/vmware (i686) guest: <pre> bryanm@vmware02:~$ iperf -c web-stage02 ------------------------------------------------------------ Client connecting to web-stage02, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.162 port 39826 connected with 10.0.1.62 port 5001 [ 3] 0.0-10.0 sec 941 MBytes 789 Mbits/sec </pre> It's interesting to compare those numbers: 780MB : VMWare Host -> Guest 315MB : KVM Host -> Guest w/o virtio 1.15GB: KVM HOST -> Guest w/virtio Of course I could be missing something, but it's interesting data at least, and I can't find any sort of data anywhere out there. Moral of the story? KVM Networking sucks without virtio, but kicks ass with it. Between two debian etch hosts (running vmware) on the same switch:

bryanm@vmware02:~$ iperf -c vmware01
------------------------------------------------------------
Client connecting to vmware01, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.1.162 port 46332 connected with 10.0.1.161 port 5001
[  3]  0.0-10.0 sec  1.09 GBytes    940 Mbits/sec
bryanm@vmware02:~$ uname -a
Linux vmware02 2.6.18-6-amd64 #1 SMP Sun Feb 10 17:50:19 UTC 2008 x86_64 GNU/Linux
bryanm@vmware02:~$ lsb_release -a
LSB Version:	core-2.0-noarch:core-3.0-noarch:core-3.1-noarch:core-2.0-amd64:core-3.0-amd64:core-3.1-amd64
Distributor ID:	Debian
Description:	Debian GNU/Linux 4.0 (etch)
Release:	4.0
Codename:	etch

From said debian/vmware host, to a debian/vmware (i686) guest:

bryanm@vmware02:~$ iperf -c web-stage02
------------------------------------------------------------
Client connecting to web-stage02, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.1.162 port 39826 connected with 10.0.1.62 port 5001
[  3]  0.0-10.0 sec    941 MBytes    789 Mbits/sec

It’s interesting to compare those numbers:
780MB : VMWare Host -> Guest
315MB : KVM Host -> Guest w/o virtio
1.15GB: KVM HOST -> Guest w/virtio

Of course I could be missing something, but it’s interesting data at least, and I can’t find any sort of data anywhere out there. Moral of the story? KVM Networking sucks without virtio, but kicks ass with it.

]]> By: btm http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-909 btm Thu, 23 Oct 2008 19:24:37 +0000 http://blog.loftninjas.org/?p=256#comment-909 @Scott: virt04 -> ops02, mon02 as guests. mon02 has virtio. <pre> bryanm@virt03:~$ iperf -c ops02 ------------------------------------------------------------ Client connecting to ops02, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.171 port 47372 connected with 10.0.1.13 port 5001 [ 3] 0.0-10.0 sec 469 MBytes 393 Mbits/sec bryanm@virt03:~$ iperf -c mon02 ------------------------------------------------------------ Client connecting to mon02, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.171 port 53974 connected with 10.0.1.33 port 5001 [ 3] 0.0-10.0 sec 737 MBytes 618 Mbits/sec </pre> I ran that a few more times and ops02 is consistently just under 400 Mbits/sec, while mon02 is consistently just over 600 Mbits/sec. Which is half of that 1.3 Gbits/sec figure. Host to host is about 940 Mbits/sec: <pre> bryanm@virt04:~$ iperf -c virt03 ------------------------------------------------------------ Client connecting to virt03, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.172 port 34888 connected with 10.0.1.171 port 5001 [ 3] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec </pre> ethtool reports: <pre> Settings for eth0: Supported ports: [ FIBRE ] Supported link modes: 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 1000baseT/Full Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: FIBRE PHYAD: 2 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: d Link detected: yes </pre> So there is some performance loss there still by using virtualization (instead of not, not specifically talking about virtio). I haven't done any tuning really. It is interesting though, to consider running guests that have a lot of data transfer between each other on the same host. @Scott:

virt04 -> ops02, mon02 as guests. mon02 has virtio.

bryanm@virt03:~$ iperf -c ops02
------------------------------------------------------------
Client connecting to ops02, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.1.171 port 47372 connected with 10.0.1.13 port 5001
[  3]  0.0-10.0 sec    469 MBytes    393 Mbits/sec
bryanm@virt03:~$ iperf -c mon02
------------------------------------------------------------
Client connecting to mon02, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.1.171 port 53974 connected with 10.0.1.33 port 5001
[  3]  0.0-10.0 sec    737 MBytes    618 Mbits/sec

I ran that a few more times and ops02 is consistently just under 400 Mbits/sec, while mon02 is consistently just over 600 Mbits/sec.

Which is half of that 1.3 Gbits/sec figure. Host to host is about 940 Mbits/sec:

bryanm@virt04:~$ iperf -c virt03
------------------------------------------------------------
Client connecting to virt03, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.0.1.172 port 34888 connected with 10.0.1.171 port 5001
[  3]  0.0-10.0 sec  1.10 GBytes    941 Mbits/sec

ethtool reports:

Settings for eth0:
	Supported ports: [ FIBRE ]
	Supported link modes:   1000baseT/Full
	Supports auto-negotiation: Yes
	Advertised link modes:  1000baseT/Full
	Advertised auto-negotiation: Yes
	Speed: 1000Mb/s
	Duplex: Full
	Port: FIBRE
	PHYAD: 2
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: g
	Wake-on: d
	Link detected: yes

So there is some performance loss there still by using virtualization (instead of not, not specifically talking about virtio). I haven’t done any tuning really. It is interesting though, to consider running guests that have a lot of data transfer between each other on the same host.

]]> By: Scott http://blog.loftninjas.org/2008/10/22/kvm-virtio-network-performance/comment-page-1/#comment-906 Scott Thu, 23 Oct 2008 02:14:33 +0000 http://blog.loftninjas.org/?p=256#comment-906 I assume the improvements exist even when communicating with another machine via the bridge? I assume the improvements exist even when communicating with another machine via the bridge?

]]>