QEMU 8 NIC limit

[JustaGuy]

While attempting to install a dual 10GB NIC to a Vyatta VM, I ran into a limitation in qemu that currently prevents adding more than 8 NICs.

My aim is for the VM to have 29x virtio NICs, 1x IDE CD-ROM & 1x virtio HDD.

I found a patch to allow 24, and changed it to 32, which is the number of total PCI slots.
View attachment max_nics.zip

I haven't figured out how to apply it yet myself, but I'll post it for the sake of whoever comes along next.

It was suggested that I submit it to qemu-dev for upstream adoption, but it may be moot with their changing to a new API.
It seems the new method of adding devices using a qdev / -device command won't encounter the same limitation as whatever it is now.

I'm not finding any documentation on how to do it that way.
Could someone point me in the right direction on how to do this?

 

[dietmar]

I just uploaded a fixed version to the pvetest repository:

# wget ftp://download.proxmox.com/debian/dists/lenny/pvetest/binary-amd64/pve-qemu-kvm_0.12.5-2_amd64.deb
# dpkg -i pve-qemu-kvm_0.12.5-2_amd64.deb

Please can you test that?

 

[JustaGuy]

Hmm, still fails to start.

 

[martin]

works here, just tested with 11 virtio nics. post your 'pveversion -v', how many nics do you use (virtio?), guest type?

 

[JustaGuy]

 pveversion -v
pve-manager: 1.6-2 (pve-manager/1.6/5087)
running kernel: 2.6.32-4-pve
proxmox-ve-2.6.32: 1.6-19
pve-kernel-2.6.32-4-pve: 2.6.32-19
qemu-server: 1.1-18
pve-firmware: 1.0-8
libpve-storage-perl: 1.0-14
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-7
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.12.5-2
ksm-control-daemon: 1.0-4

29 virtio nics
Guest Type: Linux 2.6

VMID.conf:

name: Moffatt
ide2: local:iso/vyatta-livecd-virt_VC6.1-2010.08.20_i386.iso,media=cdrom
bootdisk: virtio0
virtio0: local:333/vm-333-disk-1.raw
ostype: l26
memory: 2048
sockets: 1
onboot: 1
description: vyatta-virt-6.1-2010.08.20_i386
cores: 4
boot: dc
freeze: 0
cpuunits: 10000
acpi: 1
kvm: 1
startdate: 2010-10-05
vlan10: virtio=32:7D:86:10:B9:40
vlan1: virtio=3A:A9:A8:66:E9:97
vlan2: virtio=1A:3D:BE:5F:22:58
vlan3: virtio=76:90:A2:CE:9A:4D
vlan4: virtio=1E:64:A7:D6:01:CB
vlan0: virtio=5A:8C:92:8B:25:24
vlan67: virtio=A6:49:5C:DE:13:EB,virtio=26:CD:84:12:DE:65,virtio=1E:2D:F4:21:26:FF,virtio=8A:28:8F:3F:B9:05,virtio=4E:68:9B:3B:8A:FE,virtio=A6:51:1A:16:31:2C,virtio=0A:04:38:0C:6A:01,virtio=CA:44:5B:E5:C3:98,virtio=42:3F:A2:15:7D:C4,virtio=72:BE:13:FE:27:E4,virtio=5E:16:FA:0C:71:33,virtio=B6:C4:7D:E2:FD:6B,virtio=FE:B1:05:BA:4A:2A,virtio=AA:C9:22:0D:71:0E,virtio=82:3D:10:9A:B2:C4,virtio=E2:F8:4E:0B:BC:C7,virtio=4E:2F:B5:54:77:27,virtio=6E:28:5F:3E:1B:E3,virtio=56:9C:E1:32:CD:05,virtio=76:7A:93:85:A4:36
vlan26: virtio=1A:5F:C9:AD:B5:46
vlan27: virtio=BA:9E:A4:F7:5A:90
vlan28: virtio=3E:60:F6:76:8B:87

 

[dietmar]

It should be easy to test the maximum possible numer of NICs.

 

[JustaGuy]

In progress now, will post findings soon.

 

[JustaGuy]

This is now workable with your help, thank you.

It's still a NIC limitation, as opposed to a PCI limit.

It'll start with 33 PCI devices:
28 virtio NICs
+3 IDE HDDs
+1 IDE CD-ROM
+1 virtio HDD

It won't start with any combination that includes a 29th NIC of any variety.


Nevertheless, this is a major improvement from my point of view, thanks again.

 

[tom]

thanks for feedback. just to mention, on ESXi 4.1 you can use up to 10 virtual nic´s per VM.

 

[prometheanfire]

Do you plan on using or switching to http://git.qemu.org/qemu.git/tree/docs/qdev-device-use.txt

In testing this seems to lead to a 4x performance increase in network throughput.

 

[tom]

yes, we try to include all available feature as soon as possible.

 

[dietmar]

Do you plan on using or switching to http://git.qemu.org/qemu.git/tree/docs/qdev-device-use.txt

In testing this seems to lead to a 4x performance increase in network throughput.

Sorry, but why does a syntax change increase in network throughput?

 

[JustaGuy]

Sorry, but why does a syntax change increase in network throughput?

From the linux-kvm mailing list:

Te better choice is to use a newer command line. Not only does it avoid
the MAX_NICS limitation, but it also enables standard virtio-net offload
accelerations.

 

[tom]

already in kvm 0.13, see pvetest.