There is now a limit # of virtio-devices/drives?

Owi

New Member
Aug 13, 2011
13
0
1
Hi, I recently upgraded from 1.8 to 2.1, and just noticed the limit on devices introduced.

As far as I understand from this thread there might be a problem with qemu-kvm which will crash the vm if exceeding x virtio devices.

One of the vm's is a file server, and have over 10 drives attached in 1.8, but it will not load more than 5 in 2.1.
The GUI doesnt allow for more than 5 virtio devices, and cli method is not promising either.

Code:
root@multifatso:/etc/pve/nodes/multifatso/qemu-server# qm set 109 --virtio6 /dev/disk/by-id/scsi-SATA_SAMSUNG_HD204UI.....
Unknown option: virtio6
400 unable to parse option
qm set <vmid> [OPTIONS]

I'm sorry if I have missed a post explaining this - I have read this forum for the last 3 days, but only come up with more questions.
Is this supposed to be like this now - ie. I should not use Proxmox at all for my fileserver (I know it's not recommended to put a fileserver in a vm), decomission my cluster and let the other server run all vm's, or is it a coming fix for it?


I'm hoping I just misunderstood something.
 
Last edited:

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
14,700
663
133
the current device model does not allow more, but this limit will change in future.

if you use the latest kvm form the pvetest repository (1.1rc3), no virtio crashes should occur if you use the maxmimum virtio devices
 

Owi

New Member
Aug 13, 2011
13
0
1
Ok. Thanks for the quick clarification.

Guess I can set up a couple of vms with 5 drives each instead until it's changed.
I can only end up learning more, so I cant go wrong.

Just out of curiousity; is there a technical reason for this limitation or simply a chosen design?

I read that sticky yesterday evening, but havent figured out how to actually use the pvetest repository yet.
Do you have a link for it, or is it rtfm?

edit:rtfm I guess.
added
deb http://download.proxmox.com/debian squeeze pvetest
to source.list, and apt-get dist-upgrade seemed to do what I wanted.

I find myself more and more often say to myself: I should know this - but it's blank.
There's been damage for sure...
 
Last edited:

tom

Proxmox Staff Member
Staff member
Aug 29, 2006
14,700
663
133
Ok. Thanks for the quick clarification.

Guess I can set up a couple of vms with 5 drives each instead until it's changed.
I can only end up learning more, so I cant go wrong.

Just out of curiousity; is there a technical reason for this limitation or simply a chosen design?

a technical reason, but the limitation will be removed as soon as we implemented the new device model.

I read that sticky yesterday evening, but havent figured out how to actually use the pvetest repository yet.
Do you have a link for it, or is it rtfm?

we have two repositories, pve is the stable one, pvetest is for testing.

you need to edit /etc/apt/sources.list to enable pvetest, see:

Code:
...
# PVE test packages provided by proxmox.com
deb [URL]http://download.proxmox.com/debian[/URL] squeeze pvetest
...

if you are not familiar with apt, I recommend the http://debian-handbook.info/
 

Owi

New Member
Aug 13, 2011
13
0
1
Thanks alot for that information.

I am quite familiar with debian, and have been using it for some time.
But due to an incident in 2009 knowledge have a way of turning into I-should-know-this.

I have never heard of http://debian-handbook.info/ though, but after reading up on it I figure it's well worth the money.
I'll order one right now. Thanks for the tips.
 

spirit

Famous Member
Apr 2, 2010
4,446
320
103
www.odiso.com
The main reason of fixing the number of devices, is that now proxmox use fixed pci slot for devices, so we have 31 pci slots available.(X vitio-disk, X vitio_net,ide,videocard,scsi controllers,usb controller,....)
with proxmox 1.X, pci slots was not fixed, so sometimes this cause bug in windows (network address change by example),disk order bug, and some other bugs.

This 31 pci slots limits sould be no more a limit soon. (they are a new pci bridge device in qemu 1.1 to go higher than 31)
 

dups

Member
Jun 8, 2011
10
0
21
The main reason of fixing the number of devices, is that now proxmox use fixed pci slot for devices, so we have 31 pci slots available.(X vitio-disk, X vitio_net,ide,videocard,scsi controllers,usb controller,....)
with proxmox 1.X, pci slots was not fixed, so sometimes this cause bug in windows (network address change by example),disk order bug, and some other bugs.

This 31 pci slots limits sould be no more a limit soon. (they are a new pci bridge device in qemu 1.1 to go higher than 31)

Sorry to dig up an old thread but I'm running into this exact issue with Proxmox 2.1 and qemu 1.1. When should this be changed to allow more than 5 drives?

Thanks!
 

Owi

New Member
Aug 13, 2011
13
0
1
Hi,
It's ok in pvetest repository. (16 virtio disks - 32 nics)

This is great news, but the 16 drives and 32 nics choice makes me curious.

I personally have 20 and 24 slot chassis, so I can easily see 32 drives and 16 nics as a reasonable solution.
At what scenario would 16/32 be needed/preferred?

Is it more complicated to implement larger number of virtio disk vs nics?
 

Owi

New Member
Aug 13, 2011
13
0
1
Why do you need more that 16 drives inside a VM (that sounds strange to me)?

It propably does, but it's not as much a requirement as a thought.

Well, I use proxmox to learn - as it is a safe place to live by trial-by-error.
One of my servers is a fileserver with about 20 drives, where I train in both linux distributions and different development languages.
This is purely for self-education.
I can work from several installations on the same files without the need to reinstall from scratch each time.
I often work with handling large files in several copies over several drives. 20 drives isnt that much these days.
Ideally this woldn't be virtualized at all, but there's to many benefits with proxmox for someone in my situation to not use it.



So this is why I can see the use of more then 16 drives, but I'm having problems seing the need for 32 nics.
Then again, my knowledge is limited and this is not meant to be critique - but rather a wish to get enlightened.
 
Last edited:

dietmar

Proxmox Staff Member
Staff member
Apr 28, 2005
16,771
396
103
Austria
www.proxmox.com
So this is why I can see the use of more then 16 drives,

You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).

I really think this is enough.


but I'm having problems seing the need for 32 nics.

AFAIK, some people run complex firewall with up to 32 ports (at least they requested that feature).
 

Owi

New Member
Aug 13, 2011
13
0
1
You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).

AFAIK, some people run complex firewall with up to 32 ports (at least they requested that feature).


I hadn't thought about the firewall scenario in proxmox at those scales - again a limitation of not being in a production environment.
I manage just fine as is, but I always need to know why stuff is the way it is.

And you helped me with that today.


Thanks alot.
 

kalleyne

New Member
May 22, 2013
21
0
1
You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).
Forgive me while I reopen this more than two year old thread.Are we still limited to 16 VIRTIO disks on the recent versions of Proxmox VE 3.4?
 

kalleyne

New Member
May 22, 2013
21
0
1
technically, it could be easy to add support for virtio-scsi to 512 disks.(2controllers with 256 disks)
Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would function? Thanks.
 

spirit

Famous Member
Apr 2, 2010
4,446
320
103
www.odiso.com
>>Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would >>function? Thanks.

virtio-scsi are like sas drives, they use a controller which can handle 256 drive.

virtio-blk, use 1 controller for 1 disk, so you are limited by virtual pci slots of the qemu machines.


you can edit
/usr/share/perl5/PVE/QemuServer.pm, and change MAX_SCSI_DISKS to 512.

(then restart pvedaemon service).

It should be enough.

 

kalleyne

New Member
May 22, 2013
21
0
1
>>Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would >>function? Thanks.virtio-scsi are like sas drives, they use a controller which can handle 256 drive.virtio-blk, use 1 controller for 1 disk, so you are limited by virtual pci slots of the qemu machines.you can edit /usr/share/perl5/PVE/QemuServer.pm, and change MAX_SCSI_DISKS to 512.(then restart pvedaemon service).It should be enough.
spirit, thank you for this useful bit of information. I'll certainly give this a try once I can conclude some PCI passthrough testing. Thanks.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!