There is now a limit # of virtio-devices/drives?

[Owi]

Hi, I recently upgraded from 1.8 to 2.1, and just noticed the limit on devices introduced.

As far as I understand from this thread there might be a problem with qemu-kvm which will crash the vm if exceeding x virtio devices.

One of the vm's is a file server, and have over 10 drives attached in 1.8, but it will not load more than 5 in 2.1.
The GUI doesnt allow for more than 5 virtio devices, and cli method is not promising either.

root@multifatso:/etc/pve/nodes/multifatso/qemu-server# qm set 109 --virtio6 /dev/disk/by-id/scsi-SATA_SAMSUNG_HD204UI.....
Unknown option: virtio6
400 unable to parse option
qm set <vmid> [OPTIONS]

I'm sorry if I have missed a post explaining this - I have read this forum for the last 3 days, but only come up with more questions.
Is this supposed to be like this now - ie. I should not use Proxmox at all for my fileserver (I know it's not recommended to put a fileserver in a vm), decomission my cluster and let the other server run all vm's, or is it a coming fix for it?


I'm hoping I just misunderstood something.

 

[tom]

the current device model does not allow more, but this limit will change in future.

if you use the latest kvm form the pvetest repository (1.1rc3), no virtio crashes should occur if you use the maxmimum virtio devices

 

[Owi]

Ok. Thanks for the quick clarification.

Guess I can set up a couple of vms with 5 drives each instead until it's changed.
I can only end up learning more, so I cant go wrong.

Just out of curiousity; is there a technical reason for this limitation or simply a chosen design?

I read that sticky yesterday evening, but havent figured out how to actually use the pvetest repository yet.
Do you have a link for it, or is it rtfm?

edit:rtfm I guess.
added
deb http://download.proxmox.com/debian squeeze pvetest
to source.list, and apt-get dist-upgrade seemed to do what I wanted.

I find myself more and more often say to myself: I should know this - but it's blank.
There's been damage for sure...

 

[tom]

Ok. Thanks for the quick clarification.

Guess I can set up a couple of vms with 5 drives each instead until it's changed.
I can only end up learning more, so I cant go wrong.

Just out of curiousity; is there a technical reason for this limitation or simply a chosen design?

a technical reason, but the limitation will be removed as soon as we implemented the new device model.

I read that sticky yesterday evening, but havent figured out how to actually use the pvetest repository yet.
Do you have a link for it, or is it rtfm?

we have two repositories, pve is the stable one, pvetest is for testing.

you need to edit /etc/apt/sources.list to enable pvetest, see:

...
# PVE test packages provided by proxmox.com
deb [URL]http://download.proxmox.com/debian[/URL] squeeze pvetest
...

if you are not familiar with apt, I recommend the http://debian-handbook.info/

 

[Owi]

Thanks alot for that information.

I am quite familiar with debian, and have been using it for some time.
But due to an incident in 2009 knowledge have a way of turning into I-should-know-this.

I have never heard of http://debian-handbook.info/ though, but after reading up on it I figure it's well worth the money.
I'll order one right now. Thanks for the tips.

 

[spirit]

The main reason of fixing the number of devices, is that now proxmox use fixed pci slot for devices, so we have 31 pci slots available.(X vitio-disk, X vitio_net,ide,videocard,scsi controllers,usb controller,....)
with proxmox 1.X, pci slots was not fixed, so sometimes this cause bug in windows (network address change by example),disk order bug, and some other bugs.

This 31 pci slots limits sould be no more a limit soon. (they are a new pci bridge device in qemu 1.1 to go higher than 31)

 

[dups]

The main reason of fixing the number of devices, is that now proxmox use fixed pci slot for devices, so we have 31 pci slots available.(X vitio-disk, X vitio_net,ide,videocard,scsi controllers,usb controller,....)
with proxmox 1.X, pci slots was not fixed, so sometimes this cause bug in windows (network address change by example),disk order bug, and some other bugs.

This 31 pci slots limits sould be no more a limit soon. (they are a new pci bridge device in qemu 1.1 to go higher than 31)

Sorry to dig up an old thread but I'm running into this exact issue with Proxmox 2.1 and qemu 1.1. When should this be changed to allow more than 5 drives?

Thanks!

 

[dietmar]

When should this be changed to allow more than 5 drives?

Soon (it is already implemented).

 

[spirit]

Hi,
It's ok in pvetest repository. (16 virtio disks - 32 nics)

 

[Owi]

Hi,
It's ok in pvetest repository. (16 virtio disks - 32 nics)

This is great news, but the 16 drives and 32 nics choice makes me curious.

I personally have 20 and 24 slot chassis, so I can easily see 32 drives and 16 nics as a reasonable solution.
At what scenario would 16/32 be needed/preferred?

Is it more complicated to implement larger number of virtio disk vs nics?

 

[dietmar]

Is it more complicated to implement larger number of virtio disk vs nics?

Why do you need more that 16 drives inside a VM (that sounds strange to me)?

 

[Owi]

Why do you need more that 16 drives inside a VM (that sounds strange to me)?

It propably does, but it's not as much a requirement as a thought.

Well, I use proxmox to learn - as it is a safe place to live by trial-by-error.
One of my servers is a fileserver with about 20 drives, where I train in both linux distributions and different development languages.
This is purely for self-education.
I can work from several installations on the same files without the need to reinstall from scratch each time.
I often work with handling large files in several copies over several drives. 20 drives isnt that much these days.
Ideally this woldn't be virtualized at all, but there's to many benefits with proxmox for someone in my situation to not use it.



So this is why I can see the use of more then 16 drives, but I'm having problems seing the need for 32 nics.
Then again, my knowledge is limited and this is not meant to be critique - but rather a wish to get enlightened.

 

[dietmar]

So this is why I can see the use of more then 16 drives,

You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).

I really think this is enough.

but I'm having problems seing the need for 32 nics.

AFAIK, some people run complex firewall with up to 32 ports (at least they requested that feature).

 

[Owi]

You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).

AFAIK, some people run complex firewall with up to 32 ports (at least they requested that feature).

I hadn't thought about the firewall scenario in proxmox at those scales - again a limitation of not being in a production environment.
I manage just fine as is, but I always need to know why stuff is the way it is.

And you helped me with that today.


Thanks alot.

 

[kalleyne]

You can have 4 IDE disks, 14 SCSI disks, 16 VIRTIO disks and 6 SATA disks (= 40 disks).

Forgive me while I reopen this more than two year old thread.Are we still limited to 16 VIRTIO disks on the recent versions of Proxmox VE 3.4?

 

[dietmar]

Are we still limited to 16 VIRTIO disks on the recent versions of Proxmox VE 3.4?

Yes

 

[spirit]

technically, it could be easy to add support for virtio-scsi to 512 disks.

(2controllers with 256 disks)

 

[kalleyne]

technically, it could be easy to add support for virtio-scsi to 512 disks.(2controllers with 256 disks)

Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would function? Thanks.

 

[spirit]

>>Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would >>function? Thanks.

virtio-scsi are like sas drives, they use a controller which can handle 256 drive.

virtio-blk, use 1 controller for 1 disk, so you are limited by virtual pci slots of the qemu machines.


you can edit /usr/share/perl5/PVE/QemuServer.pm, and change MAX_SCSI_DISKS to 512.

(then restart pvedaemon service).

It should be enough.

 

[kalleyne]

>>Hello spirit.Can you provide a little more information on how this can be achieved? Can it be achieved today? I'm assuming that this would work with SAS drives because of the SCSI relationship but can you provide a few more details on how this would >>function? Thanks.virtio-scsi are like sas drives, they use a controller which can handle 256 drive.virtio-blk, use 1 controller for 1 disk, so you are limited by virtual pci slots of the qemu machines.you can edit /usr/share/perl5/PVE/QemuServer.pm, and change MAX_SCSI_DISKS to 512.(then restart pvedaemon service).It should be enough.

spirit, thank you for this useful bit of information. I'll certainly give this a try once I can conclude some PCI passthrough testing. Thanks.