Increase maximum USB devices in vm.conf

K

Kevin Latimer

Member
Feb 11, 2016
5
0
21
United Kingdom
Hi everyone,

I've been trying to pass a number of USB devices through to a guest KVM and couldn't for the life of me understand why one of them wasn't getting through until I read that that the vm.conf file supports usb0 to usb4 only, while I was using all the way up to usb7...

Is this a limitation in KVM/QEMU or has this number of USB devices not really been considered a typical use case for passthrough and no reason to go past 5 assignments in the conf? I'm leaning towards the former, as I can see the QEMU USB root hub in the VM looks like it is capped at 6 ports (ehci-pci/6p).

Is there a workaround anyone can think of? My devices all sit behind a USB hub and the controller is dedicated to them so I have no issues passing the lot through in their entirety. I don't have VT-d so can't pass the PCI controller through though.

Is it possible to pass the hub through as a single device and if so, how would you specify that in vm.conf? I've tried combinations like "usb1: host=1-2" and "usb1: host=1-2.0" to no success.

While I've only had a cursory glance at the QEMU docs, I wonder if it's possible to either create a 12 port hub, or 2 6-port hubs? If so, anyone know how this might work in the PVE world?

Thanks for any help,

Kev
 
hi,
i tried to test this (because i found no real answers in the qemu/kvm docs), but
i have not enough usb devices here

i will definitely try it next week.

a workaround is to passthrough your usb hub, this should work (could not test this, as i do not have one here)

i hope this helps
 
Hi Dominik,

Thanks for the reply and apologies it's taken me so long to respond.

What would the correct way to pass the USB hub through? It appears as follows:

Code: 
root@pve:~# lsusb -t
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/8p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
    |__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/7p, 480M

The last hub is the one I want to pass through. I tried using "usb0: host=1-2" and "usb0: host=1-2.0" in the vm.conf but neither seemed to pass the hub through.

Cheers,

Kev
 
hi,

sorry for the delay.
I tested a little around and found the following:
i also could not passthrough a usb-hub (which is strange, i will look into this further)
as for the maximum number of devices, i have to do a little more testing, since the
documentation of qemu/kvm usb passthrough is not very thorough

a possible workaround for you is maybe this:
https://pve.proxmox.com/wiki/USB_Devices_in_Virtual_Machines#Assign_Devices_to_an_already_Running_VM

you can add usb devices to your vm at runtime
caution: you have to remove them again before shutting down the guest, because otherwise kvm does not
return them to the host (or you could replug them)

hope this helps
Dominik
 
Also interested in this, so far I have not been able to pass through a USB hub, but only single devices, which i less than optimal if you frequently plug/unplug stuff like USB sticks.
I also tried doing a PCI pass through of of the USB ports, but it fails, and the entire host reboots after a lot of garbage is printed on the screen.
 
Hi Dominik,

Again, apologies for the delay in replying...

You're right, the add at runtime works - I can add as many devices as I like. On first add, it creates an 8 port virtual hub. When I add my 6th device, it "connects" and additional hub to the first virtual hub, to which my USB devices get added to.

However, I've hit a couple of issues with this approach. Having the USB at boot-time seems to create a USB device with a consistent identifier. Adding them while running makes the VM see them as completely new devices. Because I'm using NetworkManager in this scenario, it sees them as new devices requiring configuration, which requires human intervention.

In addition, I've also found this to be somewhat less stable, with the occasional segfault in the guest. This requires a restart of the host to regain use of the USB devices. I've tried the EHCI and XHCI USB hub, neither appearing particularly stable.

So it seems that QM/KVM is happy to have more than 5 assignments (with a nested hub approach) but I think we need to figure out how to do this at boot time - can we "hack" the startup command somehow?

Cheers,

Kev
 
hi,

in general it is not hard to increase the usb device limit for our configuration,
it also should work if we would increase the limit and it should add the virtual hubs on start
what concerns me is this:
Kevin Latimer said:
In addition, I've also found this to be somewhat less stable, with the occasional segfault in the guest.
Click to expand...

We really do not want to enable more usb ports when it is not really stable,
but i will investigate and discuss this internally if i have the time
 
Hi Dominik,

Thanks for the reply, I don't think the stability issue should be a concern, I don't see an issue when the devices are configured at boot, only when hot-added. If I add 4 devices at boot, it is quite stable, but the same 4 devices when hot-added can cause a segfault, which appears linked to the EHCI module in dmesg. I think hot-adding the devices in my usage case is the issue, not the QEMU USB code :)

I can test any patches, if that helps.

Cheers,

Kev
 
Bump

Im in the same situation.
I have a 7 port USB hub that I want to pass through.
but I can only add usb0-4..
Best would be to be able to pass the whole hub.
What also is weird is that when I unplug the usb hub sometimes it gets a new port number so I have to remove and add the ports again on the GUI...

Any updates?
 
What we ended up doing was to use the qm "-args" command to add our USB devices, which allows you to specify more devices than the 5 the GUI/conf permits. For example:

qm set 100 -args "-device usb-ehci,id=ehci,addr=0x5 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.1 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.3 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.4 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.5 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.7 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.8 \"

However, this has an untended consequence like that I mentioned earlier in that once you add the 6th device, it creates a nested virtual hub behind the first one. While this shouldn't necessarily cause an issue, the unintended consequence is that virtual hub is connected at USB full speed rather than high speed, so devices 6 and higher are limited to 12 Mbps.

The first alternative we tried was to pass the whole USB root through using VT-d/IOMMU. However, of our two hardware variants, the first doesn't support IOMMU while the second has the USB and our NICs in the same IOMMU group, so while the passthrough works great for USB, we lose our networking :-(

The second method we've tried is to add a second USB controller, like this:

qm set 100 -args "-device usb-ehci,id=ehci,addr=0x5 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.1 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.3 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.4 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.5 \
-device usb-host,bus=ehci.0,hostbus=1,hostport=7.7 \
-device nec-usb-xhci,id=xhci,addr=0x6 \
-device usb-host,bus=xhci.0,hostbus=1,hostport=7.2 \
-device usb-host,bus=xhci.0,hostbus=1,hostport=7.6 \
-device usb-host,bus=xhci.0,hostbus=1,hostport=7.8 \
-device usb-host,bus=xhci.0,hostbus=1,hostport=7.9"

You'll see we've had to add the second group of devices using the XHCI controller, it's apparently possible to add more than one EHCI by using a different id but startup wasn't always reliable for us (note: in PVE 5.0 you can use the qemu-xhci USB device from QEMU 2.9). However, XHCI just hasn't proved stable for us in our use case yet, so we do need to revisit this one at a later date. We definitely see less latency in the devices though, so XHCI is the way forward, they just reset after a short time though.

As an out of the box idea, we even tried experimenting with USB/IP, adding devices to the USB/IP server in the host and creating a VHCI bus in the VM, using a dedicated bridge. Results were... well... mixed :)

Passing the entire hub through seems like the best option - it would be great for us too. However, after a lot of digging around lists and docs, I don't think it's as clear cut as that. You are reusing the same code that connects the virtual HID devices such as pointer/tablet and the like so you're actually connecting your USB devices to this same controller rather than "passing them through" to the OS. Multifunction devices can have one driver on the VM, while leaving the other device visible on the host. You can read more on QEMU USB at https://github.com/qemu/qemu/blob/master/docs/usb2.txt

I'm sure once we get back to testing, we'll find ways of working around lot of these quirks but for now for our use case, EHCI is the most stable. One of the other methods might work well for someone else though.
 
set for VM configuration:
machine: q35

and edit:
/usr/share/perl5/PVE/QemuServer.pm

find and replace 5 to 9:
my $MAX_USB_DEVICES = 9;

restart pve-manager.service or reboot host

add usb devices in VM config (sample):
usb0: host=2-1.4
usb1: host=2-1.3
usb2: host=2-1.5
usb3: host=2-1.6
usb4: host=2-1.2
usb5: host=2-1.1
usb6: host=1-1.6
usb7: host=1-1.5

It's work, but we need control /usr/share/perl5/PVE/QemuServer.pm update and replace 5 to 9 if needed
 
efeu said:
If the increasement of max usb devices is that easy, I wonder why those changes aren't included? Any reason for that?
Click to expand...
well there were reports that running a vm with many usb devices were unstable, and i did not have the time yet to run tests with more than 5 devices,
but if you can run tests (with the manual change above) and it works, you can open an enhancement request here (https://bugzilla.proxmox.com/) for this
 
dcsapak said:
well there were reports that running a vm with many usb devices were unstable, and i did not have the time yet to run tests with more than 5 devices,
but if you can run tests (with the manual change above) and it works, you can open an enhancement request here (https://bugzilla.proxmox.com/) for this
Click to expand...

I am working on testing with 6 devices and if all goes well I will open the enhancement request!
 
I need to send 12 usb drives to the server. I changed the file
/usr/share/perl5/PVE/QemuServer.pm
my $MAX_USB_DEVICES = 12;

turn off host server and start, but 1 have max 5 usb device:
2018-12-05_10-32-25.png

PVE 5.2, VM WinServer 2016

if I manually change the VM configuration file
101.conf
usb0: host=1-11
usb1: host=1-12
usb2: host=1-6
usb3: host=1-8
usb4: host=1-9
usb5: host=2-5
usb6: host=2-7

still doesn't work
 
Now I added a line to the configuration file
qemu-server/104.conf
Code: 
balloon: 600
bootdisk: scsi0
cores: 14
hotplug: usb
memory: 13500
name: Win2016Terminal
net0: virtio=32:F5:27:C0:64:30,bridge=vmbr0
numa: 0
onboot: 1
ostype: win10
scsi0: lvm-vm:103/vm-103-disk-0.raw,size=87G
scsihw: virtio-scsi-pci
shares: 29000
smbios1: uuid=2dcece8d-29c0-4a32-92ab-15244891fac4
sockets: 1
usb0: host=c1a5:0502
usb1: host=0a89:0003
usb2: host=c1a5:0502
usb3: host=c1a5:0502
usb4: host=c1a5:0502
vga: std
vmgenid: d9434c9a-f07b-4799-a615-eadbb2d21ae0
machine: q35
machine: q35
stop vm, start vm
but nothing has changed. :(
 
Did not notice that when updating Proxmox settings in the file returned back. Way to worker. I already have 7 USB working.Thanks for way!
 
Dear support.

You may have $ MAX_USB_DEVICES = 12 after upgrading to QemuServer.pm config. be
Currently, the value is 5, and it is uncomfortable to change after each update.
/usr/share/perl5/PVE/QemuServer.pm
$ MAX_USB_DEVICES = 12;

Best regards,
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back