USB 3 Multi-drive External HDD Enclosure - Passthrough not possible?

[theCiscoGeek]

I'm hoping to get this to work in PVE, as it's the only reason for me to use that evil V-ware hypervisor.

My current use case for this is passing through two multi-drive hdd enclosures, to two separate vm's running Minio. *Note that the examples only show a single enclosure because I'm trying to get it working with one at a time.* The USB passthrough works beautifully in VMware. However in PVE 6.4 and 7.0 Beta-1 the virtual machine complains about UAS/Streams:

[    1.852384] usb 3-1: USB controller 0000:01:1b.0 does not support streams, which are required by the UAS driver.
[    1.852819] usb 3-1: Please try an other USB controller if you wish to use UAS.

And only sees three of the four disks:

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
loop0    7:0    0 69.9M  1 loop /snap/lxd/19188
loop1    7:1    0 55.4M  1 loop /snap/core18/1944
loop2    7:2    0 31.1M  1 loop /snap/snapd/10707
loop3    7:3    0 32.3M  1 loop /snap/snapd/12398
loop4    7:4    0 55.5M  1 loop /snap/core18/2074
loop5    7:5    0 67.6M  1 loop /snap/lxd/20326
sda      8:0    0   32G  0 disk
├─sda1   8:1    0    1M  0 part
└─sda2   8:2    0   32G  0 part /
sdb      8:16   0  1.8T  0 disk
sdc      8:32   0  1.8T  0 disk
sdd      8:48   0  1.8T  0 disk

LSUSB from the guest:

/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 3: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
    |__ Port 4: Dev 2, If 0, Class=Hub, Driver=hub/8p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
    |__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M

LSUSB from the host (during passthrough):

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M
    |__ Port 4: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=usbfs, 5000M
        |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=usbfs, 5000M
        |__ Port 3: Dev 5, If 0, Class=Mass Storage, Driver=usbfs, 5000M
        |__ Port 4: Dev 6, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/7p, 480M
        |__ Port 6: Dev 4, If 0, Class=Communications, Driver=cdc_ether, 480M
        |__ Port 6: Dev 4, If 1, Class=CDC Data, Driver=cdc_ether, 480M
    |__ Port 13: Dev 5, If 0, Class=Hub, Driver=hub/4p, 480M

LSUSB from the host when VM is off:

/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 10000M
    |__ Port 4: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 1: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
        |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=uas, 5000M
        |__ Port 3: Dev 5, If 0, Class=Mass Storage, Driver=uas, 5000M
        |__ Port 4: Dev 6, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/16p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/7p, 480M
        |__ Port 6: Dev 4, If 0, Class=Communications, Driver=cdc_ether, 480M
        |__ Port 6: Dev 4, If 1, Class=CDC Data, Driver=cdc_ether, 480M
    |__ Port 13: Dev 5, If 0, Class=Hub, Driver=hub/4p, 480M

And finally, info usbhost

# info usbhost
  Bus 2, Addr 6, Port 4.4, Speed 5000 Mb/s
    Class 00: USB device 1e91:a4a7, Mercury Elite Pro Quad D
  Bus 2, Addr 5, Port 4.3, Speed 5000 Mb/s
    Class 00: USB device 1e91:a4a7, Mercury Elite Pro Quad C
  Bus 2, Addr 4, Port 4.2, Speed 5000 Mb/s
    Class 00: USB device 1e91:a4a7, Mercury Elite Pro Quad B
  Bus 2, Addr 3, Port 4.1, Speed 5000 Mb/s
    Class 00: USB device 1e91:a4a7, Mercury Elite Pro Quad A
  Bus 1, Addr 4, Port 1.6, Speed 480 Mb/s
    Class 02: USB device 04b3:4010, XClarity Controller

One work around that I know I could technically do would be to create storage on each of these at the PVE level and then mount a virtual disk on each drive to the VM. This adds a layer of abstraction that I'm trying desperately to avoid, especially since it adds complexity to swapping out a bad drive.

While I have some questions about other Proxmox features, this is currently the only blocker for me moving away from VMware for good. Is there any tips/tricks to passing this through a different way? The device ID's are the same for every drive on both enclosures, so I didn't think that would be an option.

Any help is most appreciated!

 

[Alloc]

I'm having the same issue here (USB3 SSD hosting an OS installation, getting the error about the controller not supporting streams when assigning the device to a VM). Were you able to somehow fix this?

 

[jeroenrnl]

I had the same issue. But it worked before on this system. When I opened the USB configuration for the passthrough, USB3 was not ticked. I ticked it, and it showed it would change the configuration from `USB3=yes` to `USB3=1`. I had to shut down and restart the VM in question to get the config to activate, but after that, my USB3 drive would connect... Hope this helps you as well.

 

[Alloc]

Thanks for the info. This is a different situation though. In my case at least USB3 works, it seems like the virtual controller does not support a specific USB3 feature though. Most likely making the connection slower (?), at least it feels that way and would make some sense with that feature missing.