Passed through USB3 HDD doesn't work anymore after restart: "usb 3-1: Not enough host controller resources for new device state."

[Istria]

Hello,

Introduction:
I have been using Proxmox with a couple of VMs and LXC containers. One of the VMs is my NAS (OpenMediaVault 5).

Hardware:
- HP T630 Thin Client
- USB 3.0 HDD Enclosure (Lacie) with separate power supply.

In Proxmox, I passed through the USB device to the OpenMediaVault VM. And that worked fine.

The problem:
Yesterday, I switched to a different ISP and got a new modem/router. The old modem/router worked on subnet 192.168.178.x. The new one is on 192.168.2.x. This was not configurable, so I had to change all my VMs and LXC containers to this new subnet.

After I did that, I restarted Proxmox. Then I noticed my NAS HDD was no longer accessible. After some investigation, I figurerd out the USB HDD was not recognized by OpenMediaVault. Only the (virtual) drive OMV is installed on was visible. Multiple restarts of both the OMV VM and entire Proxmox did not solve this issue. Also replugging the USB cable did not fix it. Plugging the drive into a USB2 port does work, but because of the speed bottleneck, not a viable solution.

Errors I found:
It seems to be a similar problem to these topics I found. But most of these people had this issue when using lots of USB devices. In my case, there are only 2 USB devices in total connected to the whole machine. And the USB HDD is the only one on the USB3 port.

https://forum.proxmox.com/threads/u...devices-this-xhci-host-supports-is-32.100040/

https://superuser.com/questions/731751/not-enough-host-controller-resources-for-new-device-state

https://bugzilla.redhat.com/show_bug.cgi?id=1376579

https://github.com/pimox/pimox7/issues/48

The most obvious error I found was when doing 'dmesg':

[ 5.348994] usb 3-1: New USB device found, idVendor=059f, idProduct=106f, bcdDevice= 0.01 [ 5.349007] usb 3-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1 [ 5.349012] usb 3-1: Product: P9233 [ 5.349016] usb 3-1: Manufacturer: LaCie [ 5.349019] usb 3-1: SerialNumber: 0000NL32CJT4 [ 5.381179] usbcore: registered new interface driver usbhid [ 5.381188] usbhid: USB HID core driver [ 5.392885] usbcore: registered new interface driver usb-storage [ 5.418424] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input4 [ 5.423083] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:01.2-1/input0 [ 5.445671] usb 3-1: Not enough host controller resources for new device state. [ 5.446229] xhci_hcd 0000:01:1b.0: xHCI xhci_drop_endpoint called with disabled ep (____ptrval____) [ 5.446238] xhci_hcd 0000:01:1b.0: xHCI xhci_drop_endpoint called with disabled ep (____ptrval____) [ 5.446242] xhci_hcd 0000:01:1b.0: xHCI xhci_drop_endpoint called with disabled ep (____ptrval____) [ 5.446274] xhci_hcd 0000:01:1b.0: Trying to add endpoint 0x81 without dropping it. [ 5.446281] usb 3-1: Not enough bandwidth for altsetting 0 [ 5.480245] uas: probe of 3-1:1.0 failed with error -12[/B] [ 5.480406] usbcore: registered new interface driver uas

The entire 'dmesg' output is attached to this post.

When I do 'lsusb', the 'LaCie' USB HDD Enclosure is found:
root@omv:~# root@omv:~# lsusb Bus 003 Device 002: ID 059f:106f LaCie, Ltd P9233 Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU USB Tablet Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

When I do 'lsblk', the HDD is NOT found:
root@omv:~# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 8G 0 disk ├─sda1 8:1 0 7G 0 part / ├─sda2 8:2 0 1K 0 part └─sda5 8:5 0 975M 0 part [SWAP] sr0 11:0 1 1024M 0 rom

Perhaps also relevant to mention: When the USB device is NOT passed through to any VM, in Proxmox under "Disks", I do see the HDD as 'sdc' (sda and sdb are internal SSDs used for 'local' and 'data'. When I pass the USB device through, 'sdc' disappears from this list (which makes sense).

What I tried:
- Restarting proxmox multiple times (did not fix).
- Remove the USB passthrough and shutdown the old OMV VM and make a new VM and install a whole new instance of (newer version) OMV6 (I had OMV5 running before). Passthrough the USB device to the new VM and new OMV6: Same problem.
- Made a new VM and installed TrueNAS: Same problem.
- Pass through the PCI XHCI controller instead of only the USB device. But then even under 'lsusb' it doesn't show up.
- Plug the USBHDD into a USB2 port and pass that through: That does work. The drive shows up in the OMV VM. But of course, too slow because of the USB2 bottleneck.

Now what?
So I've exhausted my know-how and googling abilities to fix this. The only thing remaining for me is to reinstall the entire Proxmox environment and start from scratch. But before I do that, I thought to make this topic and hope someone can point me in the right direction.

Summerized / TLDR:
- I had OMV with external USB3.0 HDD passed through working fine for months.
- After changing IP-adresses of VMs and LXCs and restarting Proxmox, the passed through HDD no longer shows up in OMV or under 'lsblk' (it does under 'lsusb').
- In the output of 'dmesg', I get the error: "usb 3-1: Not enough host controller resources for new device state."

Thanks in advance for the help! Let me know if you need more info.

 

[Istria]

Small bumb, since it's been a couple days. Anyone have a clue why I'm suddenly getting this error? Any help would be greatly appreciated!

In the meantime, I plugged it into a USB2.0 port, which works. But I'd really like the USB3.0 speed back. =)

 

[fweber]

Hi, do I understand correctly that the USB3 passthrough used to work before you had to change the IP addresses? If yes, that's strange, I don't currently see how those could be related.

To troubleshoot this further, could you post the VM configuration for the attemped USB3 passthrough case, i.e., the output of qm config <your-vmid>?

 

[Istria]

Thanks for your help.
Indeed. It has worked fine for months passed through with USB3 (verified working by transfer speed of 80-100MB/s). Until the restart after changıng the IP addresses. I also don't see how those could be related. I just thought I'd mention it since its the only thing I changed.

Here is the output of the requested command. I reverted to the old situation first, having the drive plugged into a USB3 port. After that I removed the passthrough and added it again. And restarted the VM. Then I did the "qm config 104":

```
root@proxmox:~# qm config 104
agent: 1
boot: order=scsi0;ide2;net0
cores: 2
cpu: host
ide2: none,media=cdrom
memory: 1024
meta: creation-qemu=7.1.0,ctime=1676411911
name: OMV6
net0: virtio=26:ADD:B1:02:67,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: data:vm-104-disk-0,discard=on,iothread=1,size=8
G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=0652ca9d-8075-4afe-9b1b-ebf6c8263cfa
sockets: 1
startup: order=1,up=60
usb0: host=059f:106f
vmgenid: 3cdc211c-d1a0-48b5-9bda-4ed58adf16d1
root@proxmox:~#
```

P.S., I also tried adding ",usb3=1" manually (before when I was trying to solve it). That also had no effect. When adding the USB passthrough, the "use USB3" checkbox was greyed out, but checked.
I also tried passing through the port instead of the device, but that also resulted in the same errors in 'dmesg'.

 

[fweber]

Hi, thanks for the config, it looks okay to me (just FYI, you can also use [CODE]/[/CODE] tags).
I did find a related thread [1] where the solution was to disable uas in the guest VM, maybe you could give this a try?

[1] https://forum.proxmox.com/threads/u...but-hard-disk-is-not-listed.56374/post-259861

 

[Istria]

Yay! Thanks a lot!

I came across that thread as well, but the commands didn't work for me at the time. I thought it was maybe a debian/Ubuntu difference, so I continued my search. Should've spent a bit more time trying to make it work.

I tried it now using commands I found here:
https://unix.stackexchange.com/questions/441668/debian-usb3-hdd-uas-i-o-errors

echo options usb-storage quirks=174c:55aa:u | tee /etc/modprobe.d/blacklist-uas.conf update-initramfs -u reboot

That worked. The drive is now working on the USB3 port. Thanks a lot! =)

I also tried reverting the change, and then I get the error again. So UAS enabled = error. UAS disabled = working.

But what is the consequence of this change? Will I "miss" having UAS enabled somehow? And what could be the reason it worked before with UAS enabled (I never disabled it before when it was working)?

 

[fweber]

Cool, glad to hear you got it working!

Regarding negative consequences of disabling UAS: Honestly, I don't know. As far as I understand it, UAS *should* provide better performance, but this was obviously not the case for you.

And I'm not sure why it worked before either -- maybe there was a guest kernel update which only came into effect after rebooting?

 

[Istria]

I did some testing now. And although the drive works on the USB3 port, its transfer speed is severely compromised compared to before. It seems like it's working on USB2 speeds.

Before, I would transfer files from my Windows laptop via SMB (wired) to the drive with 80 MB/s average.
Now, it averages only 20-25MB/s.

I also tried copying a file from the USB hdd to a different directory on the same USB hdd via NFS through one of the Debian LXC containers I'm running on the same proxmox host:
rsync -ah --progress /mnt/MoviesSeriesOMV1/Movies/MoviesPublic/'Armageddon (1998).mkv' /mnt/MoviesSeriesOMV1
This also averages around 20 MB/s.

So although it works on the USB3 port, unfortunately, I don't get speeds past what I got on the USB2 port. I guess that UAS thing is quite important in using the USB3 speeds.

So this is progress, but not quite there yet.

 

[fweber]

That's unfortunate. Could you please show the new dmesg output from the guest? Also, it might be interesting to inspect the VM commandline, which you can print using qm showcmd <vmid> --pretty (on the host).

 

[Istria]

Here is the output of showcmd:

root@proxmox:~# qm showcmd 104 --pretty /usr/bin/kvm \ -id 104 \ -name 'OMV6,debug-threads=on' \ -no-shutdown \ -chardev 'socket,id=qmp,path=/var/run/qemu-server/10 4.qmp,server=on,wait=off' \ -mon 'chardev=qmp,mode=control' \ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd .sock,reconnect=5' \ -mon 'chardev=qmp-event,mode=control' \ -pidfile /var/run/qemu-server/104.pid \ -daemonize \ -smbios 'type=1,uuid=0652ca9d-8075-4afe-9b1b-ebf6c82 63cfa' \ -smp '2,sockets=1,cores=2,maxcpus=2' \ -nodefaults \ -boot 'menu=on,strict=on,reboot-timeout=1000,splash= /usr/share/qemu-server/bootsplash.jpg' \ -vnc 'unix:/var/run/qemu-server/104.vnc,password=on' \ -cpu host,+kvm_pv_eoi,+kvm_pv_unhalt \ -m 1024 \ -object 'iothread,id=iothread-virtioscsi0' \ -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0, addr=0x1e' \ -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0, addr=0x1f' \ -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0, addr=0x5' \ -device 'vmgenid,guid=3cdc211c-d1a0-48b5-9bda-4ed58a df16d1' \ -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0 x2' \ -device 'qemu-xhci,p2=15,p3=15,id=xhci,bus=pci.1,add r=0x1b' \ -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \ -device 'usb-host,bus=xhci.0,port=1,vendorid=0x059f, productid=0x106f,id=usb0' \ -device 'VGA,id=vga,bus=pci.0,addr=0x2' \ -chardev 'socket,path=/var/run/qemu-server/104.qga,s erver=on,wait=off,id=qga0' \ -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \ -device 'virtserialport,chardev=qga0,name=org.qemu.g uest_agent.0' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,ad dr=0x3,free-page-reporting=on' \ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:3bb 98ba7c23' \ -drive 'if=none,id=drive-ide2,media=cdrom,aio=io_uri ng' \ -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id =ide2,bootindex=101' \ -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,ad dr=0x1,iothread=iothread-virtioscsi0' \ -drive 'file=/dev/data/vm-104-disk-0,if=none,id=driv e-scsi0,discard=on,format=raw,cache=none,aio=io_uring, detect-zeroes=unmap' \ -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id =0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bo otindex=100' \ -netdev 'type=tap,id=net0,ifname=tap104i0,script=/va r/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu- server/pve-bridgedown,vhost=on' \ -device 'virtio-net-pci,mac=26:AD:DD:B1:02:67,netdev =net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,t x_queue_si ze=1024,bootindex=102' \ -machine 'type=pc+pve0' root@proxmox:~#

The dmesg output I attached in a txt file, since it's very large.

Thanks again for the help.

 

[fweber]

Thanks! I don't see anything wrong with the command. I'm actually starting to suspect this may be more of a USB hardware issue, and not related to virtualization.

As you already noted, the original error Not enough host controller resources for new device state. hints at too many USB(3) devices being in use. You wrote that you have only 2 USB devices plugged in, which is not a lot, but maybe there are some additional internal USB devices (e.g. Ethernet?). Could you check using lsusb -t on the host? Also, could you try mounting the HDD on the host (with UAS enabled) and checking the speed?

 

[Istria]

Here is the output of lsusb -t:

root@proxmox:/mnt/testmnt/OPENMEDIAVAULT/Media/MoviesS eries/Movies/MoviesPublic/Prometheus (2012)# lsusb -t /: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_ hcd/4p, 5000M |__ Port 1: Dev 2, If 0, Class=Mass Storage, Drive r=uas, 5000M /: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_ hcd/4p, 480M /: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci- pci/2p, 480M |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M |__ Port 3: Dev 3, If 0, Class=Vendor Specific Class, Driver=usbfs, 12M

After shutting down the NAS VM, I mounted the drive on the proxmox host (mount /dev/sdc1 /mnt/testmnt/) and copied a 2GB file using 'rsync -h --progress' from one directory to the other on the same mount.
The average speed after transfer was 40MB/s. But it was very jumpy, starting at +-150MB/s (I assume first filling up some cache), then dropping to as low as 10MB/s and after that yoyoing up and down between 15 and 60-70 MB/s until the transfer finished.

Now, I must mention that the (EXT4 partitioned) drive is 95% full. So maybe it has to go all over the place to write, making it more random write than sequential. It just came to me this might be the reason. But it is still double the (average) speed compared to doing the same test in the VM with UAS disabled.

I will try to make some free space later today to see if that makes any difference.

 

[fweber]

Thanks for checking! So do I understand correctly that -- on the host -- you get USB3 speeds at least some of the time?

I spoke with a colleague who had some more ideas. As I learned, from qemu-server>=7.2-7 on, PVE switched [1] from nec-usb-xhci to qemu-xhci for QEMU USB adapters, which the guest is using now according to your showcmd output:

-device 'qemu-xhci,p2=15,p3=15,id=xhci,bus=pci.1,add r=0x1b'

A hypothesis: When USB3 worked properly in the guest, you might have used nec-usb-xhci. Some time after, you might have updated to qemu-server>=7.2-7, however, the switch to qemu-xhci did not come into effect immediately, but only after a reboot, and it could be the case that qemu-xhci does not play well with USB3/your USB3 controller.

Hence, you could try the following:

1. In the guest, revert the uas blacklisting workaround (remove the line from /etc/modprobe.d/blacklist-uas.conf, run update-initramfs -u)
2. In PVE, change the VM machine version to 7.0. You can change the machine version in Hardware->Machine (check Advanced first). With this, PVE should revert to using nec-usb-xhci.
3. Start the VM, check if you can mount the HDD in the guest now

If this doesn't help, could you post again the output of qm config <vmid> and qm showcmd --pretty <vmid>, and also of pveversion -v, so we can check the qemu-server version?

[1] https://bugzilla.proxmox.com/show_bug.cgi?id=4324

 

[Istria]

Hi,

Thanks for the effort. Very much appreciated. Unfortunately no joy.

I reverted the uas blacklisting and did the update-initramfs -u.
After that I shut down the container and changed the Hardware>Machine>Version to 7.0.
Then I started the VM again, but unfortunately, the drive does not show in the 'lsblk' output in the guest VM. Then I tried to restart the proxmox host as well, but no change.

Here is the output of 'qm config 104':
root@proxmox:~# qm config 104 agent: 1 boot: order=scsi0;ide2;net0 cores: 2 cpu: host ide2: none,media=cdrom machine: pc-i440fx-7.0 memory: 1024 meta: creation-qemu=7.1.0,ctime=1676411911 name: OMV6 net0: virtio=26:AD:DD:B1:02:67,bridge=vmbr0,firewall=1 numa: 0 onboot: 1 ostype: l26 scsi0: data:vm-104-disk-0,discard=on,iothread=1,size=8G,ssd=1 scsihw: virtio-scsi-single smbios1: uuid=0652ca9d-8075-4afe-9b1b-ebf6c8263cfa sockets: 1 startup: order=1,up=60 usb0: host=059f:106f vmgenid: 3cdc211c-d1a0-48b5-9bda-4ed58adf16d1

Output of 'qm showcmd 104 --pretty':
root@proxmox:~# qm showcmd 104 --pretty /usr/bin/kvm \ -id 104 \ -name 'OMV6,debug-threads=on' \ -no-shutdown \ -chardev 'socket,id=qmp,path=/var/run/qemu-server/104.qmp,server=on,wait=off' \ -mon 'chardev=qmp,mode=control' \ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ -mon 'chardev=qmp-event,mode=control' \ -pidfile /var/run/qemu-server/104.pid \ -daemonize \ -smbios 'type=1,uuid=0652ca9d-8075-4afe-9b1b-ebf6c8263cfa' \ -smp '2,sockets=1,cores=2,maxcpus=2' \ -nodefaults \ -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ -vnc 'unix:/var/run/qemu-server/104.vnc,password=on' \ -cpu host,+kvm_pv_eoi,+kvm_pv_unhalt \ -m 1024 \ -object 'iothread,id=iothread-virtioscsi0' \ -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \ -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \ -device 'pci-bridge,id=pci.3,chassis_nr=3,bus=pci.0,addr=0x5' \ -device 'vmgenid,guid=3cdc211c-d1a0-48b5-9bda-4ed58adf16d1' \ -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \ -readconfig /usr/share/qemu-server/pve-usb.cfg \ -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \ -device 'usb-host,vendorid=0x059f,productid=0x106f,id=usb0' \ -device 'VGA,id=vga,bus=pci.0,addr=0x2' \ -chardev 'socket,path=/var/run/qemu-server/104.qga,server=on,wait=off,id=qga0' \ -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \ -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:3bb98ba7c23' \ -drive 'if=none,id=drive-ide2,media=cdrom,aio=io_uring' \ -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=101' \ -device 'virtio-scsi-pci,id=virtioscsi0,bus=pci.3,addr=0x1,iothread=iothread-virtioscsi0' \ -drive 'file=/dev/data/vm-104-disk-0,if=none,id=drive-scsi0,discard=on,format=raw,cache=none,aio=io_uring,detect-zeroes=unmap' \ -device 'scsi-hd,bus=virtioscsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,rotation_rate=1,bootindex=100' \ -netdev 'type=tap,id=net0,ifname=tap104i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \ -device 'virtio-net-pci,mac=26:AD:DD:B1:02:67,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=102' \ -machine 'type=pc-i440fx-7.0+pve0' root@proxmox:~#

And the output of 'pveversion -v':
root@proxmox:~# pveversion -v proxmox-ve: 7.3-1 (running kernel: 5.15.85-1-pve) pve-manager: 7.3-6 (running version: 7.3-6/723bb6ec) pve-kernel-helper: 7.3-4 pve-kernel-5.15: 7.3-2 pve-kernel-5.15.85-1-pve: 5.15.85-1 pve-kernel-5.15.83-1-pve: 5.15.83-1 pve-kernel-5.15.39-4-pve: 5.15.39-4 pve-kernel-5.15.30-2-pve: 5.15.30-3 ceph-fuse: 15.2.16-pve1 corosync: 3.1.7-pve1 criu: 3.15-1+pve-1 glusterfs-client: 9.2-1 ifupdown2: 3.1.0-1+pmx3 ksm-control-daemon: 1.4-1 libjs-extjs: 7.0.0-1 libknet1: 1.24-pve2 libproxmox-acme-perl: 1.4.3 libproxmox-backup-qemu0: 1.3.1-1 libpve-access-control: 7.3-1 libpve-apiclient-perl: 3.2-1 libpve-common-perl: 7.3-2 libpve-guest-common-perl: 4.2-3 libpve-http-server-perl: 4.1-5 libpve-storage-perl: 7.3-2 libspice-server1: 0.14.3-2.1 lvm2: 2.03.11-2.1 lxc-pve: 5.0.2-1 lxcfs: 5.0.3-pve1 novnc-pve: 1.3.0-3 proxmox-backup-client: 2.3.3-1 proxmox-backup-file-restore: 2.3.3-1 proxmox-mail-forward: 0.1.1-1 proxmox-mini-journalreader: 1.3-1 proxmox-offline-mirror-helper: 0.5.1-1 proxmox-widget-toolkit: 3.5.5 pve-cluster: 7.3-2 pve-container: 4.4-2 pve-docs: 7.3-1 pve-edk2-firmware: 3.20220526-1 pve-firewall: 4.2-7 pve-firmware: 3.6-3 pve-ha-manager: 3.5.1 pve-i18n: 2.8-2 pve-qemu-kvm: 7.1.0-4 pve-xtermjs: 4.16.0-1 qemu-server: 7.3-3 smartmontools: 7.2-pve3 spiceterm: 3.2-2 swtpm: 0.8.0~bpo11+2 vncterm: 1.7-1 zfsutils-linux: 2.1.9-pve1 root@proxmox:~#

New dmesg ouput is attached below.

 

[Istria]

Just after posting the above post, I noticed that when set to v7.0, the "USB3" checkbox under USB Passthrough is no longer greyed out. And it was not checked.
After checking it and restarting the VM, the drive shows up and can be mounted in the guest VM. =) =) =)

I really have to go to bed now (early morning tomorrow), so I will do a speedtest tomorrow afternoon. But attached is the dmesg output after enabling "use USB3" checkbox.

This part caught my eye, but I'll have to look into it with more detail tomorrow:
[ 3.633391] usb 3-1: USB controller 0000:02:1b.0 does not support streams, wh ich are required by the UAS driver. [ 3.633403] usb 3-1: Please try an other USB controller if you wish to use UA S.

EDIT: Couldn't help myself. Did a speedtest via NFS (rsync) from a guest LXC. Speed bouncing between 10 and 25 MB/s.
I'll fiddle with it more tomorrow. But we are making progress. =)

 

[fweber]

Glad to hear there is some progress But if I understand correctly, even though the VM now at least recognizes the HDD without blacklisting uas, the speed is still subpar, right?

To troubleshoot this further, it would be great if we could get logs from a VM boot when it was able to get to USB3 speed.

First of all, do I understand correctly that VM 104 is the newly created OMV6 VM you mentioned in the first post (according to its ctime, it was created very recently)?

Do you still have the old VM around? If yes, could you post its config? Also, could you try extracting a journal of a boot which got USB3 speeds? You can use (inside the VM) journalctl --list-boots to list boot numbers, pick one which is sufficiently old (e.g. -37), and use journalctl -b -37 to print the respective journal. Also, it would be interesting to check /var/log/apt/history.log on the host and the guest, to see what upgrades were installed between the last boot of the VM which got USB3 speeds, and the first boot at which it stopped working.

 

[ZKallo]

Hi,
I have had the same problem.
I upgraded the nodes from version 7.1 to 7.3 because the Windows updates were stuck every time. Almost all of the Windows vm-s' type of machines is pci-i440fx-5.1. (I don't know when and why should I change the version number of the machine.) The vm didn't recognized the USB3 disk was used to backup before. There was a warning at the USB controller that it could not start. At one point of the workaround I changed the version of the type of the machine to 7.1. The result was the same. I followed Friedrich's guide and I changed it to 7.0 and the solution was great. I could set the type of the USB to USB3 and the disk is working now at speed 70-90MB/sec as before.

 

[fiona]

Hi,

Hi,
I have had the same problem.
I upgraded the nodes from version 7.1 to 7.3 because the Windows updates were stuck every time. Almost all of the Windows vm-s' type of machines is pci-i440fx-5.1. (I don't know when and why should I change the version number of the machine.)

what is the the version of pve-qemu-kvm before and after the upgrade (you can check var/log/apt/history.log). Windows is rather sensible to machine version changes, so usually, you shouldn't change it at all.

The vm didn't recognized the USB3 disk was used to backup before. There was a warning at the USB controller that it could not start.

Can you share the exact warning?

At one point of the workaround I changed the version of the type of the machine to 7.1. The result was the same. I followed Friedrich's guide and I changed it to 7.0 and the solution was great. I could set the type of the USB to USB3 and the disk is working now at speed 70-90MB/sec as before.

Happy to hear that you found a workaround But it's strange that it stopped working after the upgrade. If you'd like to debug the issue, maybe you can create a clone of the VM to test further. It would be interesting to know what the latest version of pve-qemu-kvm was, where it worked with pc-i440fx-5.1. You can use e.g. apt install pve-qemu-kvm=7.1.0-4 and similar to test different versions.

 

[ZKallo]

Hi,

what is the the version of pve-qemu-kvm before and after the upgrade (you can check var/log/apt/history.log). Windows is rather sensible to machine version changes, so usually, you shouldn't change it at all.

pve-qemu-kvm:amd64 - from 6.1.1-1 to 7.1.0-4. This upgrade solved the Windows Update restart hanging problem.

Can you share the exact warning?

USB device warning. It was interesting that after a disabling-enabling operation the warning sign was not there at the USB device and the disk appeared as drive in the operating system for a short time. Then the sign appeared again and the disk disappeared from the drives.

Happy to hear that you found a workaround But it's strange that it stopped working after the upgrade. If you'd like to debug the issue, maybe you can create a clone of the VM to test further. It would be interesting to know what the latest version of pve-qemu-kvm was, where it worked with pc-i440fx-5.1. You can use e.g. apt install pve-qemu-kvm=7.1.0-4 and similar to test different versions.

I think the version of pve-qemu-kvm was 6.1.1-1 at the time of pc-i440fx-5.1 or a previous version when the USB drive worked normally. Unfortunately the USB docking station for SATA drive went wrong just at the time of the upgrade of PVE7.1->7.3 that's why I'm not sure what and when worked correctly.
The solution was setting the pc-i440fx version to 7.0 instead of 7.1 but I don't know it was the cause that the mode of the network interface of the guest vm changed from fix ip address to dhcp or an earlier Windows upgrade was the reason.