Latest Update 5.13.19-4-pve broke my QEMU PCIe Sharing. Works with 5.13.19-3

I’m guessing this is going to get fixed soon as it’s a huge problem, right? :D
 
I haven't seen a post of a staff member in this thread, so I don't know if they are aware of this problem yet...
I pinged @oguz before, not sure if there's someone else we should ping. Hopefully soon. Thanks!
 
I pinged @oguz before, not sure if there's someone else we should ping. Hopefully soon. Thanks!
I haven't seen a post of a staff member in this thread, so I don't know if they are aware of this problem yet...

FWIW, we're aware of it and will look more closely into it in the next work days, for now please just boot an older, previously working kernel, that's always a valid workaround. FYI: The problematic kernel is not yet on the enterprise repositories, so production workloads shouldn't be affected by this.
 
Last edited:
I'm having the same issue with a Intel 82574L network card connected via PCIe. I'm passing a SAS controller (LSI clone) to the same virtual machine. My guess is that might fail to, but it never get's that far as the attempt to bind the network card fails. Kernel version 5.13.19-3 wouldn't boot on my system at all, but I'm not sure if that is related.

I'm using an Asus ROG STRIX B350-F GAMING motherboard with a Ryzen 2700 CPU if that matters.
 
FWIW, we're aware of it and will look more closely into it in the next work days, for now please just boot an older, previously working kernel, that's always a valid workaround. FYI: The problematic kernel is not yet on the enterprise repositories, so production workloads shouldn't be affected by this.
Awesome! Thanks for your great work :)
 
Also confirmed - 5.13.19-4-pve broke my system as well - get extremely hugh IO delays and systems hangs HARD! Can't even perform proper shutdown through KVM - console totally frozen!! Went back to previous kernel - all is well . . .

Same issue for me. It throws an error to the serial console. See below starting at 38.667995:
 

Attachments

  • KernelBug.txt
    8 KB · Views: 13
FWIW, we're aware of it and will look more closely into it in the next work days, for now please just boot an older, previously working kernel, that's always a valid workaround. FYI: The problematic kernel is not yet on the enterprise repositories, so production workloads shouldn't be affected by this.
Good to hear. Thank you.
 
Same here - problem occurs only with newest kernel 5.13.19-4-pve, switching back to 5.13.19-3-pve works fine.
Sometimes pve comes up, but virtual machines cannot be started

For the ones running a headless system without the possibility to select a menu entry during boot via Grub, here a workaround to boot the former kernel, which runs fine for me:
- open file /etc/default/grub with editor (e.g. with nano or vi)
- change entry GRUB_DEFAULT=0 to GRUB_DEFAULT="1>2", which means select second main menu entry and then 3rd submenu entry (which is the former kernel 5.13.19-3-pve)
- save file
- run "update-grub" from terminal
- reboot
=> system starts up fine again, everything working as expected :)
 
Last edited:
Had the same issue with USB-passthrough not working.
Went with the 5.15 kernel and now its back to working again.
 
Same here - problem occurs only with newest kernel 5.13.19-4-pve, switching back to 5.13.19-3-pve works fine.
Sometimes pve comes up, but virtual machines cannot be started

For the ones running a headless system without the possibility to select a menu entry during boot via Grub, here a workaround to boot the former kernel, which runs fine for me:
- open file /etc/default/grub with editor (e.g. with nano or vi)
- change entry GRUB_DEFAULT=0 to GRUB_DEFAULT="1>2", which means select second main menu entry and then 3rd submenu entry (which is the former kernel 5.13.19-3-pve)
- save file
- run "update-grub" from terminal
- reboot
=> system starts up fine again, everything working as expected :)
Perfect. I solved my problem with the solution provided by Struppie
 
Maybe the problem here is not the passthrough but getting the kernel to release the controller and/or drives? Reports about that have popped up.
Have people here tried blacklisting the drivers and early binding to vfio-pci? This work-around might not be possible if the system has multiple controllers of the same type.
 
Last edited:
Maybe the problem here is not the passthrough but getting the kernel to release the controller and/or drives? Reports about that have popped up.
The latter is only block device related, that can affect HBA's or disk pass-through, but not general PCI pass through, and it's also only a problem on 5.13 FWICT, as the back ported patch relied on behavior added only in later kernels.

In general (not meant to the specific post I'm replying):
FWICT, there are at least two distinct issues, if not even more, and in addition to that there are two major kernel versions (5.13 and 5.15) and people start mixing that all up which isn't really helpful in determining and tackling the different underlying issues. It'd be great if people would refrain from "me too" (without very specific setups about HW, use case, VM config, ...) and piggybacking on other threads just because something isn't working there either, that just adds noise making it harder to filter/pinpoint the actual problems.
 
  • Like
Reactions: Feni
Seems that some devices can be passed through, others not so much. But actually I'm not sure it's the passthrough that is the problem... Because on the 5.13.19-4-pve I couldn’t run lsusb....
(and I could pass a GPU, but not my USB controller)...

I came straight from 6.4, så no 5.13.19-3-pve kernel to roll back to.... So now I'm running on 5.15, fingers crossed I don't run in to major issues???
 
Last edited:
Seems that some devices can be passed through, others not so much. But actually I'm not sure it's the passthrough that is the problem...

I posted in a separate thread, thinking my issue was unrelated. However, it seems it may be. The kernel messages in the bug report, refereced in post 10, are somewhat similar to mine.
Mine occurs when attempting to disconnect sata hotplug drives on the host.
I'm not currently passing any hardware devices to QEMU guests, though I do have IOMMU enabled in BIOS and in PVE.
 
Last edited:
  • Like
Reactions: leesteken
FYI I fixed the issue by updating to the latest kernel:
Code:
apt update
apt install pve-kernel-5.15
Hello Matt, How did you install pve-kernel-5.15? When I tried those commands it couldn't find the kernel version.
 
Had the same issue with USB-passthrough not working.
Went with the 5.15 kernel and now its back to working again.
Hello,
Any steps or instructions to update to 5.15?

Below steps doesn't work for me, as the 'apt' is not able to find the kernel version.

apt update
apt install pve-kernel-5.15
 
  • Like
Reactions: aabbccdd

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!