trouble shooting problematic passthrough (audio)

[mcdull]

some windows guest has experience bsod sometimes when passing through AMD onboard audio. Whether or not having ACS enabled in kernel behaves the same.
After bsod, the VM can no longer locate the audio device even after reboot of the VM.
My VM has the primary VGA (nvidia GTX1080), audio (AMD onboard audio), usb controller (AMD onboard usb controller) passthrough.
video passthrough always works, but audio will cause bosd. And the USB will sometimes return code 10 in device manager and not usable.

I am totally out of the blue on how to debug it. any hints?

 

[leesteken]

Sometimes it helps to prevent the Proxmox host from initializing the devices, by either blacklisting the drivers or binding them to the vfio-pci driver early.
Maybe the devices don't reset properly or reliably and you need BIOS updates or newer kernel (for Proxmox) or newer drivers (for Windows).
Problems with USB and/or audio controllers are not uncommon. Have you considered using the audio device of the GPU and/or using a add-in PCIe card for USB (that is known to work with passthrough)? Knowing the brand and type of your motherboard and CPU might help narrow the search for work-arounds.
Is there any information in the Proxmox Syslog (or journaltctl -b 0) when the VM crashes?

 

[mcdull]

I have blacklisted the driver already and the vfio-pci already picking up the device.
BIOS and Proxmox are most up-to-date but uses win11 default driver.
I already have 2 gpu and 2 M2 which makes other PCIe option very limited.
My hardware lists below:-
Asrock Taichi X470
CPU: AMD 3900X
2 x GPU: 1080x1 for passthrough, 1050Ti x 1 for vGPU unlock
RAM: 4x32GB = 128GB

 

[leesteken]

Indeed X470 typically allows for passthrough of the two x16 slots (running in x8), one M.2 slot (x4 PCIe), audio device and one USB controller.

I know that for my ASRock X470 Master SLI motherboard a lot of BIOS versions break passthrough completely, some have very nice IOMMU groups, and some have terrible groups. Have you tried newer or older BIOS versions? I pass the on-board audio device to one VM and the USB controller to another, both running Ubuntu without any issues with a 2700X and BIOS 3.70, which is comparable to the 3.90 version for your board.

On my previous system, on-board audio had crackling audio problems unless I let Proxmox initialize the device and load snd_hda_intel to initialize the codecs or something. Maybe you could try this?

In principle the current setup works for PCI passthrough, except when it unexpectedly fails while running this VM. Any information in the Proxmox Syslog (or journaltctl) when the VM crashes? Maybe Windows 10 has more mature drivers for your hardware? Would it be worth to try an Ubuntu VM (or just run the Live CD)? I have no experience with Windows, sorry.

 

[mcdull]

did not check the journalctl yet, but the syslog shows nothing.
Weird thing is that after windows VM crash it cannot re-initialize the audio device even with a restart.
taichi bios is quite stable in terms of IOMMU group.
around 12 groups if no ACS enabled in BIOS
around 22 groups if ACS enabled in BIOS
around 44 groups if ACS also enabled in kernel boot parameter.

 

[leesteken]

Sounds like the audio device does not reset properly, and will never work for a second time after a shutdown of the VM. Maybe someone will add a work-around in vfio or the Linux kernel, once they find out how to make it reset completely. Please try this with starting the VM with a Ubuntu 21.10 installer/live DVD; then we might narrow it down to Windows drivers.

 

[mcdull]

It does not happens frequently, like once every 2 weeks, so unless i replace my with with ubuntu as a daily drivers, its really difficult to simulate the issue.

 

[leesteken]

It does not happens frequently, like once every 2 weeks, so unless i replace my with with ubuntu as a daily drivers, its really difficult to simulate the issue.

RIght, that's hard to pin down then. Hopefully drivers and operating systems will improve over time and the problem will diminish.

 

[mcdull]

RIght, that's hard to pin down then. Hopefully drivers and operating systems will improve over time and the problem will diminish.

I experience another fatal error just now.
I started another VM which utilize a vgpu. And the whole proxmox crashed. All containers are not working, and the VMs still running fine.
I can ssh into the proxmox but for every command I entered, it returns with #Input/output error

Anything that I can check before I force reboot the host?

 

[leesteken]

That does not sound related. Please check your power supply and memory if you have crashes that are not easy to trigger and don't appear to have a common cause.