Passthrough onboard audio and integrated gra

[CAppa2]

Hi everyone, I'm new to proxmox. I am trying to passthrough the motherboard integrated sound card to a Windows 10 VM for my 5.1 amplifier.
System run on:

• Amd Ryzen 3 2200G
• GIGABYTE B450M DS3H (latest BIOS update F60h)
• 8GB RAM DDR4

I found and followed THIS and THIS guides and I was able to passthrough a PCIe SATA card into another VM, and it works perfectly. Now ... the problem is that when I try to passthrough the sound card and start the Windows 10 VM the system reboots by itself and obviously the VM not work.
I am trying in parallel to passthrough the IGPU of the Ryzen 3 (Vega 8) at the same VM and it behaves in the same way.

For now I have always tried to passthrough either the IGPU or the sound card, never together (even if that is the final goal). In addition, the VM starts correctly if I don't passthrough anything. I also tried with a UEFI BIOS system in VM configuration, nothing changes.

I lost 2 days without coming to a solution, I don't know what to do anymore. Hope someone can help me find a solution, thanks a lot in advance.

Here are some useful dumps to understand why nothing works and if I made a mistake in the configuration

---

Windows 10 VM configuration​

agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
boot: order=scsi0;ide2;net0
cores: 1
cpu: host,hidden=1,flags=+pcid
ide2: local:iso/virtio-win-0.1.190.iso,media=cdrom,size=489986K
machine: q35
memory: 2048
name: Windows
net0: virtio=7E:57:F6:D3:44:1B,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
scsi0: local-lvm:vm-101-disk-0,size=40G
scsihw: virtio-scsi-pci
smbios1: uuid=24857be2-d336-4eff-bb34-0dbd875af7b2
sockets: 1
startup: order=2
vga: std
vmgenid: c680e2ec-5c53-4cdd-97a0-7d3b4b9ba188

[As mentioned before, only one of the two devices, never together]
hostpci0: 08:00.6,rombar=0    [sound card]
hostpci1: 08:00.0;08:00.1,pcie=1,rombar=0  [IGPU + HDMI audio]

---

Same cat of commands executed following the PCIe passthrough guides
​

Dump grub file (commented lines removed in this dump, not in the file):

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction vfio-pci.ids=1002:15d8,1002:15de,1022:15e3 nofb nomodeset video=vesafb:off,efifb:off"
GRUB_CMDLINE_LINUX=""

GRUB_DISABLE_OS_PROBER=true

GRUB_DISABLE_RECOVERY="true"

Kernel modules:

root@pve:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Dump command: dmesg | grep -e DMAR -e IOMMU -e AMD-Vi

[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.026213] DMAR: IOMMU enabled
[    0.876817] pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter.
[    0.878333] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.878333] pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada):
[    0.878336] AMD-Vi: Interrupt remapping enabled
[    0.878336] AMD-Vi: Virtual APIC enabled
[    0.878498] AMD-Vi: Lazy IO/TLB flushing enabled

root@pve:~# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1022:15e3   [Audio card mobo]
options vfio-pci ids=1002:15d8,1002:15de disable-vga=1    [AMD IGPU + HDMI Audio]

IOMMU groups:

root@pve:~#  find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/17/devices/0000:06:00.0
/sys/kernel/iommu_groups/7/devices/0000:00:14.3
/sys/kernel/iommu_groups/7/devices/0000:00:14.0
/sys/kernel/iommu_groups/25/devices/0000:08:00.6
/sys/kernel/iommu_groups/15/devices/0000:03:04.0
/sys/kernel/iommu_groups/5/devices/0000:00:08.1
/sys/kernel/iommu_groups/23/devices/0000:08:00.3
/sys/kernel/iommu_groups/13/devices/0000:03:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:01.3
/sys/kernel/iommu_groups/21/devices/0000:08:00.1
/sys/kernel/iommu_groups/11/devices/0000:02:00.1
/sys/kernel/iommu_groups/1/devices/0000:00:01.1
/sys/kernel/iommu_groups/18/devices/0000:06:00.1
/sys/kernel/iommu_groups/8/devices/0000:00:18.3
/sys/kernel/iommu_groups/8/devices/0000:00:18.1
/sys/kernel/iommu_groups/8/devices/0000:00:18.6
/sys/kernel/iommu_groups/8/devices/0000:00:18.4
/sys/kernel/iommu_groups/8/devices/0000:00:18.2
/sys/kernel/iommu_groups/8/devices/0000:00:18.0
/sys/kernel/iommu_groups/8/devices/0000:00:18.7
/sys/kernel/iommu_groups/8/devices/0000:00:18.5
/sys/kernel/iommu_groups/26/devices/0000:09:00.0
/sys/kernel/iommu_groups/16/devices/0000:05:00.0
/sys/kernel/iommu_groups/6/devices/0000:00:08.2
/sys/kernel/iommu_groups/24/devices/0000:08:00.4
/sys/kernel/iommu_groups/14/devices/0000:03:01.0
/sys/kernel/iommu_groups/4/devices/0000:00:08.0
/sys/kernel/iommu_groups/22/devices/0000:08:00.2
/sys/kernel/iommu_groups/12/devices/0000:02:00.2
/sys/kernel/iommu_groups/2/devices/0000:00:01.2
/sys/kernel/iommu_groups/20/devices/0000:08:00.0
/sys/kernel/iommu_groups/10/devices/0000:02:00.0
/sys/kernel/iommu_groups/0/devices/0000:00:01.0
/sys/kernel/iommu_groups/19/devices/0000:07:00.0
/sys/kernel/iommu_groups/9/devices/0000:01:00.0

In alternative more detailed dump PCIe devices (running lspci -v command) in the attached txt file

Thank you so much again!!

 

[leesteken]

I have not yet read anywhere on this forum or others about anyone having success with passthrough of integrated Ryzen GPU. But there are people having similar issues (on this forum and others). Maybe you can find a post somewhere that explains how to make it work?
The audio device (connected to the motherboard 3.5mm jacks) should work. Why are you using rombar=0? Have you tried without it?
You have PCIe ACS overrides enabled, which means that we cannot check the actual IOMMU groups. Maybe the audio device is not alone in its IOMMU group without the ACS override? These groups also tend to change with BIOS versions.
AMD-Vi: Unable to read/write to IOMMU perf counter indicates to me that your IOMMU setting is on Auto but you need to set it to Enabled in your motherboard BIOS settings for PCI passthrough to work.
Your BIOS version might be too new for PCI passthrough to work perfectly with your Zen1 APU. Have you tried versions F42h or F50? Earlier version are known to break PCI passthrough, but newer version can also have issues (unless you use a much newer CPU).

 

[CAppa2]

Thank you very much for your availability.
The fact that no one was able to pass the AMD IGPU does not give me hope, and it demoralizes me a little. To be honest I was hoping someone could help me with this.

Anyway i followed your directions and divided the attempts into 2 groups: before BIOS downgrade and after BIOS downgrade So here are the results:

---

​

Before BIOS downgrade​

First o all I checked the IOMMU setting in the BIOS. Actually I had set IOMMU in Enabled mode, but for safety I double checked and it is actually set in Enabled. Below the BIOS images at IOMMU and AMD-vi settings for completeness


I assume that the IOMMU setting results Auto to the system is due to some BIOS problem as you said:

Earlier version are known to break PCI passthrough, but newer version can also have issues

As a second thing I tried to removerombar=0 from the VM settings... but unfortunately the behavior is the same, the system reboots by itself as soon as I start the VM.
Then i disabled PCIe ACS overrides by removing pcie_acs_override=downstream,multifunction in the grub file, and now it is:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt vfio-pci.ids=1002:15d8,1002:15de,1022:15e3 nofb nomodeset video=vesafb:off,efifb:off"

(i only included the line i edited the rest of the file is as before)

and checked IOMMU groups

root@pve:~#  find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/7/devices/0000:00:14.3
/sys/kernel/iommu_groups/7/devices/0000:00:14.0
/sys/kernel/iommu_groups/5/devices/0000:00:08.1
/sys/kernel/iommu_groups/13/devices/0000:08:00.4
/sys/kernel/iommu_groups/13/devices/0000:08:00.2
/sys/kernel/iommu_groups/13/devices/0000:08:00.3
/sys/kernel/iommu_groups/13/devices/0000:08:00.1
/sys/kernel/iommu_groups/13/devices/0000:08:00.6
/sys/kernel/iommu_groups/3/devices/0000:00:01.3
/sys/kernel/iommu_groups/11/devices/0000:07:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:01.1
/sys/kernel/iommu_groups/8/devices/0000:00:18.3
/sys/kernel/iommu_groups/8/devices/0000:00:18.1
/sys/kernel/iommu_groups/8/devices/0000:00:18.6
/sys/kernel/iommu_groups/8/devices/0000:00:18.4
/sys/kernel/iommu_groups/8/devices/0000:00:18.2
/sys/kernel/iommu_groups/8/devices/0000:00:18.0
/sys/kernel/iommu_groups/8/devices/0000:00:18.7
/sys/kernel/iommu_groups/8/devices/0000:00:18.5
/sys/kernel/iommu_groups/6/devices/0000:00:08.2
/sys/kernel/iommu_groups/14/devices/0000:09:00.0
/sys/kernel/iommu_groups/4/devices/0000:00:08.0
/sys/kernel/iommu_groups/12/devices/0000:08:00.0
/sys/kernel/iommu_groups/2/devices/0000:00:01.2
/sys/kernel/iommu_groups/10/devices/0000:03:00.0
/sys/kernel/iommu_groups/10/devices/0000:02:00.2
/sys/kernel/iommu_groups/10/devices/0000:02:00.0
/sys/kernel/iommu_groups/10/devices/0000:06:00.0
/sys/kernel/iommu_groups/10/devices/0000:02:00.1
/sys/kernel/iommu_groups/10/devices/0000:05:00.0
/sys/kernel/iommu_groups/10/devices/0000:03:01.0
/sys/kernel/iommu_groups/10/devices/0000:03:04.0
/sys/kernel/iommu_groups/10/devices/0000:06:00.1
/sys/kernel/iommu_groups/0/devices/0000:00:01.0
/sys/kernel/iommu_groups/9/devices/0000:01:00.0

And actually the sound card (which should be the device 0000:08:0.6) shares the IOMMU group (13) with other devices. So for science I tried to passthrough first just the device 0000:08:0.6, and after all the iommu group which also includes IGPU + HDMI Audio (modifying the VM .conf file by removing the .6, as indicated here)

AND IT WORKS!! ....... halfway. To clarify, these are the devices seen by the VM (Sorry for windows set in Italian language)




Now, I have also installed the radeon software drivers but windows still does not recognize the video card as seen in the picture (and also of the USB HUBs). Could it be a problem related to the ROM-bar setting?
Also, for reasons unknown to me, he doesn't see the sound card at all. I also don't understand why it sees some devices as removable.

Given the current state of affairs, I didn't find it necessary to downgrade the BIOS (or change the VM's BIOS from SeaBIOS to UEFI.

Maybe these changes can help solve these problems?

Here are some dumps to verify the configuration

---

cat /etc/pve/qemu-server/101.conf

agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
boot: order=scsi0;ide2;net0
cores: 1
cpu: host,hidden=1,flags=+pcid
hostpci0: 08:00,pcie=1
ide2: local:iso/virtio-win-0.1.190.iso,media=cdrom,size=489986K
machine: q35
memory: 2048
name: Windows
net0: virtio=7E:57:F6:D3:44:1B,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
scsi0: local-lvm:vm-101-disk-0,size=40G
scsihw: virtio-scsi-pci
smbios1: uuid=24857be2-d336-4eff-bb34-0dbd875af7b2
sockets: 1
startup: order=2
vga: std
vmgenid: c680e2ec-5c53-4cdd-97a0-7d3b4b9ba188

Command extract: dmesg | grep -e DMAR -e IOMMU -e AMD-Vi (Full dump attached in the dmseg.txt file)

[    0.027311] DMAR: IOMMU enabled
[    0.871768] pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter.
[    0.873340] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.873341] pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada):
[    0.873344] AMD-Vi: Interrupt remapping enabled
[    0.873344] AMD-Vi: Virtual APIC enabled
[    0.873523] AMD-Vi: Lazy IO/TLB flushing enabled

Detailed dump of PCI devices in the lspci -v.txt file

---

THANK YOU SO MUCH FOR YOUR HELP ... AND FOR THE TIME YOU DEDICATED TO ME!!!

 

[leesteken]

Good. We learned that the devices in group 13 do not separate cleanly and securely. All devices in a single group need to be passed to the same VM. Overriding ACS allows you to ignore this, but it does not always work (as in this case).
On other motherboards and using similar processors, the on-board audio device is often in a separate IOMMU group. Your combination of CPU, BIOS and motherboard does not work, which may change with earlier (or later) BIOS versions.
I believe that Windows drivers for the integrated GPU won't expect a passed through GPU and will assume that the CPU and GPU are connected in a specific way, that does not match the virtual hardware/motherboard of the VM. Passthrough to a Linux VM is often easier, but I have not read anyone having success. Why are you using hidden=1 (and pcid, which I thought was not supported by AMD)? This might be useful for NVidia but please try without it because the drivers need to identify the specific CPU (host is a good choice for this) and GPU.
I still don't understand the Unable to read/write to IOMMU perf counter. If all virtualization settings are Enabled, then I believe this indicates that the BIOS is not optimal for this CPU.
The best AGESA version for Zen(+) and PCI passthrough (and not too old to flash) is ComboAM4 1.0.0.4B, which is F50 for your motherboard.
EDIT: Flashing the BIOS might not be worth the risk if you cannot achieve your goal of GPU passthrough anyway.

 

[CAppa2]

Hello, I took some time to do some tests.

On other motherboards and using similar processors, the on-board audio device is often in a separate IOMMU group. Your combination of CPU, BIOS and motherboard does not work, which may change with earlier (or later) BIOS versions.

I had seen it looking around, in fact it seems strange to me ... The strange thing is that, however, having included the whole IOMMU group, the VM still does not see any sound card.

Why are you using hidden=1 (and pcid, which I thought was not supported by AMD)?

I honestly did not know that the pcid was not supported by AMD, I just followed the guide which explained how to do the GPU passthrough hoping it would work for me too.

I still followed your advice, removing hidden=1,flags=+pcid in the .conf file of the VM. However, the system began to behave strangely. When I start the VM, the whole "server" restarts ... later, when the system comes back online, if I start the VM everything works.
Where by "it works" I mean that the VM starts normally but device management is always the same as before, that is:



I also noticed that if I try to have windows update or install the IGPU drivers, it tells me "the best drivers are already installed for this device".
Then I started to experiment (out of desperation)

---

TESTS​

1. I tried to manually install the Vega 8 drivers via the windows device manager. and Windows crashes​
2. After a while I realized that actually doing this:​

   And actually the sound card (which should be the device 0000:08:0.6) shares the IOMMU group (13) with other devices. So for science I tried to passthrough first just the device 0000:08:0.6, and after all the iommu group which also includes IGPU + HDMI Audio (modifying the VM .conf file by removing the .6, as indicated here)

   Looking at the IOMMU groups, I'm going through 2. Because the IGPU is in the IOMMU group 12, everything else in the 13​

   root@pve:~# find /sys/kernel/iommu_groups/ -type l | grep -e 12 -e 13
   /sys/kernel/iommu_groups/13/devices/0000:08:00.4
   /sys/kernel/iommu_groups/13/devices/0000:08:00.2
   /sys/kernel/iommu_groups/13/devices/0000:08:00.3
   /sys/kernel/iommu_groups/13/devices/0000:08:00.1
   /sys/kernel/iommu_groups/13/devices/0000:08:00.6
   
   /sys/kernel/iommu_groups/12/devices/0000:08:00.0

   Is it a problem?

I tried uninstalling the radeon drivers and reinstalling them with the Auto-Detect and Install tool by AMD. The installation fails and does not even find any AMD devices ... Not exactly reality

So I think you are absolutely right when you say:

I believe that Windows drivers for the integrated GPU won't expect a passed through GPU and will assume that the CPU and GPU are connected in a specific way, that does not match the virtual hardware/motherboard of the VM.

Well considering the state of the art, the only test I have left to do is flash the BIOS hoping that for some reason it will work. I'd be content with just being able to pass through the sound card and have a stable system (which does not reboot every time I try to start a VM)

---

Actual configuration
​

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt vfio-pci.ids=1002:15d8,1002:15de,1022:15e3  nofb nomodeset video=vesafb:off,efifb:off"

root@pve:~# cat /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1

[    0.027211] DMAR: IOMMU enabled
[    0.876682] pci 0000:00:00.2: AMD-Vi: Unable to read/write to IOMMU perf counter.
[    0.878244] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.878245] pci 0000:00:00.2: AMD-Vi: Extended features (0x4f77ef22294ada):
[    0.878248] AMD-Vi: Interrupt remapping enabled
[    0.878248] AMD-Vi: Virtual APIC enabled
[    0.878378] AMD-Vi: Lazy IO/TLB flushing enabled

I noticed that in this dump there are many lines that say:
iommu ivhd0: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=08:00.0 address=0x23799e410]
the address changes for each line, is normal?

VM .conf

agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
boot: order=scsi0;ide2;net0
cores: 1
cpu: host
hostpci0: 08:00.0,pcie=1
ide2: local:iso/virtio-win-0.1.190.iso,media=cdrom,size=489986K
machine: q35
memory: 2048
name: Windows
net0: virtio=7E:57:F6:D3:44:1B,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
scsi0: local-lvm:vm-101-disk-0,size=40G
scsihw: virtio-scsi-pci
smbios1: uuid=24857be2-d336-4eff-bb34-0dbd875af7b2
sockets: 1
startup: order=2
vga: std
vmgenid: c680e2ec-5c53-4cdd-97a0-7d3b4b9ba188

root@pve:~# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1022:15e3   [Audio card mobo]
options vfio-pci ids=1002:15d8,1002:15de disable-vga=1    [AMD IGPU + HDMI Audio]

 

[leesteken]

If some devices do not reset properly, it is common that PCI passthrough works once, but not after shutting down the VM and starting it again. Keep this in mind when testing, and sorry for not mentioning this before.
The audio device should be alone in a IOMMU group, if CPU, motherboard and BIOS align correctly. The Ryzen 2200G is an older Zen1 chip, but I have not heard about problems passing through the audio device (only Zen2 before it got fixed), but it also difficult to find success stories about Zen1 audio device because most people only passthrough discrete GPUs and USB controllers.
Could you run this command to get a better overview of your groups and devices? for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done

EDIT: Someone got it working!

 

[CAppa2]

Thanks for the reply and for the link! Actually An HTPC/Server is exactly what I need to do. To be more precise with this server I wanted to make 3 VMs:

1. Open Media Vault + various dockers for file sharing and more (This is already configured and fully functional with even a pass through of a sata card)
2. Windows VM with IGPU passtrough + onboard audio card (with the addition of Equalizer APO for deeper control of the card). This is to drive my 4K projector and 5.1 amplifier
3. PfSense with different firewalls

Here is the dump produced by the command:

IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 10 02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller [1022:43d5] (rev 01)
IOMMU group 10 02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller [1022:43c8] (rev 01)
IOMMU group 10 02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge [1022:43c6] (rev 01)
IOMMU group 10 03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 10 03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 10 03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 10 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 10 06:00.0 Ethernet controller [0200]: Intel Corporation 82576NS Gigabit Network Connection [8086:150a] (rev 01)
IOMMU group 10 06:00.1 Ethernet controller [0200]: Intel Corporation 82576NS Gigabit Network Connection [8086:150a] (rev 01)
IOMMU group 11 07:00.0 Non-Volatile memory controller [0108]: Realtek Semiconductor Co., Ltd. Device [10ec:5762] (rev 01)
IOMMU group 12 08:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Picasso [1002:15d8] (rev c9)
IOMMU group 13 08:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller [1002:15de]
IOMMU group 13 08:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU group 13 08:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e0]
IOMMU group 13 08:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e1]
IOMMU group 13 08:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
IOMMU group 14 09:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
IOMMU group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 3 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 4 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 5 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A [1022:15db]
IOMMU group 6 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus B [1022:15dc]
IOMMU group 7 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU group 7 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 8 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0 [1022:15e8]
IOMMU group 8 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1 [1022:15e9]
IOMMU group 8 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2 [1022:15ea]
IOMMU group 8 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3 [1022:15eb]
IOMMU group 8 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4 [1022:15ec]
IOMMU group 8 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5 [1022:15ed]
IOMMU group 8 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6 [1022:15ee]
IOMMU group 8 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7 [1022:15ef]
IOMMU group 9 01:00.0 RAID bus controller [0104]: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230] (rev 11)

So what I understand from the link to the libreElec (post 8 and 10) forum, is it enough to do a kernel update to have one more chance than passthrough and IGPU audio? If so, how do I proceed? (Forgive me, I'm not a Linux guru). Although it seems strange to me ... the posts are from 2018 I should have the kernel updated by now .. (my proxmox version is 6.3-2)

The audio device should be alone in a IOMMU group, if CPU, motherboard and BIOS align correctly

From what I see it does not appear to be alone, but together with other devices in group 13, and only the IGPU in group 12

 

[leesteken]

Actually, it is in group 13, and so is the audio part of the iGPU and two USB controllers. This is not a good grouping for passthrough. Either your motherboard or the BIOS version is preventing success.
I don't think you are going to be able to passthrough the iGPU any time soon. And I may have been mistaken and they meant something else with audio passthrough. LibreELEC does not use PCI passthrough, and I think they meant digital (DTS/DD) audio passthrough. I'm really sorry...
If you were planning on passing through network devices to a pfSense VM, you'll need better IOMMU grouping for those too. Otherwise, why not use the Proxmox (Linux) Firewall that is build-in?
Other than trying BIOS version F50 (with a CMOS clear and going through all BIOS settings to make sure they are Enabled), I don't know what to do.
And I have no hopes for the iGPU passthrough and little for passthrough of an additional discrete GPU (because you have a B450 chipset).

 

[CAppa2]

All this destroys all my plans...

Otherwise, why not use the Proxmox (Linux) Firewall that is build-in?

I was thinking of setting up a firewall for the entire home network and for all the devices connected to it, honestly I don't know if proxmox is capable of it or how to do it

Other than trying BIOS version F50

I did downgrade the BIOS anyway and now these are the IOMMU groups:

IOMMU group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 0 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 0 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 0 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0] [1022:15d3]
IOMMU group 0 01:00.0 RAID bus controller [0104]: Marvell Technology Group Ltd. 88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230] (rev 02)
IOMMU group 0 02:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset USB 3.1 XHCI Controller [1022:43d5] (rev 01)
IOMMU group 0 02:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset SATA Controller [1022:43c8] (rev 01)
IOMMU group 0 02:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Bridge [1022:43c6] (rev 01)
IOMMU group 0 03:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 0 03:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 0 03:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 400 Series Chipset PCIe Port [1022:43c7] (rev 01)
IOMMU group 0 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 0 06:00.0 Ethernet controller [0200]: Intel Corporation 82576NS Gigabit Network Connection [8086:150a] (rev 01)
IOMMU group 0 06:00.1 Ethernet controller [0200]: Intel Corporation 82576NS Gigabit Network Connection [8086:150a] (rev 01)
IOMMU group 0 08:00.0 Non-Volatile memory controller [0108]: Realtek Semiconductor Co., Ltd. Device [10ec:5762] (rev 01)
IOMMU group 1 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU group 1 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A [1022:15db]
IOMMU group 1 00:08.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus B [1022:15dc]
IOMMU group 1 09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Picasso [1002:15d8] (rev c9)
IOMMU group 1 09:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller [1002:15de]
IOMMU group 1 09:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor [1022:15df]
IOMMU group 1 09:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e0]
IOMMU group 1 09:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1 [1022:15e1]
IOMMU group 1 09:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) HD Audio Controller [1022:15e3]
IOMMU group 1 0a:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 61)
IOMMU group 2 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
IOMMU group 2 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU group 3 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0 [1022:15e8]
IOMMU group 3 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1 [1022:15e9]
IOMMU group 3 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2 [1022:15ea]
IOMMU group 3 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3 [1022:15eb]
IOMMU group 3 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4 [1022:15ec]
IOMMU group 3 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5 [1022:15ed]
IOMMU group 3 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6 [1022:15ee]
IOMMU group 3 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7 [1022:15ef]

Not exactly an improvement ... quite the opposite.
This is a big defeat ... I think I will be forced to change HW or separate all systems... virtualization, with passthrough, is a black hole

 

[leesteken]

Looks like people are working on a fix in the kernel for the perf counter issue. No idea if or when this will reach Proxmox.
EDIT: For someone it was just a case of enabling more in the BIOS.

 

[ghstudio]

Thanks for the reply and for the link! Actually An HTPC/Server is exactly what I need to do. To be more precise with this server I wanted to make 3 VMs:

1. Open Media Vault + various dockers for file sharing and more (This is already configured and fully functional with even a pass through of a sata card)
2. Windows VM with IGPU passtrough + onboard audio card (with the addition of Equalizer APO for deeper control of the card). This is to drive my 4K projector and 5.1 amplifier
3. PfSense with different firewalls

So what I understand from the link to the libreElec (post 8 and 10) forum, is it enough to do a kernel update to have one more chance than passthrough and IGPU audio? If so, how do I proceed? (Forgive me, I'm not a Linux guru). Although it seems strange to me ... the posts are from 2018 I should have the kernel updated by now .. (my proxmox version is 6.3-2)


From what I see it does not appear to be alone, but together with other devices in group 13, and only the IGPU in group 12

I am trying to essentially the same on a NUC8i5.

1) Debian/Kodi or Libreelec with HDMI audio
2) Home Assistant (supervised)
3) some network storage capability (all I really need is Samba) which I can install as part of Home assistant
4) PfSense for network VPN security

I am running 1, 2 and 3 with Debian 10 and Docker. pfSense, however requires it's own machine, so I think I have to introduce Proxmox.

Ideally, I'd set up Debian, etc in a VM and pfSense in it's own VM, but no one seems to have figured out how to do audio passthru on a NUC which has multiple devices in the same group as audio.

My plan is to install proxmox under Debian rather than Debian under Proxmox. While that will remove many of the proxmox operational benefits (which I don't care about), it should, at least in theory, let me run everything on one NUC.

 

[semanticbeeng]

the sound card (which should be the device 0000:08:0.6) shares the IOMMU group (13) with other devices. So for science I tried to passthrough first just the device 0000:08:0.6, and after all the iommu group which also includes IGPU + HDMI Audio

Nice learning about how IOMMU groups can be used and what fails.

May I piggyback to see if I understand my hardware config well?

IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Coffee Lake-S GT2 [UHD Graphics P630] [8086:9bf6] (rev 05)
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation 10th Gen Core Processor Host Bridge/DRAM Registers [8086:9b54] (rev 02)

IOMMU group 2 00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 02)
IOMMU group 2 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU104GLM [Quadro RTX 5000 Mobile / Max-Q] [10de:1eb5] (rev a1)
IOMMU group 2 01:00.1 Audio device [0403]: NVIDIA Corporation TU104 HD Audio Controller [10de:10f8] (rev a1)
IOMMU group 2 01:00.2 USB controller [0c03]: NVIDIA Corporation TU104 USB 3.1 Host Controller [10de:1ad8] (rev a1)
IOMMU group 2 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU104 USB Type-C UCSI Controller [10de:1ad9] (rev a1)

IOMMU group 3 00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 02)
IOMMU group 4 00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
IOMMU group 5 00:12.0 Signal processing controller [1180]: Intel Corporation Comet Lake PCH Thermal Controller [8086:06f9]

IOMMU group 6 00:14.0 USB controller [0c03]: Intel Corporation Comet Lake USB 3.1 xHCI Host Controller [8086:06ed]
IOMMU group 6 00:14.2 RAM memory [0500]: Intel Corporation Comet Lake PCH Shared SRAM [8086:06ef]

IOMMU group 7 00:15.0 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH Serial IO I2C Controller #0 [8086:06e8]
IOMMU group 7 00:15.1 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH Serial IO I2C Controller #1 [8086:06e9]

IOMMU group 8 00:16.0 Communication controller [0780]: Intel Corporation Comet Lake HECI Controller [8086:06e0]

IOMMU group 9 00:1b.0 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #17 [8086:06c0] (rev f0)

IOMMU group 10 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:06b8] (rev f0)
IOMMU group 11 00:1c.7 PCI bridge [0604]: Intel Corporation Comet Lake PCIe Port #8 [8086:06bf] (rev f0)
IOMMU group 12 00:1d.0 PCI bridge [0604]: Intel Corporation Comet Lake PCI Express Root Port #9 [8086:06b0] (rev f0)

IOMMU group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:068e]
IOMMU group 13 00:1f.3 Audio device [0403]: Intel Corporation Comet Lake PCH cAVS [8086:06c8]
IOMMU group 13 00:1f.4 SMBus [0c05]: Intel Corporation Comet Lake PCH SMBus Controller [8086:06a3]
IOMMU group 13 00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake PCH SPI Controller [8086:06a4]
IOMMU group 13 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (11) I219-LM [8086:0d4c]

IOMMU group 14 02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
IOMMU group 15 04:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU group 16 05:00.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU group 17 05:01.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU group 18 05:02.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU group 19 05:04.0 PCI bridge [0604]: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] [8086:15ea] (rev 06)
IOMMU group 20 06:00.0 System peripheral [0880]: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] [8086:15eb] (rev 06)
IOMMU group 21 2c:00.0 USB controller [0c03]: Intel Corporation JHL7540 Thunderbolt 3 USB Controller [Titan Ridge 4C 2018] [8086:15ec] (rev 06)
IOMMU group 22 54:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader [10ec:525a] (rev 01)
IOMMU group 23 55:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO [144d:a80a]

What am thinking, please advise if correct

1 IOMMU group 2 contains all Nvidia and the PCIe controlled so it seems like a good unit of passthrough.
Could pass it nicely to a VM to use GPU for machine learning but could not pass the audio card separately (to a Windows VM for high quality sound, for example).

2 Audio is in group 2 and IOMMU group 13 together with the Ethernet controller.
Does not seem that audio card to be usable in passthrough ...

3 Maybe use one of the IOMMU group 15 , 16, 17 or 18 Thunderbolt 3 ports to attach a quality sound/audio card and the passthrough all that to a Windows VM for quality sound use cases.
Is this the getaway solution around audio hardware coupled in unusable IOMMU groups and not usable for passthrough?

4 NVMe SSD Controllers are in IOMMU group 14 and IOMMU group 23 ; nice and not relevant for passthrough.

Right?
Thoughts?