audio is same IOMMU group as ethernet - how pass through?

[stooovie]

Hi, I have managed to somehow get GPU passthrough in Win11 guest working fine, but audio is now giving me headaches. GPU is in its own IOMMU group 0, but audio share IOMMU with multiple system-critical devices that cannot be shared between host and guest without instability.

IOMMU Group 10:


00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection I219-LM [8086:156f] (rev 21)

Is it possible to only pass through the audio controller itself? If so, how? When I pass the audio controller via GUI, the entire screen in the guest keeps blinking and the whole thing is weird and unstable. Using USB audio is not an option as I got no standalone speakers. Thanks!

 

[leesteken]

You cannot share devices from the same IOMMU group between VMs and/or the Proxmox host. Devices in the same group can pass information to each other without the IOMMU noticing. Since PCIe devices can read/write memory, that would be a security problem where the VM can read the entire memory of the Proxmox host.
If you don't care, then you can try using kernel parameter pcie_acs_override=downstream,multifunction to break IOMMU device isolation.

 

[stooovie]

I already have that parameter in my grub line. Any special instructions after that? When I pass the audio controller simply via GUI, screen on guest (win 11) is blinking (??!!). It's fine when I un-pass the audio again.

 

[leesteken]

I already have that parameter in my grub line. Any special instructions after that?

Does you Proxmox use GRUB or systemd-boot? Did you follow the instructions from the manual? Maybe you already double checked, but many people don't know.
What is the output of cat /proc/cmdline? There are no guarantees whether the (5) functions of a specific device (00:1f) can be split, and whether they will work that way.

 

[stooovie]

my system boots with GRUB AFAIK (/etc/kernel/pve-efiboot-uuids does not exist, efibootmgr -v returns "EFI variables are not supported on this system". I did follow the manual, yes, but I'm not infallible indeed.

cat /proc/cmdline returns

BOOT_IMAGE=/boot/vmlinuz-5.15.107-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off initcall_blacklist=sysfb_init video=vesa:off disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915

Thanks for your help!

 

[leesteken]

my system boots with GRUB AFAIK (/etc/kernel/pve-efiboot-uuids does not exist, efibootmgr -v returns "EFI variables are not supported on this system". I did follow the manual, yes, but I'm not infallible indeed.

I do think systemd-boot requires (U)EFI and ZFS, so it's unlikely that you are mistaken. Your kernel command line also suggests GRUB.

cat /proc/cmdline returns

BOOT_IMAGE=/boot/vmlinuz-5.15.107-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction initcall_blacklist=sysfb_init video=simplefb:off video=vesafb:off initcall_blacklist=sysfb_init video=vesa:off disable_vga=1 vfio_iommu_type1.allow_unsafe_interrupts=1 kvm.ignore_msrs=1 modprobe.blacklist=radeon,nouveau,nvidia,nvidiafb,nvidia-gpu,snd_hda_intel,snd_hda_codec_hdmi,i915

video=simplefb:off video=vesafb:off video=vesa:off don't do anything anymore (and you only ever needed one of those). There is no point in repeating initcall_blacklist=sysfb_init. I though disable_vga=1 was a vfio-pci parameter, not a kernel parameter. You don't need to blacklist drivers you don't use. The Proxmox manual suggest putting module options in /etc/modprobe.d, not the kernel command line.

Looks like pcie_acs_override is active, so there is nothing more you can do. Maybe use different hardware that is known to support your specific use case. Maybe use a separate PCIe or USB soundcard? Or setup pulseaudio/pipewire over the (virtual) network?

 

[stooovie]

BTW the blinking is an issue only in Windows. In Linux (Ubuntu, Fedora, Vanilla...), I can pass through audio just fine BUT GPU pass through only works with the display's native resolution, which is too bad because it's a 4k TV and the iGPU 620 can only do 4k in 30 Hz. Anything less goes to black screen (no signal) immediately.

 

[stooovie]

Thanks, that GRUB line does indeed need cleaning - it's possible there's a conflict somewhere but I've had the same issue with Linux guests with any GRUB line, so it's probably not that.

 

[stooovie]

So turns out I had to actually DISABLE pcie_acs_override=downstream,multifunction from my GRUB line, even if the audio card shares IOMMU group wth a bunch of other things including ethernet. With acs_override, my screen would blink non-stop while audio was passed through. No audio passed through, no blinking.

I now have a perfect Win 11 VM with audio and GPU passed through. Took four full days but worth it

My grub line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt initcall_blacklist=sysfb_init"

my /etc/modprobe.d/vfio.conf:

options vfio-pci ids=8086:5916,8086:9d71

(that's ID of GPU and audio)

Win 11 in UEFI mode (Q35, OMVF), set up like this:

agent: 1
args: -device vfio-pci,host=00:02.0,romfile=/etc/pve/qemu-server/i915ovmf.rom,x-igd-opregion=on
balloon: 0
bios: ovmf
boot: order=ide0;ide2;net0
cores: 3
cpu: host
efidisk0: storage:vm-108-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:00:1f.3
ide0: storage:vm-108-disk-1,size=64G
ide2: local:iso/virtio-win-0.1.229.iso,media=cdrom,size=522284K
machine: pc-q35-7.2
memory: 4096
meta: creation-qemu=7.2.0,ctime=1684231737
name: Win11Tinyx64
net0: e1000=5A:64:55:93:18:1C,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=50838f84-55c8-4cab-bffd-0d1890f6c092
sockets: 1
tablet: 0
tags: windows
tpmstate0: storage:vm-108-disk-2,size=4M,version=v2.0
usb0: host=1-3.3
usb1: host=1-3.1
usb2: host=1-3.4
vga: none
vmgenid: 8bca5789-dbdc-47ea-921a-f920c83a5cc5

 

[leesteken]

So turns out I had to actually DISABLE pcie_acs_override=downstream,multifunction from my GRUB line, even if the audio card shares IOMMU group wth a bunch of other things including ethernet. With acs_override, my screen would blink non-stop while audio was passed through. No audio passed through, no blinking.

I have never read about something like that. Can you please show all the groups without the override using this command: for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done?

I now have a perfect Win 11 VM with audio and GPU passed through. Took four full days but worth it

Glad it worked out in the end.

 

[stooovie]

Sure, here:

IOMMU group 0 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers [8086:5904] (rev 02)
IOMMU group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
IOMMU group 10 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
IOMMU group 10 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
IOMMU group 10 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
IOMMU group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection I219-LM [8086:156f] (rev 21)
IOMMU group 11 01:00.0 Non-Volatile memory controller [0108]: Kingston Technology Company, Inc. Device [2646:5017] (rev 03)
IOMMU group 12 02:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU group 13 03:00.0 Network controller [0280]: Intel Corporation Wireless 7260 [8086:08b1] (rev 73)
IOMMU group 1 00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 620 [8086:5916] (rev 02)
IOMMU group 2 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller [8086:9d2f] (rev 21)
IOMMU group 2 00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Thermal subsystem [8086:9d31] (rev 21)
IOMMU group 3 00:15.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #0 [8086:9d60] (rev 21)
IOMMU group 3 00:15.1 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #1 [8086:9d61] (rev 21)
IOMMU group 4 00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-LP CSME HECI #1 [8086:9d3a] (rev 21)
IOMMU group 5 00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-LP SATA Controller [AHCI mode] [8086:9d03] (rev 21)
IOMMU group 6 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #1 [8086:9d10] (rev f1)
IOMMU group 7 00:1c.2 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #3 [8086:9d12] (rev f1)
IOMMU group 8 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 [8086:9d18] (rev f1)
IOMMU group 9 00:1e.0 Signal processing controller [1180]: Intel Corporation Sunrise Point-LP Serial IO UART Controller #0 [8086:9d27] (rev 21)
IOMMU group 9 00:1e.4 SD Host controller [0805]: Intel Corporation Device [8086:9d2b] (rev 21)

This is now my entire grub line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt initcall_blacklist=sysfb_init"

and everything works fine. Home Assistant VM, couple of LXC containers including Jellyfin (without HW acceleration, not required) and Win 10 and 11 VMs (one at a time), full GPU and audio passthrough with acceleration.

I do have an issue with all flavors of Linux I've tried - they all run via HDMI but the moment I set something lower than the native resolution (needed for 60Hz in 4k), screen goes blank.

But Windows VMs run great.

 

[amiguelezg]

stooovie, thank very much for your configuration. ​

I also spend 3-4 days working on it but without result. Using your configuration it works but with the problem that I loose the network in the proxmox system because the network and the audio are in the same IOMMU group.
I see in your machine both are also in the same group, do you have the same problem ?

How did you get the file i915ovmg.rom ? did you compilate it ?

 

[stooovie]

hi, i did not have this issue with networking/audio, I did have a different one - under heavy load, I lost audio altogether and had to reboot the VM to get it back.

As for the ROM, I did try both downloaded and compiled, both worked.

BUT! As of PM8.x, the passthrough stopped working altogether, the host crashes immediately after launching the VM with GPU PT. I reported it and tried to fix it, to no avail. I don't use it anymore.

 

[amiguelezg]

I am on Proxmox 8 and with the limitation of the same group for audio / network it works.

For me the problem was that the proxmox load the driver, even it is in the blacklist.
Try to create a file : /etc/modprobe.d/intel-uhd-passthru.conf with the code

install i915 /bin/false

For me the problem was that the proxmox load the driver, even it is in the blacklist.
You can check if it is in use with the command in the shell
modprobe -r i915

 

[beralt]

Hey @stooovie ,

I have the same problem as you do: separating the audio and network adapter IOMMU groups does not work on my hardware (Intel NUC8i5).

You said you got it working somehow on your Win 11 in UEFI mode (Q35, OMVF) machine, but I am not sure what "it" really entails.
I n your config I only see hostpci0, so do I correctly assume that you did not passthrough the audio device after all.

Just to be clear:
When you say audio works, do you mean audio works in a remote (RDP/VNC) session, or do you mean that you have video and audio on the HDMI/DP Port of your hosts graphics card?

My goal is to get video and audio from a linux guest VM to output via the HDMI port on the host, and I am trying to figure out whether you managed to do that.

 

[chromium]

I have the same issue. Trying to get hdmi audio in Linux Mint from an Intel 630 igpu. I can get video no worries, but can't get any sound.

If I pass through the audio on 00:1f.3, it kills the network on the proxmox box because it's in the same iommu group

@stooovie: How are you passing it through just fine in Linux?

I tried adding pcie_acs_override=downstream,multifunction to GRUB_CMDLINE_LINUX_DEFAULT but I still don't have any audio in the VM and it breaks networking

 

[leesteken]

If I pass through the audio on 00:1f.3, it kills the network on the proxmox box because it's in the same iommu group

I tried adding pcie_acs_override=downstream,multifunction to GRUB_CMDLINE_LINUX_DEFAULT but I still don't have any audio in the VM and it breaks networking

Did you check with cat /proc/cmdline if the override was active? Maybe you can use a Ethernet USB-dongle instead of the built-in network port?

 

[chromium]

Did you check with cat /proc/cmdline if the override was active? Maybe you can use a Ethernet USB-dongle instead of the built-in network port?

It turns out I'm not even using grub I added it to /etc/kernel/cmdline and it still didn't help.

I can get the guest to see the Cannon Lake PCH cAVS, I can see it when I do lspci. It still doesn't appear as a sound output though and as soon as the guest boots I lose network on the proxmox server, interestingly network still works on the guest. I think I have to give up for now. I've tried all the suggestions in this post and elsewhere, I guess you really need separate iommu groups per device. Not sure how the OP managed to get it working.

I really need the audio via hdmi as I don't want to hook up speakers to this PC

 

[leesteken]

It turns out I'm not even using grub I added it to /etc/kernel/cmdline and it still didn't help.

Can you show /etc/kernel/cmdline and cat /proc/cmdline? Sometimes people put the parameters on multiple lines, which does not work for systemd-boot.

EDIT: Nevermind, if the device does not function when it is passed through then just splitting the group won't make it work. I fear that the audio device or its driver just doesn't work with passthrough, which is common with integrated graphics.

 

[chromium]

Can you show /etc/kernel/cmdline and cat /proc/cmdline? Sometimes people put the parameters on multiple lines, which does not work for systemd-boot.

EDIT: Nevermind, if the device does not function when it is passed through then just splitting the group won't make it work. I fear that the audio device or its driver just doesn't work with passthrough, which is common with integrated graphics.

For the record:

cat /proc/cmdline
initrd=\EFI\proxmox\6.2.16-15-pve\initrd.img-6.2.16-15-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction

I also tried adding initcall_blacklist=sysfb_init and removing pcie_acs_override=downstream,multifunction like the OP

Here's the rest of my config:

cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=8086:3e92,8086:a348

cat /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

lspci -n -s 00:02.0 -v
00:02.0 0300: 8086:3e92 (prog-if 00 [VGA controller])
        DeviceName: Onboard - Video
        Subsystem: 1028:085a
        Flags: bus master, fast devsel, latency 0, IRQ 255, IOMMU group 0
        Memory at 90000000 (64-bit, non-prefetchable) [size=16M]
        Memory at 80000000 (64-bit, prefetchable) [size=256M]
        I/O ports at 3000 [size=64]
        Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
        Capabilities: [40] Vendor Specific Information: Len=0c <?>
        Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
        Capabilities: [ac] MSI: Enable- Count=1/1 Maskable- 64bit-
        Capabilities: [d0] Power Management version 2
        Capabilities: [100] Process Address Space ID (PASID)
        Capabilities: [200] Address Translation Service (ATS)
        Capabilities: [300] Page Request Interface (PRI)
        Kernel driver in use: vfio-pci
        Kernel modules: i915

lspci -n -s 00:1f.3 -v
00:1f.3 0403: 8086:a348 (rev 10)
        DeviceName: Onboard - Sound
        Subsystem: 1028:085a
        Flags: fast devsel, IRQ 255, IOMMU group 9
        Memory at 91230000 (64-bit, non-prefetchable) [disabled] [size=16K]
        Memory at 91000000 (64-bit, non-prefetchable) [disabled] [size=1M]
        Capabilities: [50] Power Management version 3
        Capabilities: [80] Vendor Specific Information: Len=14 <?>
        Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel, snd_sof_pci_intel_cnl

I also tried using /etc/modprobe.d/blacklist.conf, but I believe that I don't need to if I use /etc/modprobe.d/vfio.conf. It didn't help anyway

Thanks for your help. I'm both a Proxmox and Linux noob, so this has been a good learning experience, even if I haven't got the result that I want