Passhtrough 8gen intel Nuc

I had problems for several days trying to pass through my GPU (while other devices were easily passed) with a black screen. Very frustrating.

The solution seemed to be related to how Proxmox was being booted. Once I switched from standard booting to UEFI booting (in mobo BIOS), the GPU passed through just fine. (Also, the GPU must be bootable with UEFI–––and most recently made GPUs are UEFI bootable.) Further, you need to adjust your blacklist for your GPU (see Spoiler; if the file is changed, remember to run "update-initramfs -u -k all").

blacklist amdgpu
blacklist snd_hda_intel
blacklist xhci_hcd
blacklist nouveau
blacklist nvidia
blacklist nvidiafb

I also found suggestions to adjust the /etc/modprobe.d/vfio.conf file; see next Spoiler.

options vfio-pci ids=1002:687f,1002:aaf8
(using " disable_vga=1" after above declaration, seemed not to be necessary)
options vfio_iommu_type1 allow_unsafe_interrupts=1

Some other issues to watch in the VM config file are using "bios: ovmf", "machine: q35", and remembering to change "vga: vmware" to "vga: none" along with the usual "hostpci0: xx:00,pcie=1,x-vga=1".

If a rom file is required, it needs to be placed in folder "/usr/share/kvm/myGPU.rom" and then change the VM config file to something like "hostpci0: xx:00,pcie=1,x-vga=1,romfile=myGPU.rom".
 
Last edited:
  • Like
Reactions: augustopaulo
For good measure here are the changes I've made - I'm only trying to passthrough the GPU, I don't need any audio at the moment:

/etc/default/grub
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifb:0ff pcie_acs_override=downstream"
GRUB_CMDLINE_LINUX=""

/etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

/etc/modprobe.d/blacklist.conf
blacklist i915

/etc/modprobe.d/vfio.conf
options vfio-pci ids=8086:3ea5

/etc/pve/qemu-server/105.conf
bootdisk: ide0
cores: 1
hostpci0: 00:02,pcie=1,x-vga=1
ide0: local-lvm:vm-105-disk-0,size=32G
ide2: local:iso/Windows.iso,media=cdrom
machine: q35
memory: 2048
name: test
net0: e1000=8E:9B:DC:74:AA:51,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=e0b0ecd3-5d54-47c1-8256-7e53b83978c4
sockets: 1
vga: none
vmgenid: da249683-19a7-4d4e-ac68-7b330f4b0124

I've also tried with a UEFI bios for the VM but that didn't make any difference.
 
Uh, I did not get any notifications on updates.
I just wanted to check again to make it work, but after an Update of PVE to 6.2-4 I am not able to get GPU kernel blacklisted. VGA controller is still on i915 and not on vfio-pci, though i915 is listed in my blacklist. Same issue seems to be with the soundcard.

does anyone severe the same issue?

lacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915
00:02.0 VGA compatible controller: Intel Corporation Iris Plus Graphics 655 (rev 01) (prog-if 00 [VGA controller])
Subsystem: Intel Corporation Iris Plus Graphics 655
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at bf000000 (64-bit, non-prefetchable) [size=16M]
Memory at 80000000 (64-bit, prefetchable) [size=256M]
I/O ports at 4000
[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: [40] Vendor Specific Information: Len=0c <?>
Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [ac] MSI: Enable- Count=1/1 Maskable- 64bit-
Capabilities: [d0] Power Management version 2
Capabilities: [100] Process Address Space ID (PASID)
Capabilities: [200] Address Translation Service (ATS)
Capabilities: [300] Page Request Interface (PRI)
Kernel modules: i915
 

Thanks. I had to do changes on the vfio.conf etc. as with the update on the new kernel there were some changes implemented. A workarround I found with this thread, which is working fine for me: https://forum.proxmox.com/threads/igd-passthrough-setup-broke-with-proxmox-6-2.69670/#post-313176
Did you manage to passthrough audio? Every time I add:
hostpci1: 00:1f.3

to my configuration, the complete network of my NUC crashes. As I am now able to see the the VM as GPU passthrough is working, I can see that this is starting as usual. It only hangs for 2mins at the network controller. In addition there is no access to any container or VM via Webinterface or ssh. Also the connection to the host gets lost. As the Ethernet Controller (00:1f.6) is in the same group as the audio, I guess it must be an issue with the group
 
Thanks. I had to do changes on the vfio.conf etc. as with the update on the new kernel there were some changes implemented. A workarround I found with this thread, which is working fine for me: https://forum.proxmox.com/threads/igd-passthrough-setup-broke-with-proxmox-6-2.69670/#post-313176
Did you manage to passthrough audio? Every time I add:
hostpci1: 00:1f.3

to my configuration, the complete network of my NUC crashes. As I am now able to see the the VM as GPU passthrough is working, I can see that this is starting as usual. It only hangs for 2mins at the network controller. In addition there is no access to any container or VM via Webinterface or ssh. Also the connection to the host gets lost. As the Ethernet Controller (00:1f.6) is in the same group as the audio, I guess it must be an issue with the group


If your NUC has IOMMU Grouping like mine then you are correct and this would explain why you lose network when passing through the audio:

IOMMU Group 11: 00:1f.0 ISA bridge [0601]: Intel Corporation Cannon Point-LP LPC Controller [8086:9d84] (rev 30) 00:1f.3 Audio device [0403]: Intel Corporation Cannon Point-LP High Definition Audio Controller [8086:9dc8] (rev 30) 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Point-LP SMBus Controller [8086:9da3] (rev 30) 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller [8086:9da4] (rev 30) 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (6) I219-V [8086:15be] (rev 30)

You can see the Ethernet controller and Audio device are in the same IOMMU group and from my understanding all devices in a group must stay together. I'm not sure of this, but you might be able to separate the devices into different groups with the ACS override patch. Please let us know your results if you try this. Still it's nice to see this thread active and people having success with this. The NUC is such a great device to run a virtual environment on. I have two Windows 10 VM's, one running Blue Iris for my security cameras and another for Pentesting. I also have Plex on Ubuntu server and the NUC handles all three very well even with heavy transcoding on Plex!
 
If your NUC has IOMMU Grouping like mine then you are correct and this would explain why you lose network when passing through the audio:

IOMMU Group 11: 00:1f.0 ISA bridge [0601]: Intel Corporation Cannon Point-LP LPC Controller [8086:9d84] (rev 30) 00:1f.3 Audio device [0403]: Intel Corporation Cannon Point-LP High Definition Audio Controller [8086:9dc8] (rev 30) 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Point-LP SMBus Controller [8086:9da3] (rev 30) 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller [8086:9da4] (rev 30) 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (6) I219-V [8086:15be] (rev 30)

You can see the Ethernet controller and Audio device are in the same IOMMU group and from my understanding all devices in a group must stay together. I'm not sure of this, but you might be able to separate the devices into different groups with the ACS override patch. Please let us know your results if you try this. Still it's nice to see this thread active and people having success with this. The NUC is such a great device to run a virtual environment on. I have two Windows 10 VM's, one running Blue Iris for my security cameras and another for Pentesting. I also have Plex on Ubuntu server and the NUC handles all three very well even with heavy transcoding on Plex!

Thank you for your hint. This makes sense, but it didn't work out. I got exactly the same setup like you in group 11.
Grub looks like this right now:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8
3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction"

When I add downstream I don't see any changes. When I add downstream,multifunction the entries appear in group 12, but there is no separation.

But, ACS override seems to be enabled: dmesg | grep -i acs
Code:
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.41-1-pve root=/dev/mapper/pve-root ro i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.013057] ACPI: FACS 0x000000007A2E7080 000040
[    0.072420] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.4.41-1-pve root=/dev/mapper/pve-root ro i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction

For me I am already a couple of containers. With the Ubuntu VM and KODI I wanted to step on using it connected to my TV. Any further ideas or things I am still doing wrong?
 
Thank you for your hint. This makes sense, but it didn't work out. I got exactly the same setup like you in group 11.
Grub looks like this right now:
Code:
GRUB_CMDLINE_LINUX_DEFAULT="i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8
3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction"

When I add downstream I don't see any changes. When I add downstream,multifunction the entries appear in group 12, but there is no separation.

But, ACS override seems to be enabled: dmesg | grep -i acs
Code:
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.41-1-pve root=/dev/mapper/pve-root ro i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction
[    0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[    0.013057] ACPI: FACS 0x000000007A2E7080 000040
[    0.072420] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.4.41-1-pve root=/dev/mapper/pve-root ro i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on video=efifb:off vfio-pci.ids=8086:3ea5,8086:9dc8 pcie_acs_override=downstream,multifunction

For me I am already a couple of containers. With the Ubuntu VM and KODI I wanted to step on using it connected to my TV. Any further ideas or things I am still doing wrong?

If you change:
pcie_acs_override=
to the specific device you want to be in a different IOMMU Group does that work?

pcie_acs_override =
[PCIE] Override missing PCIe ACS support for:
downstream
All downstream ports - full ACS capabilties
multifunction
All multifunction devices - multifunction ACS subset
id:nnnn:nnnn
Specfic device - full ACS capabilities
Specified as vid:did (vendor/device ID) in hex


According to this, that should put the specific device (in your case Audio) in a separate IOMMU Group.
 
If you change:
pcie_acs_override=
to the specific device you want to be in a different IOMMU Group does that work?

pcie_acs_override =
[PCIE] Override missing PCIe ACS support for:
downstream
All downstream ports - full ACS capabilties
multifunction
All multifunction devices - multifunction ACS subset
id:nnnn:nnnn
Specfic device - full ACS capabilities
Specified as vid:did (vendor/device ID) in hex


According to this, that should put the specific device (in your case Audio) in a separate IOMMU Group.

Thank you for the hint, but this also did not work out. Until now I checked:

pcie_acs_override=downstream,multifunction,id:8086:9dc8
pcie_acs_override=downstream,id:8086:9dc8
pcie_acs_override=id:8086:9dc8

I even started making a new kernel and recognised, that the patch seems to be already included in the recent one - so no need to mak your own kernel with patch.

I had a closer look to the entry with multifunction. In that case only the Wireless AC is separated from group 4. So the patch seems to be working:
lspci
00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers (rev 08)
00:02.0 VGA compatible controller: Intel Corporation Iris Plus Graphics 655 (rev 01)
00:08.0 System peripheral: Intel Corporation Skylake Gaussian Mixture Model
00:12.0 Signal processing controller: Intel Corporation Cannon Point-LP Thermal Controller (rev 30)
00:14.0 USB controller: Intel Corporation Cannon Point-LP USB 3.1 xHCI Controller (rev 30)
00:14.2 RAM memory: Intel Corporation Cannon Point-LP Shared SRAM (rev 30)
00:14.3 Network controller: Intel Corporation Cannon Point-LP CNVi [Wireless-AC] (rev 30)
00:16.0 Communication controller: Intel Corporation Cannon Point-LP MEI Controller (rev 30)
00:17.0 SATA controller: Intel Corporation Cannon Point-LP SATA Controller [AHCI Mode] (rev 30)
00:1c.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1c.4 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1d.6 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1f.0 ISA bridge: Intel Corporation Cannon Point-LP LPC Controller (rev 30)
00:1f.3 Audio device: Intel Corporation Cannon Point-LP High Definition Audio Controller (rev 30)
00:1f.4 SMBus: Intel Corporation Cannon Point-LP SMBus Controller (rev 30)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller (rev 30)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (6) I219-V (rev 30)
6d:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981
6e:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS522A PCI Express Card Reader (rev 01)

for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:02.0
/sys/kernel/iommu_groups/2/devices/0000:00:08.0
/sys/kernel/iommu_groups/3/devices/0000:00:12.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/5/devices/0000:00:14.3
/sys/kernel/iommu_groups/6/devices/0000:00:16.0
/sys/kernel/iommu_groups/7/devices/0000:00:17.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.4
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1d.6
/sys/kernel/iommu_groups/12/devices/0000:00:1f.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:00:1f.4
/sys/kernel/iommu_groups/12/devices/0000:00:1f.5
/sys/kernel/iommu_groups/12/devices/0000:00:1f.6
/sys/kernel/iommu_groups/13/devices/0000:6d:00.0
/sys/kernel/iommu_groups/14/devices/0000:6e:00.0
 
Does anyone has experience with Thunderbolt 3? I guess a solution with an external GPU is too expensive, but I am thinking about a network adapter to override the internal network adapter. Does this work out, or will the same controller be used?
 
Thank you for the hint, but this also did not work out. Until now I checked:

pcie_acs_override=downstream,multifunction,id:8086:9dc8
pcie_acs_override=downstream,id:8086:9dc8
pcie_acs_override=id:8086:9dc8

I even started making a new kernel and recognised, that the patch seems to be already included in the recent one - so no need to mak your own kernel with patch.

I had a closer look to the entry with multifunction. In that case only the Wireless AC is separated from group 4. So the patch seems to be working:
lspci
00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers (rev 08)
00:02.0 VGA compatible controller: Intel Corporation Iris Plus Graphics 655 (rev 01)
00:08.0 System peripheral: Intel Corporation Skylake Gaussian Mixture Model
00:12.0 Signal processing controller: Intel Corporation Cannon Point-LP Thermal Controller (rev 30)
00:14.0 USB controller: Intel Corporation Cannon Point-LP USB 3.1 xHCI Controller (rev 30)
00:14.2 RAM memory: Intel Corporation Cannon Point-LP Shared SRAM (rev 30)
00:14.3 Network controller: Intel Corporation Cannon Point-LP CNVi [Wireless-AC] (rev 30)
00:16.0 Communication controller: Intel Corporation Cannon Point-LP MEI Controller (rev 30)
00:17.0 SATA controller: Intel Corporation Cannon Point-LP SATA Controller [AHCI Mode] (rev 30)
00:1c.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1c.4 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1d.6 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0)
00:1f.0 ISA bridge: Intel Corporation Cannon Point-LP LPC Controller (rev 30)
00:1f.3 Audio device: Intel Corporation Cannon Point-LP High Definition Audio Controller (rev 30)
00:1f.4 SMBus: Intel Corporation Cannon Point-LP SMBus Controller (rev 30)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller (rev 30)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (6) I219-V (rev 30)
6d:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981
6e:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS522A PCI Express Card Reader (rev 01)

for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:02.0
/sys/kernel/iommu_groups/2/devices/0000:00:08.0
/sys/kernel/iommu_groups/3/devices/0000:00:12.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/5/devices/0000:00:14.3
/sys/kernel/iommu_groups/6/devices/0000:00:16.0
/sys/kernel/iommu_groups/7/devices/0000:00:17.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.4
/sys/kernel/iommu_groups/10/devices/0000:00:1d.0
/sys/kernel/iommu_groups/11/devices/0000:00:1d.6
/sys/kernel/iommu_groups/12/devices/0000:00:1f.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:00:1f.4
/sys/kernel/iommu_groups/12/devices/0000:00:1f.5
/sys/kernel/iommu_groups/12/devices/0000:00:1f.6
/sys/kernel/iommu_groups/13/devices/0000:6d:00.0
/sys/kernel/iommu_groups/14/devices/0000:6e:00.0

Since you are not able to separate the Ethernet Controller into a different group using the ACS override patch does that indicate through the NUC's design that it is perhaps "hard wired" to that grouping?
 
Has anyone got the GPU passthrough working with Proxmox 6.2-4? Care to share the config please? :) Would love to use Proxmox for everything, but I really need to utilize the display that I just mounted..
 
Has anyone got the GPU passthrough working with Proxmox 6.2-4? Care to share the config please? :) Would love to use Proxmox for everything, but I really need to utilize the display that I just mounted..

I think there is some issue arising after the update. I had a perfectly working system on which (may I be damned :D) I ran an update, and now passthrough doesn't work anymore. The config is the usual one posted around in these forums.
(I should add that it did only work in linux and not in Windows 10...)
 
I think there is some issue arising after the update. I had a perfectly working system on which (may I be damned :D) I ran an update, and now passthrough doesn't work anymore. The config is the usual one posted around in these forums.
(I should add that it did only work in linux and not in Windows 10...)

Do you or anyone else know at what version it was still working? Then I could try to downgrade.. If anyone had a solution for the current version that would be the best though. Really, really need this to work!

(Doesn't matter which VM OS, need to display a browser, currently been trying with Ubuntu)
 
Do you or anyone else know at what version it was still working? Then I could try to downgrade.. If anyone had a solution for the current version that would be the best though. Really, really need this to work!

(Doesn't matter which VM OS, need to display a browser, currently been trying with Ubuntu)

Sorry my bad, I didn't expect this sort of "regression bug", so I didn't verify at what version I was when everything was working. Honestly I didn't update anything for probably a couple of months, so...
 
Sorry my bad, I didn't expect this sort of "regression bug", so I didn't verify at what version I was when everything was working. Honestly I didn't update anything for probably a couple of months, so...

Understandable, thanks for the quick reply :)
 
I had it running with 6.1.7. Suddenly but not anymore I don't know why? Update ? Now with 6.2.11 I just can't get it to work. I have an Intel Iris 655. (Intel Nuc 8 i7 )This is also passed through and recognized. BUT CODE 43 ... does anyone have any ideas?
 
I had it running with 6.1.7. Suddenly but not anymore I don't know why? Update ? Now with 6.2.11 I just can't get it to work. I have an Intel Iris 655. (Intel Nuc 8 i7 )This is also passed through and recognized. BUT CODE 43 ... does anyone have any ideas?

Same here, since 6.1.something the passthrough stopped working both in linux and in windows (well, I NEVER managed to have it working in windows to be honest, but in linux I had zero issues...)
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!