Help needed for Using GPU passthrough on intel NUC

[razhot]

Hello
I have an Intel NUC (model BOXNUC8i5BEH, with Core i5-8259U Processor and Intel Iris Plus Graphics 655), with 16GB RAM, one 240GB SSD and 2TB HDD

In addition of several VM (MariaDB, FreeNAS, Jeedom...), I would like to have a VM with Kodi running, with this NUC connected to a TV with the HDMI.

I have read several thread in this forum and other google search, to setup the GPU passthrough and I have setup the host and VM in the following way

Spoiler: pveversion -v

proxmox-ve: 6.1-2 (running kernel: 5.3.18-3-pve)
pve-manager: 6.1-11 (running version: 6.1-11/f2f18736)
pve-kernel-helper: 6.1-9
pve-kernel-5.3: 6.1-6
pve-kernel-5.3.18-3-pve: 5.3.18-3
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve8
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libproxmox-acme-perl: 1.0.2
libpve-access-control: 6.0-7
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-1
libpve-guest-common-perl: 3.0-10
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve2
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.1-6
pve-cluster: 6.1-8
pve-container: 3.1-4
pve-docs: 6.1-6
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.1-2
pve-firmware: 3.0-7
pve-ha-manager: 3.0-9
pve-i18n: 2.1-1
pve-qemu-kvm: 4.1.1-4
pve-xtermjs: 4.3.0-1
qemu-server: 6.1-20
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1

Step 1
I started to update /etc/default/grub file with
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
and update it with
update-grub
and did a reboot

/etc/default/grub

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
GRUB_CMDLINE_LINUX=""
GRUB_DISABLE_OS_PROBER=true
GRUB_DISABLE_RECOVERY="true"
GRUB_FONT=/boot/grub/fonts/DejaVuSansMono36.pf2

then I checked that I have now IOMMU with
dmesg | grep -e DMAR -e IOMMU

Spoiler: dmesg | grep -e DMAR -e IOMMU

[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
[ 0.012566] ACPI: DMAR 0x0000000079DEADE0 0000A8 (v01 INTEL NUC8i5BE 0000004F 01000013)
[ 0.094170] DMAR: IOMMU enabled
[ 0.211662] DMAR: Host address width 39
[ 0.211663] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.211670] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[ 0.211671] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.211675] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[ 0.211676] DMAR: RMRR base: 0x00000079d32000 end: 0x00000079d51fff
[ 0.211676] DMAR: RMRR base: 0x0000007b800000 end: 0x0000007fffffff
[ 0.211685] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.211688] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.211690] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.214426] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 1.214672] DMAR: No ATSR found
[ 1.214747] DMAR: dmar0: Using Queued invalidation
[ 1.214762] DMAR: dmar1: Using Queued invalidation
[ 1.227963] DMAR: Intel(R) Virtualization Technology for Directed I/O

and checked also for the group with
find /sys/kernel/iommu_groups/ -type l | sort

Spoiler: find /sys/kernel/iommu_groups/ -type l | sort

/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/10/devices/0000:00:1d.6
/sys/kernel/iommu_groups/11/devices/0000:00:1f.0
/sys/kernel/iommu_groups/11/devices/0000:00:1f.3
/sys/kernel/iommu_groups/11/devices/0000:00:1f.4
/sys/kernel/iommu_groups/11/devices/0000:00:1f.5
/sys/kernel/iommu_groups/11/devices/0000:00:1f.6
/sys/kernel/iommu_groups/12/devices/0000:6e:00.0
/sys/kernel/iommu_groups/1/devices/0000:00:02.0
/sys/kernel/iommu_groups/2/devices/0000:00:08.0
/sys/kernel/iommu_groups/3/devices/0000:00:12.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.0
/sys/kernel/iommu_groups/4/devices/0000:00:14.2
/sys/kernel/iommu_groups/4/devices/0000:00:14.3
/sys/kernel/iommu_groups/5/devices/0000:00:16.0
/sys/kernel/iommu_groups/6/devices/0000:00:17.0
/sys/kernel/iommu_groups/7/devices/0000:00:1c.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.4
/sys/kernel/iommu_groups/9/devices/0000:00:1d.0

Step2
I added VFIO modules in the file

/etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Step 3
I run
dmesg | grep 'remapping'
to check about IOMMU interrupt remapping
I got following result
[ 0.211690] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.214426] DMAR-IR: Enabled IRQ remapping in x2apic mode
so I understand from other thread that it should be ok, and therefore I did nothing more for the remapping, in particular, I did not run echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

Step 4
To blackilist the driver, I did add to the /etc/modprobe.d/blacklist.conf the driver that I have seen for the video and audio (got name with lspci -v )

/etc/modprobe.d/blacklist.conf

blacklist i915
blacklist snd_hda_intel
blacklist sof_pci_dev

Step 5
To handle the VFIO, I did for the GPU
lspci -n -s 00:02.0
to get
00:02.0 0300: 8086:3ea5 (rev 01)
and for the audio
lspci -n -s 00:1f.3
to get
00:1f.3 0403: 8086:9dc8 (rev 30)
with this 2 UID, I did add them in the file

/etc/modprobe.d/vfio.conf

options vfio-pci ids=8086:3ea5
options vfio-pci ids=8086:9dc8

and then update it with
update-initramfs -u
and the I did a reboot

Step 6
First before creating the GPU passthrough KODI VM, I check that old ones where still running fine, which was the case
To create this Kodi VM, I did it with the following parameters

• System: put Graphic card : none (this is what I saw in different thread, not so sure that it is the right choice)
• BIOS = OMVF (UEFI) and created EFI disk
• Machine = q35
• kept CPU to default kvm64
• Ram: kept Ballonning = Yes (what is the meaning?)
• Network: use the bridge vmbr0 and kept firewall to Yes (better with or without firewall?)

The I added the 2 PCI hardware

• Video
  • All Functions: No (not sure of that it is the best choice?)
  • Rom-Bar: Yes (not sure of that it is the best choice?)
  • Primary GPU: No (not sure of that it is the best choice?)
  • PCI Express: No (not sure of that it is the best choice?)
• Audio
  • All Functions: No (not sure of that it is the best choice?)
  • Rom-Bar: Yes (not sure of that it is the best choice?)
  • Primary GPU: No
  • PCI Express: No

and I added the keyboard and mouse USB

here is the file /etc/pve/qemu-server/107.conf of this VM's settings

bios: ovmf
bootdisk: scsi0
cores: 2
efidisk0: VM_SSD:100/vm-100-disk-1.qcow2,size=128K
hostpci0: 00:02.0
hostpci1: 00:1f.3
ide2: ISO_HDD:iso/ubuntu-20.04-live-server-amd64.iso,media=cdrom
machine: q35
memory: 2048
name: Kodi
net0: virtio=2E:AD:B0:61:50:94,bridge=vmbr0,firewall=1
numa: 0
ostype: l26
scsi0: VM_SSD:100/vm-100-disk-0.qcow2,size=12G
scsihw: virtio-scsi-pci
smbios1: uuid=cafcc154-9ee8-4663-8308-56f850d43b8c
sockets: 1
usb0: host=03f0:0641,usb3=1
usb1: host=03f0:0024,usb3=1
vga: none
vmgenid: c734e49d-5567-421a-a8f1-553424f96053

RESULTS:
When I start the VM to boot on the ubuntu installation, the TV become black at once and nothing is going on after. It looks that the network goes down, as the ssh console with the host break and the proxmox webpage is not responding anymore too (by the way, the ethernet 00:1f.6 is on the same group 11 than audio 00:1f.3, could it be a reason?)

I have tried few changes to the initial setup described above, but none of them helped. thing that I tried:

change 1
run echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
reboot
Not working better, deleted this file

change 2
change the line in /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifbff pcie_acs_override=downstream"
update-grub
reboot
Not working better, but keep it this way for next tests

change 3
in the VM hardware

• For the PCI Video, put Primary VGA=Yes
• change Network=e1000, remove firewall

hostpci0: 00:02,x-vga=1
net0: e1000=2E:AD:B0:61:50:94,bridge=vmbr0

Not working better

change 4
in VM hardware

• chage PCI-Audio = all function
• put back network to virtio (but still without firewall

hostpci1: 00:1f
net0: virtio=2E:AD:B0:61:50:94,bridge=vmbr0

Not working better


So I am out of idea, if someone has some feedback of what can be wrong, your help is welcome.

Thanks in advance for your time. When the issue is found and I get it working, my goal is to make a full tutorial with this answer for others to get clear steps.

 

[razhot]

for info, the quite full result of spci -v (i add to cut a bit due to the size limit)

Spoiler: lspci -v

00:00.0 Host bridge: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers (rev 08)
Subsystem: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers
Flags: bus master, fast devsel, latency 0
Capabilities: [e0] Vendor Specific Information: Len=10 <?>
Kernel driver in use: skl_uncore

00:02.0 VGA compatible controller: Intel Corporation Iris Plus Graphics 655 (rev 01) (prog-if 00 [VGA controller])
Subsystem: Intel Corporation Iris Plus Graphics 655
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at bf000000 (64-bit, non-prefetchable) [size=16M]
Memory at 80000000 (64-bit, prefetchable) [size=256M]
I/O ports at 4000
[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
Capabilities: [40] Vendor Specific Information: Len=0c <?>
Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [ac] MSI: Enable- Count=1/1 Maskable- 64bit-
Capabilities: [d0] Power Management version 2
Capabilities: [100] Process Address Space ID (PASID)
Capabilities: [200] Address Translation Service (ATS)
Capabilities: [300] Page Request Interface (PRI)
Kernel modules: i915

00:08.0 System peripheral: Intel Corporation Skylake Gaussian Mixture Model
Subsystem: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th Gen Core Processor Gaussian Mixture Model
Flags: fast devsel, IRQ 11

00:12.0 Signal processing controller: Intel Corporation Cannon Point-LP Thermal Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP Thermal Controller
Flags: fast devsel, IRQ 16

Kernel driver in use: intel_pch_thermal
Kernel modules: intel_pch_thermal

00:14.0 USB controller: Intel Corporation Cannon Point-LP USB 3.1 xHCI Controller (rev 30) (prog-if 30 [XHCI])
Subsystem: Intel Corporation Cannon Point-LP USB 3.1 xHCI Controller
Flags: bus master, medium devsel, latency 0, IRQ 126
Memory at 404ac00000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [70] Power Management version 2
Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
Capabilities: [90] Vendor Specific Information: Len=14 <?>
Kernel driver in use: xhci_hcd

00:14.2 RAM memory: Intel Corporation Cannon Point-LP Shared SRAM (rev 30)
Subsystem: Intel Corporation Cannon Point-LP Shared SRAM
Flags: fast devsel
Capabilities: [80] Power Management version 3

00:14.3 Network controller: Intel Corporation Cannon Point-LP CNVi [Wireless-AC] (rev 30)
Subsystem: Intel Corporation Cannon Point-LP CNVi [Wireless-AC]
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at 404ac10000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [c8] Power Management version 3
Capabilities: [d0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [80] MSI-X: Enable+ Count=16 Masked-
Capabilities: [100] #00
Capabilities: [14c] Latency Tolerance Reporting
Capabilities: [164] Vendor Specific Information: ID=0010 Rev=0 Len=014 <?>
Kernel driver in use: iwlwifi
Kernel modules: iwlwifi

00:16.0 Communication controller: Intel Corporation Cannon Point-LP MEI Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP MEI Controller
Flags: bus master, fast devsel, latency 0, IRQ 130
Memory at 404ac17000 (64-bit, non-prefetchable) [size=4K]
Capabilities: [50] Power Management version 3
Capabilities: [8c] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [a4] Vendor Specific Information: Len=14 <?>
Kernel driver in use: mei_me
Kernel modules: mei_me

00:17.0 SATA controller: Intel Corporation Cannon Point-LP SATA Controller [AHCI Mode] (rev 30) (prog-if 01 [AHCI 1.0])
Subsystem: Intel Corporation Cannon Point-LP SATA Controller [AHCI Mode]
Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 127
Memory at c0a24000 (32-bit, non-prefetchable) [size=8K]
Memory at c0a27000 (32-bit, non-prefetchable)
I/O ports at 4090
I/O ports at 4080
I/O ports at 4060
Memory at c0a26000 (32-bit, non-prefetchable) [size=2K]
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [70] Power Management version 3
Capabilities: [a8] SATA HBA v1.0
Kernel driver in use: ahci
Kernel modules: ahci

00:1c.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 122
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
Capabilities: [40] Express Root Port (Slot-), MSI 00
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Intel Corporation Cannon Point-LP PCI Express Root Port
Capabilities: [a0] Power Management version 3
Kernel driver in use: pcieport

00:1c.4 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 123
Bus: primary=00, secondary=02, subordinate=6c, sec-latency=0
I/O behind bridge: 00005000-00005fff
Memory behind bridge: 90000000-be0fffff
Prefetchable memory behind bridge: 0000004000000000-0000004049ffffff
Capabilities: [40] Express Root Port (Slot+), MSI 00
Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [90] Subsystem: Intel Corporation Cannon Point-LP PCI Express Root Port
Capabilities: [a0] Power Management version 3
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Access Control Services
Capabilities: [150] Precision Time Measurement
Capabilities: [220] #19
Capabilities: [250] Downstream Port Containment
Kernel driver in use: pcieport

00:1d.0 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 124
Bus: primary=00, secondary=6d, subordinate=6d, sec-latency=0
Kernel driver in use: pcieport

00:1d.6 PCI bridge: Intel Corporation Cannon Point-LP PCI Express Root Port (rev f0) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0, IRQ 125
Bus: primary=00, secondary=6e, subordinate=6e, sec-latency=0
I/O behind bridge: 00003000-00003fff
Memory behind bridge: c0000000-c09fffff
Prefetchable memory behind bridge: 000000404a100000-000000404aafffff
Kernel driver in use: pcieport

00:1f.0 ISA bridge: Intel Corporation Cannon Point-LP LPC Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP LPC Controller
Flags: bus master, medium devsel, latency 0

00:1f.3 Audio device: Intel Corporation Cannon Point-LP High Definition Audio Controller (rev 30) (prog-if 80)
Subsystem: Intel Corporation Cannon Point-LP High Definition Audio Controller
Flags: bus master, fast devsel, latency 32, IRQ 11
Memory at c0a20000 (64-bit, non-prefetchable) [size=16K]
Memory at 404ab00000 (64-bit, non-prefetchable) [size=1M]
Capabilities: [50] Power Management version 3
Capabilities: [80] Vendor Specific Information: Len=14 <?>
Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel, sof_pci_dev

00:1f.4 SMBus: Intel Corporation Cannon Point-LP SMBus Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP SMBus Controller
Flags: medium devsel, IRQ 16
Memory at 404ac16000 (64-bit, non-prefetchable)
I/O ports at efa0
Kernel driver in use: i801_smbus
Kernel modules: i2c_i801

00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP SPI Controller
Flags: fast devsel
Memory at fe010000 (32-bit, non-prefetchable) [size=4K]

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (6) I219-V (rev 30)
Subsystem: Intel Corporation Ethernet Connection (6) I219-V
Flags: bus master, fast devsel, latency 0, IRQ 129
Memory at c0a00000 (32-bit, non-prefetchable) [size=128K]
Capabilities: [c8] Power Management version 3
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Kernel driver in use: e1000e
Kernel modules: e1000e

6e:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS522A PCI Express Card Reader (rev 01)
Subsystem: Intel Corporation RTS522A PCI Express Card Reader
Physical Slot: 18
Flags: bus master, fast devsel, latency 0, IRQ 128
Memory at c0000000 (32-bit, non-prefetchable) [size=4K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [70] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-00-00-01-00-4c-e0-00
Capabilities: [150] Latency Tolerance Reporting
Capabilities: [158] L1 PM Substates
Kernel driver in use: rtsx_pci
Kernel modules: rtsx_pci

 

[detxm80]

hi, i've nuc8i3beh and if i use pve 6.1.7 i've succesfully enable gpu pass, but if use pve 6.2.4 using the same guide (it live on another forum) i can't enable it

 

[razhot]

I get it half working...

I upgraded Proxmox to the last version (6.2) that is out since yesterday

Spoiler: PVEVERSION -V

proxmox-ve: 6.2-1 (running kernel: 5.4.34-1-pve)
pve-manager: 6.2-4 (running version: 6.2-4/9824574a)
pve-kernel-5.4: 6.2-1
pve-kernel-helper: 6.2-1
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.34-1-pve: 5.4.34-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 2.0.1-1+pve8
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libproxmox-acme-perl: 1.0.3
libpve-access-control: 6.1-1
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-2
libpve-guest-common-perl: 3.0-10
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve2
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-1
pve-cluster: 6.1-8
pve-container: 3.1-5
pve-docs: 6.2-4
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-2
pve-qemu-kvm: 5.0.0-2
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1

Step 1, having now the line in GRUB as described in change 2 of the initial post (basically, just having now the following line in /etc/default/grub )

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifbff pcie_acs_override=downstream"

Step 2 (/etc/modules), 3 (IOMMU interrupt remapping), 4 (/etc/modprobe.d/blacklist.conf ) and 5 (/etc/modprobe.d/vfio.conf ) are the same as in the initial post

For step 6, I did the following changes in VM hardware

• a new line boot: dcn has been added (I changed the boot order, probably not relevant)
• change the 2 PCI devices as PCI express device
  • For Video, I have put Primary VGA=Yes
  • For Audio, I have removed the all function, and have rombar=0 (not clear if it is important or not, did not try with rombar=1)

Here is the /etc/pve/qemu-server/107.conf file

bios: ovmf
boot: dcn
bootdisk: scsi0
cores: 2
efidisk0: VM_SSD:100/vm-100-disk-1.qcow2,size=128K
hostpci0: 00:02,pcie=1,x-vga=1
hostpci1: 00:1f.3,pcie=1,rombar=0
ide2: ISO_HDD:iso/ubuntu-20.04-live-server-amd64.iso,media=cdrom
machine: q35
memory: 2048
name: Kodi
net0: virtio=2E:AD:B0:61:50:94,bridge=vmbr0
numa: 0
ostype: l26
scsi0: VM_SSD:100/vm-100-disk-0.qcow2,size=12G
scsihw: virtio-scsi-pci
smbios1: uuid=cafcc154-9ee8-4663-8308-56f850d43b8c
sockets: 1
usb0: host=03f0:0641,usb3=1
usb1: host=03f0:0024,usb3=1
vga: none
vmgenid: c734e49d-5567-421a-a8f1-553424f96053

RESULTS:
When I start the VM to boot on the ubuntu installation, the TV screen on the hdmi cable is now working (I don't know if it is because the new Proxmox version, or the small changes I did in VM hardware) , and I can proceed with Ubuntu install.

UNRESOLVED PROBLEM
The Ethernet network is still not working

• During the Ubuntu installation, it did not see the virtual network device in the VM
• The ssh console with the host stop working
• The Proxmox webpage is not responding anymore too

By the way, I get also a syslogd message on ssh console of the host before the network goes down (seems related to SMBus which is on IRQ 16, and is also in iommu group 11)

Message from syslogd@NUC at May 13 22:08:27 ...
kernel:[ 70.495692] Disabling IRQ #16

Spoiler: ommu_groups

find /sys/kernel/iommu_groups/ -type l | sort
(show only group 11)

/sys/kernel/iommu_groups/11/devices/0000:00:1f.0​

/sys/kernel/iommu_groups/11/devices/0000:00:1f.3​

/sys/kernel/iommu_groups/11/devices/0000:00:1f.4​

/sys/kernel/iommu_groups/11/devices/0000:00:1f.5​

/sys/kernel/iommu_groups/11/devices/0000:00:1f.6​

lspci -v -s 00:1f
00:1f.0 ISA bridge: Intel Corporation Cannon Point-LP LPC Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP LPC Controller
Flags: bus master, medium devsel, latency 0

00:1f.3 Audio device: Intel Corporation Cannon Point-LP High Definition Audio Controller (rev 30) (prog-if 80)
Subsystem: Intel Corporation Cannon Point-LP High Definition Audio Controller
Flags: bus master, fast devsel, latency 32, IRQ 11
Memory at c0a20000 (64-bit, non-prefetchable) size=16K
Memory at 404ab00000 (64-bit, non-prefetchable) size=1M
Capabilities: [50] Power Management version 3
Capabilities: [80] Vendor Specific Information: Len=14 <?>
Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
Kernel modules: snd_hda_intel, snd_sof_pci

00:1f.4 SMBus: Intel Corporation Cannon Point-LP SMBus Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP SMBus Controller
Flags: medium devsel, IRQ 16
Memory at 404ac16000 (64-bit, non-prefetchable) size=256
I/O ports at efa0 size=32
Kernel driver in use: i801_smbus
Kernel modules: i2c_i801

00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Point-LP SPI Controller (rev 30)
Subsystem: Intel Corporation Cannon Point-LP SPI Controller
Flags: fast devsel
Memory at fe010000 (32-bit, non-prefetchable) size=4K

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (6) I219-V (rev 30)
Subsystem: Intel Corporation Ethernet Connection (6) I219-V
Flags: bus master, fast devsel, latency 0, IRQ 129
Memory at c0a00000 (32-bit, non-prefetchable) size=128K
Capabilities: [c8] Power Management version 3
Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Kernel driver in use: e1000e
Kernel modules: e1000e

So I guess that the problem is because the audio device is in the same group than the Ethernet device and other devices as SMBus.
Is there a way to force the system so the Audio will be in its own group?
or some other ideas to get the network working?

 

[razhot]

hi, i've nuc8i3beh and if i use pve 6.1.7 i've succesfully enable gpu pass, but if use pve 6.2.4 using the same guide (it live on another forum) i can't enable it

can you show how your network is handled in the immuo groups?

 

[Steffenprox]

If you use a linux distribution for Kodi try machine type: i440fx not q35. Also try some different distributions. I use Zorin OS. Sometimes it's better to enable Default graphicscard at the installation and disable if the installation ist successful.

 

[razhot]

Sometimes it's better to enable Default graphicscard

You mean the hardware of the VM (so to remove the vga: none line in /etc/pve/qemu-server/107.conf)?
or do you mean to have it on the hostPCI line (hostpci0: 00:02,pcie=1,x-vga=1 )?

 

[Steffenprox]

The installation you can remove the line "vga: none" so that its possible to to install the system with the Console, also activate the GPU Passthrough with "hostpci0: 00:02.0,rombar=0", when you had finish the installation you can disable the console access and add "vga: none". Also you can't use Pci-Express (pcie=1) with machine type: i440fx.

 

[razhot]

I did not manage to get it working with the advice from @Steffenprox (remove vga=none line, use i440fx machine)

After reading several forums, I have tried to ungroup the Audio device from other device, with different version of /etc/default/grub

• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream,multifunction"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=multifunction"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=id:8086:9dc8"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=id:8086:9dc8,id:8086:9d84,id:8086:15be"

None of them worked, Audio and Ethernet are always in the same group. It looks that it is a limitation of the NUC hardware, and I will give up trying passthrough for the moment if no one come up with some other ideas

 

[Steffenprox]

Well if it's not working you can install proxmox on Debian with GUI and add the Multimedia repo for kodi.

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster

 

[razhot]

Yes it is also my plan. I would have prefer to get Kodi in a VM for the snapshot and backup, but it looks that I have no other choice than putting it on the host with Proxmox

 

[daftlink]

Yes it is also my plan. I would have prefer to get Kodi in a VM for the snapshot and backup, but it looks that I have no other choice than putting it on the host with Proxmox

Hello I'm stuck with the same hardware, did you find any solutions ?

 

[ghstudio]

I'd like a solution as well.....I want to run Kodi under Ubuntu so I get Kodi 18. Only Kodi 17 is supported under Debian.

 

[Ramalama]

I'd like a solution as well.....I want to run Kodi under Ubuntu so I get Kodi 18. Only Kodi 17 is supported under Debian.

you clearly never heard of the debian multimedia repo, there is even kodi 19.

@razhot
i read that you are using an 8th gen igpu, as far i know, thats the last (newest) generation where gvt-d and gvt-g is working. i wondering why no one mentioned this.
https://github.com/intel/gvt-linux/wiki/GVTd_Setup_Guide
https://github.com/intel/gvt-linux/wiki/GVTg_Setup_Guide

if you search for "gvt-g debian" in google, you will find good howtos.

Edit:
Another hint is, that i never seen anyone successfully (without errors) passed through an igpu. Since they share cache and probably more with the cpu. So even if you get it working, i wouldnt recommend it. Thats why gvt-d even exists.

cheers

 

[RobertusIT]

I did not manage to get it working with the advice from @Steffenprox (remove vga=none line, use i440fx machine)

After reading several forums, I have tried to ungroup the Audio device from other device, with different version of /etc/default/grub

• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream,multifunction"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=multifunction"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=id:8086:9dc8"
• GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=id:8086:9dc8,id:8086:9d84,id:8086:15be"

None of them worked, Audio and Ethernet are always in the same group. It looks that it is a limitation of the NUC hardware, and I will give up trying passthrough for the moment if no one come up with some other ideas

Hihave you figure out?

I config my nuc with this guide for gtv-g and I have the same issue, no audio, no ethernet, no proxmox GUi, so I'm off, needed to reinstall all... How you fix this problem?

 

[ghstudio]

The only "fix" I've found is to run proxmox under debian (NUC7i5). I would still like to find a way to install proxmox with Debian/Kodi running it's own VM.

 

[RobertusIT]

The only "fix" I've found is to run proxmox under debian (NUC7i5). I would still like to find a way to install proxmox with Debian/Kodi running it's own VM.

so have kodi and proxmox ? so not isolate kodi in a virtual machine?