Hello all. I have changed my i5 8400/H370 server after 5 years for a 14600k/Z690 one. I can't make the GPU passthrough work.
I boot with a mirror ZFS, so systemd-boot. I added the following to
The command
The command to check IOMMU groups show that indeed there are groups:
Everything seems OK but when I passthrough to a Ubuntu 22.04 VM, the GPU doesn't seem to be recongnized as there is no /dev/dri/renderD128. Any idea? Thanks
I boot with a mirror ZFS, so systemd-boot. I added the following to
/etc/kernel/cmdline and executed pve-efiboot-tool refresh :root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=ptThe command
dmesg | grep -E "IOMMU|DMAR" shows that IOMMU is enabled.
Code:
[ 0.007220] ACPI: DMAR 0x00000000670DC000 000088 (v01 INTEL EDK2 00000002 01000013)
[ 0.007253] ACPI: Reserving DMAR table memory at [mem 0x670dc000-0x670dc087]
[ 0.071665] DMAR: IOMMU enabled
[ 0.178954] DMAR: Host address width 39
[ 0.178955] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.178964] DMAR: dmar0: reg_base_addr fed90000 ver 4:0 cap 1c0000c40660462 ecap 29a00f0505e
[ 0.178967] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.178972] DMAR: dmar1: reg_base_addr fed91000 ver 5:0 cap d2008c40660462 ecap f050da
[ 0.178975] DMAR: RMRR base: 0x0000006c000000 end: 0x000000707fffff
[ 0.178978] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.178980] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.178981] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.179874] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.440867] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[ 0.508606] DMAR: No ATSR found
[ 0.508608] DMAR: No SATC found
[ 0.508610] DMAR: IOMMU feature fl1gp_support inconsistent
[ 0.508611] DMAR: IOMMU feature pgsel_inv inconsistent
[ 0.508614] DMAR: IOMMU feature nwfs inconsistent
[ 0.508616] DMAR: IOMMU feature dit inconsistent
[ 0.508617] DMAR: IOMMU feature sc_support inconsistent
[ 0.508619] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 0.508621] DMAR: dmar0: Using Queued invalidation
[ 0.508626] DMAR: dmar1: Using Queued invalidation
[ 0.509411] DMAR: Intel(R) Virtualization Technology for Directed I/OThe command to check IOMMU groups show that indeed there are groups:
for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU group %s ' "$n"; lspci -nns "${d##*/}"; done
Code:
IOMMU group 0 00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:a780] (rev 04)
IOMMU group 10 01:00.0 Non-Volatile memory controller [0108]: SK hynix Platinum P41 NVMe Solid State Drive 2TB [1c5c:1959]
IOMMU group 11 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
IOMMU group 12 03:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM961/PM961/SM963 [144d:a804]
IOMMU group 1 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:a704] (rev 01)
IOMMU group 2 00:06.0 PCI bridge [0604]: Intel Corporation Raptor Lake PCIe 4.0 Graphics Port [8086:a74d] (rev 01)
IOMMU group 3 00:14.0 USB controller [0c03]: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller [8086:7ae0] (rev 11)
IOMMU group 3 00:14.2 RAM memory [0500]: Intel Corporation Alder Lake-S PCH Shared SRAM [8086:7aa7] (rev 11)
IOMMU group 4 00:15.0 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0 [8086:7acc] (rev 11)
IOMMU group 5 00:16.0 Communication controller [0780]: Intel Corporation Alder Lake-S PCH HECI Controller #1 [8086:7ae8] (rev 11)
IOMMU group 6 00:17.0 SATA controller [0106]: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] [8086:7ae2] (rev 11)
IOMMU group 7 00:1c.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #6 [8086:7abd] (rev 11)
IOMMU group 8 00:1d.0 PCI bridge [0604]: Intel Corporation Alder Lake-S PCH PCI Express Root Port #13 [8086:7ab4] (rev 11)
IOMMU group 9 00:1f.0 ISA bridge [0601]: Intel Corporation Z690 Chipset LPC/eSPI Controller [8086:7a84] (rev 11)
IOMMU group 9 00:1f.4 SMBus [0c05]: Intel Corporation Alder Lake-S PCH SMBus Controller [8086:7aa3] (rev 11)
IOMMU group 9 00:1f.5 Serial bus controller [0c80]: Intel Corporation Alder Lake-S PCH SPI Controller [8086:7aa4] (rev 11)
IOMMU group 9 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (17) I219-V [8086:1a1d] (rev 11)Everything seems OK but when I passthrough to a Ubuntu 22.04 VM, the GPU doesn't seem to be recongnized as there is no /dev/dri/renderD128. Any idea? Thanks