IGD passthrough help: pve kernel: vfio-pci 0000:00:02.0: BAR 2: can't reserve

[sinaamos]

I am running into issues passing a intel GPU uhd630, come with many troubles.
My host is a Intel 10500es, 32G RAM, BIOS is OK, I want to pass the UHD630 to the Synology NAS (DSM918 with 6.24)
I follow this guide step by step: https://pve.proxmox.com/wiki/PCI_Passthrough, everything is OK when configure files, but it does not work after I complete the configuration.

Syslog：
pve kernel: vfio-pci 0000:00:02.0: BAR 2: can't reserve [mem 0xa0000000-0xafffffff 64bit pref] Apr 24 19:55:10 pve kernel: pcieport 0000:00:1b.0: [ 0] RxErr Apr 24 19:55:10 pve kernel: pcieport 0000:00:1b.0: AER: Corrected error received: 0000:00:1b.0 Apr 24 19:55:10 pve kernel: pcieport 0000:00:1b.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID) Apr 24 19:55:10 pve kernel: pcieport 0000:00:1b.0: device [8086:a3eb] error status/mask=00000001/00002000
this info continually generate and the log file getting larger over time

Details about configuration (PVE 7.3)
1）GRUB

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt video=vesafb:off video=efifb:off video=simplefb:off pcie_acs_override=downstream,multifunction"

2) blacklist.conf

blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
blacklist i915

3)vfio.conf

options vfio-pci ids=8086:9bc8

4)MY pci device

00:00.0 Host bridge [0600]: Intel Corporation Comet Lake-S 6c Host Bridge/DRAM Controller [8086:9b53]
00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901]
00:02.0 VGA compatible controller [0300]: Intel Corporation CometLake-S GT2 [UHD Graphics 630] [8086:9bc8]
00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
00:14.0 USB controller [0c03]: Intel Corporation Comet Lake PCH-V USB Controller [8086:a3af]
00:14.2 Signal processing controller [1180]: Intel Corporation Comet Lake PCH-V Thermal Subsystem [8086:a3b1]
00:15.0 Signal processing controller [1180]: Intel Corporation Device [8086:a3e0]
00:15.1 Signal processing controller [1180]: Intel Corporation Device [8086:a3e1]
00:16.0 Communication controller [0780]: Intel Corporation Device [8086:a3ba]
00:17.0 SATA controller [0106]: Intel Corporation 400 Series Chipset Family SATA AHCI Controller [8086:a382]
00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:a3eb] (rev f0)
00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:a39a] (rev f0)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a3c8]
00:1f.2 Memory controller [0580]: Intel Corporation Memory controller [8086:a3a1]
00:1f.3 Audio device [0403]: Intel Corporation Device [8086:a3f0]
00:1f.4 SMBus [0c05]: Intel Corporation Comet Lake PCH-V SMBus Host Controller [8086:a3a3]
01:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
02:01.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
02:05.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
02:07.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
06:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
07:00.0 Non-Volatile memory controller [0108]: Yangtze Memory Technologies Co.,Ltd Device [1e49:0001] (rev 03)
08:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)

5) VM102 SETUP

bios: seabios
boot: order=sata1
cores: 8
cpu: host
hostpci0: 0000:04:00  
hostpci1: 0000:02:00    #UHD630
hostpci2: 0000:00:17
hotplug: disk,network,usb
memory: 12288
meta: creation-qemu=7.2.0,ctime=1681674516
name: DSM918
numa: 0
ostype: l26
sata0: local:102/vm-102-disk-0.raw,size=128M,ssd=1   #Bootloader
sata1: local:102/vm-102-disk-1.raw,size=1074M,ssd=1   #Bootloader (just choose one)
scsihw: virtio-scsi-single
serial0: socket
smbios1: uuid=426981ac-0f48-4ea7-a349-4829f7832a2c
sockets: 1
startup: order=3
tablet: 1
usb0: host=0463:ffff
vmgenid: 7acd74e0-2825-42ba-b238-a6f38bbab81b

 

[leesteken]

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt video=vesafb:off video=efifb:off video=simplefb:off pcie_acs_override=downstream,multifunction"

video=vesafb:off video=efifb:off video=simplefb:off have not worked on Proxmox for some time. If you want to passthrough the boot GPU, you need to use this work-around to fix "BAR can't reserve mem". Also double check with cat /proc/cmdline because not all Proxmox installations use GRUB.

 

[sinaamos]

video=vesafb:off video=efifb:off video=simplefb:off have not worked on Proxmox for some time. If you want to passthrough the boot GPU, you need to use this work-around to fix "BAR can't reserve mem". Also double check with cat /proc/cmdline because not all Proxmox installations use GRUB.

Thankyou, BUT I am not familiar with Linux, so I don't know if my understanding is correct：
1）I should creat a file /root/fix_gpu_pass.sh
2）In the file “fix_gpu_pass.sh“ ，the content is echo 1 > /sys/bus/pci/devices/0000\:00\:02.0/remove echo 1 > /sys/bus/pci/rescan
3）use cmd ”chmod +x /root/fix_gpu_pass.sh“
4）use cmd ”crontab -e“ （what does cmd mean？Force it auto execute ？）
Then I reboot PVE, it will auto execute fix_gpu_pass.sh. Are the above steps correct?

 

[leesteken]

Thankyou, BUT I am not familiar with Linux, so I don't know if my understanding is correct：
1）I should creat a file /root/fix_gpu_pass.sh
2）In the file “fix_gpu_pass.sh“ ，the content is echo 1 > /sys/bus/pci/devices/0000\:00\:02.0/remove echo 1 > /sys/bus/pci/rescan
3）use cmd ”chmod +x /root/fix_gpu_pass.sh“
4）use cmd ”crontab -e“ （what does cmd mean？Force it auto execute ？）
Then I reboot PVE, it will auto execute fix_gpu_pass.sh. Are the above steps correct?

No, just use the much easier work-around from the post that I linked: replace video=vesafb:off video=efifb:off video=simplefb:off with initcall_blacklist=sysfb_init. It should have the same effect: no more boot messages and no host console, but also no "BAR can't reserve mem" problem.

 

[sinaamos]

No, just use the much easier work-around from the post that I linked: replace video=vesafb:off video=efifb:off video=simplefb:off with initcall_blacklist=sysfb_init. It should have the same effect: no more boot messages and no host console, but also no "BAR can't reserve mem" problem.

It works when useinitcall_blacklist=sysfb_init, but now I encountered new problems, today I reboot the nas, and it happens new trouble, syslog says ：

pve kernel: vfio-pci 0000:00:02.0: vfio_ecap_init: hiding ecap 0x1b@0x100
pve kernel: vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem

Again, I can't passthrough iGD, can boot VM, but ls /dev/dri, have no such file, so passthrough failed

root@pve:~# dmesg | grep -i vfio
[    5.608187] VFIO - User Level meta-driver version: 0.3
[    5.610954] vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    5.630336] vfio_pci: add [8086:9bc8[ffffffff:ffffffff]] class 0x000000/00000000
[   22.222373] vfio-pci 0000:00:02.0: vfio_ecap_init: hiding ecap 0x1b@0x100
[   23.187143] vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem