SOLVED BY SETTING CSM UEFI SETTINGS FOR THE PCIE LANES
IF YOU ARE USING AN X79 CHINESE MOTHERBOARD WITH AMERICAN MEGATRENDS BIOS MAKE SURE TO TURN ON UEFI EXPOSURE FOR PCIE LANES IF YOU HAVE PVE INSTALLED WITH UEFI.
YOU ALSO WILL NOT NEED TO USE A ROM FILE AS THE ROM BAR WILL BE FULLY EXPOSED AND NO ERROR WILL OCCUR
Picture attached:
![1652743271794.png 1652743271794.png](https://forum.proxmox.com/data/attachments/36/36677-625ed528e0ab55173d1f8faf67b41d50.jpg)
ORIGINAL POST:
Having an issue using main GPU passthrough on a PVE7 host. I have a patched ROM file I have attached which causes a black screen on start and then does not boot. Without the patched ROM, it doesn't recognize the GPU at all.
System Specs:
2x E5-4620v2
6x 16gb RAM
EVGA GTX1070ti
I did not add
Used this script to trim down ROM:
https://github.com/Matoking/NVIDIA-vBIOS-VFIO-Patcher
Video for another way I tried:
https://www.youtube.com/watch?v=1IP-h9IKof0&ab_channel=SpaceinvaderOne
Any help would be appreciated, thank you
IF YOU ARE USING AN X79 CHINESE MOTHERBOARD WITH AMERICAN MEGATRENDS BIOS MAKE SURE TO TURN ON UEFI EXPOSURE FOR PCIE LANES IF YOU HAVE PVE INSTALLED WITH UEFI.
YOU ALSO WILL NOT NEED TO USE A ROM FILE AS THE ROM BAR WILL BE FULLY EXPOSED AND NO ERROR WILL OCCUR
Picture attached:
![1652743271794.png 1652743271794.png](https://forum.proxmox.com/data/attachments/36/36677-625ed528e0ab55173d1f8faf67b41d50.jpg)
ORIGINAL POST:
Having an issue using main GPU passthrough on a PVE7 host. I have a patched ROM file I have attached which causes a black screen on start and then does not boot. Without the patched ROM, it doesn't recognize the GPU at all.
System Specs:
2x E5-4620v2
6x 16gb RAM
EVGA GTX1070ti
I did not add
options vfio-pci ids=10de:1b82,10de:10f0 disable_vga=1
to /etc/modprobe.d/vfio.conf
because it didn't make a difference in the black screen or "No more image in the PCI ROM" error.Used this script to trim down ROM:
https://github.com/Matoking/NVIDIA-vBIOS-VFIO-Patcher
Video for another way I tried:
https://www.youtube.com/watch?v=1IP-h9IKof0&ab_channel=SpaceinvaderOne
cat /etc/pve/qemu-server/100.conf
(trimmed down to the related items) (adding ROM file causes boot error and black screen, no vPOST)
Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,kvm=off'
balloon: 0
bios: ovmf
cores: 6
cpu: host,hidden=1,flags=+pcid
hostpci0: 0000:03:00,pcie=1,romfile=PATCHED.EVGA.rom
machine: pc-q35-6.1
memory: 12288
meta: creation-qemu=6.1.0,ctime=1650000137
numa: 1
ostype: win10
scsihw: virtio-scsi-pci
smbios1: uuid=73696d32-f77b-4d9c-bebc-b6fbee433cc2
sockets: 2
dmesg | grep -e DMAR -e IOMMU -e 3:00
Code:
[ 0.015182] ACPI: DMAR 0x000000007C040A40 000138 (v01 A M I OEMDMAR 00000001 INTL 00000001)
[ 0.015206] ACPI: Reserving DMAR table memory at [mem 0x7c040a40-0x7c040b77]
[ 0.330222] DMAR: IOMMU enabled
[ 0.696605] DMAR: Host address width 46
[ 0.696607] DMAR: DRHD base: 0x000000fbffe000 flags: 0x0
[ 0.696614] DMAR: dmar0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020de
[ 0.696617] DMAR: DRHD base: 0x000000dfffc000 flags: 0x1
[ 0.696621] DMAR: dmar1: reg_base_addr dfffc000 ver 1:0 cap d2078c106f0466 ecap f020de
[ 0.696623] DMAR: RMRR base: 0x0000007c663000 end: 0x0000007c671fff
[ 0.696626] DMAR: ATSR flags: 0x0
[ 0.696627] DMAR: RHSA base: 0x000000fbffe000 proximity domain: 0x1
[ 0.696629] DMAR: RHSA base: 0x000000dfffc000 proximity domain: 0x0
[ 0.696632] DMAR-IR: IOAPIC id 3 under DRHD base 0xfbffe000 IOMMU 0
[ 0.696634] DMAR-IR: IOAPIC id 0 under DRHD base 0xdfffc000 IOMMU 1
[ 0.696636] DMAR-IR: IOAPIC id 2 under DRHD base 0xdfffc000 IOMMU 1
[ 0.696637] DMAR-IR: HPET id 0 under DRHD base 0xdfffc000
[ 0.696639] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.697308] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 1.157850] pci 0000:03:00.0: [10de:1b82] type 00 class 0x030000
[ 1.157861] pci 0000:03:00.0: reg 0x10: [mem 0xde000000-0xdeffffff]
[ 1.157870] pci 0000:03:00.0: reg 0x14: [mem 0xc0000000-0xcfffffff 64bit pref]
[ 1.157880] pci 0000:03:00.0: reg 0x1c: [mem 0xd0000000-0xd1ffffff 64bit pref]
[ 1.157887] pci 0000:03:00.0: reg 0x24: [io 0x8000-0x807f]
[ 1.157894] pci 0000:03:00.0: reg 0x30: [mem 0xdf000000-0xdf07ffff pref]
[ 1.157901] pci 0000:03:00.0: enabling Extended Tags
[ 1.157917] pci 0000:03:00.0: BAR 3: assigned to efifb
[ 1.157989] pci 0000:03:00.0: 32.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x16 link at 0000:00:02.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
[ 1.158033] pci 0000:03:00.1: [10de:10f0] type 00 class 0x040300
[ 1.158044] pci 0000:03:00.1: reg 0x10: [mem 0xdf080000-0xdf083fff]
[ 1.158080] pci 0000:03:00.1: enabling Extended Tags
[ 1.159758] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 1.159791] acpi PNP0A03:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability LTR]
[ 1.184611] pci 0000:03:00.0: vgaarb: setting as boot VGA device
[ 1.184613] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[ 1.184628] pci 0000:03:00.0: vgaarb: bridge control possible
[ 1.264742] pci 0000:03:00.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 1.264784] pci 0000:03:00.1: D0 power state depends on 0000:03:00.0
[ 1.265221] DMAR: No SATC found
[ 1.265226] DMAR: dmar0: Using Queued invalidation
[ 1.265244] DMAR: dmar1: Using Queued invalidation
[ 1.266059] pci 0000:03:00.0: Adding to iommu group 25
[ 1.266089] pci 0000:03:00.1: Adding to iommu group 25
[ 1.269500] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 4.669564] snd_hda_intel 0000:03:00.1: Disabling MSI
[ 4.669586] snd_hda_intel 0000:03:00.1: Handle vga_switcheroo audio client
[ 4.772097] input: HDA NVidia HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input14
[ 4.772162] input: HDA NVidia HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input15
[ 4.772215] input: HDA NVidia HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input16
[ 4.772272] input: HDA NVidia HDMI/DP,pcm=9 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input17
[ 4.772330] input: HDA NVidia HDMI/DP,pcm=10 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input18
[ 4.772383] input: HDA NVidia HDMI/DP,pcm=11 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input19
[ 4.772451] input: HDA NVidia HDMI/DP,pcm=12 as /devices/pci0000:00/0000:00:02.0/0000:03:00.1/sound/card0/input20
[ 146.152872] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[ 146.172941] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[ 146.192738] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[ 151.708537] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x19@0x900
[ 151.710666] vfio-pci 0000:03:00.0: No more image in the PCI ROM
uname -r
Code:
5.13.19-6-pve
cat /etc/default/grub
Code:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt nofb nomodeset video=vesafb:off video=efifb:off video=simplefb:off"
cat /proc/cmdline
Code:
BOOT_IMAGE=/boot/vmlinuz-5.13.19-6-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt nofb nomodeset video=vesafb:off video=efifb:off video=simplefb:off
cat /etc/modules
Code:
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
Any help would be appreciated, thank you
Last edited: