Issue booting VM after installation with GPU passthrough (Ubuntu, H11DSi, EPYC 7601)

[grosfaignan]

Hello,

I’m trying to set up Proxmox on the following hardware:

• Motherboard: Supermicro H11DSi
• CPUs: 2 × AMD EPYC 7601
• RAM: 192 GB ECC
• GPU: NVIDIA RTX 3050 (passthrough using vfio-pci)

VM config :

root@pve:~# qm config 103
bios: ovmf
boot: order=scsi0
cores: 8
cpu: host,hidden=1
efidisk0: local-lvm:vm-103-disk-0,size=4M
hostpci0: 21:00.0,pcie=1,x-vga=1
hostpci1: 0000:14:00.3,pcie=1,rombar=0
ide2: local:iso/ubuntu-24.04.3-desktop-amd64.iso,media=cdrom,size=6197156K
machine: q35
memory: 16000
meta: creation-qemu=10.0.2,ctime=1756088604
name: Copy-of-VM-GpuPTTemplate
net0: e1000=BC:24:11:63:5D:19,bridge=vmbr0
ostype: l26
scsi0: local-lvm:vm-103-disk-2,discard=on,size=32G
smbios1: uuid=3a63ef58-6bf1-4057-bf90-936c6ab3a1f3
vmgenid: 2d24d441-25f5-403a-b878-8b9e83f0d046

Proxmox and kernel version :

root@pve:~# pveversion
pve-manager/9.0.3/025864202ebb6109 (running kernel: 6.14.8-2-pve)

root@pve:~# uname -a
Linux pve 6.14.8-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.14.8-2 (2025-07-22T10:04Z) x86_64 GNU/Linux

Context :

• Proxmox installs fine on the host.
• I can create a VM and boot from an ISO installer (e.g., Ubuntu) with GPU passthrough (plus keyboard and mouse).
• The installation itself works without issues.
• GPU passthrough is done via vfio-pci and an NVIDIA RTX 3050.
• Problem: after finishing the OS installation and rebooting the VM, it does not boot into the OS. Instead, the VM goes straight into the UEFI shell, as if no bootloader was found.

So, the VM works fine when booting from ISO, but once the OS is installed on disk, it seems like the EFI boot entry is missing or not being recognized.


What I’ve already tried :

• Created multiple VMs from the same template, issue is reproducible.
• Tried manually assigning a boot entry name for the ESP in the VM’s UEFI BIOS.
• Mounted the EFI partition and verified its contents:

ubuntu@ubuntu:/mnt$ ls -R /mnt/EFI
/mnt/EFI:
BOOT  ubuntu

/mnt/EFI/BOOT:
BOOTX64.EFI  fbx64.efi  mmx64.efi

/mnt/EFI/ubuntu:
BOOTX64.CSV  grub.cfg  grubx64.efi  mmx64.efi  shimx64.efi

Attempted to recreate EFI boot entry with efibootmgr:

ubuntu@ubuntu:/mnt$ sudo mount /dev/sda2 /mnt
ubuntu@ubuntu:/mnt$ sudo mount /dev/sda1 /mnt/boot/efi
ubuntu@ubuntu:/mnt$ sudo mount --bind /dev /mnt/dev
ubuntu@ubuntu:/mnt$ sudo mount --bind /proc /mnt/proc
ubuntu@ubuntu:/mnt$ sudo mount --bind /sys /mnt/sys
ubuntu@ubuntu:/mnt$ sudo chroot /mnt
root@ubuntu:/# efibootmgr -c -d /dev/sda -p 1 -L "ubuntu" -l '\EFI\ubuntu\grubx64.efi'
EFI variables are not supported on this system.
root@ubuntu:/# exit

This suggests EFI variables are not writable in this environment.


Questions :

• Has anyone seen this behavior with GPU passthrough and UEFI/OVMF?
• Is there something specific I should configure in Proxmox (e.g., EFI disk, boot order, OVMF settings) to make the VM detect the installed bootloader?
• Could this be related to NVIDIA RTX 3050 passthrough (e.g., OVMF NVRAM not persisting, or a GPU ROM/vBIOS quirk)?

Information I can provide if needed

• Output of lspci -nnk for the GPU and related devices
• Host BIOS/UEFI settings (IOMMU, ACS, SR-IOV, Above 4G decoding, etc.)
• Logs from the VM boot attempt

Thanks in advance for any advice!

 

[uzumo]

I think we should isolate whether it is a pass-through influence relationship.

If there is no pass-through, is there a problem with booting after installation?
If there is no problem, what happens if the pass-through is done post-installation?