System Overview:
Problem Description:
I’m experiencing a critical issue when assigning high RAM (>512 GB) to a Windows 11 Pro VM with GPU passthrough enabled. The VM hangs or fails to boot properly when the following conditions are met:
When GPU passthrough is disabled, the VM can boot successfully with up to 1 TB RAM. So the issue only occurs when both high memory and PCI passthrough are enabled together.
Attempting to increase the MMIO reservation using this line in the config:
args: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=0x200000000
fails with:
kvm: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=...: Invalid parameter 'value'
This seems to be due to a limitation in the default OVMF/QEMU build on Proxmox 8.4, where setting this parameter is not supported.
VM Configuration (/etc/pve/qemu-server/108.conf):
agent: enabled=1
balloon: 0
bios: ovmf
boot: order=ide0;ide2;net0
cores: 127
cpu: EPYC-Genoa
efidisk0: local-lvm:vm-108-disk-3,efitype=4m,pre-enrolled-keys=1,size=128M
hostpci0: 0000:c1:00,pcie=1
ide0: local-lvm:vm-108-disk-1,size=62G
ide2: local:iso/virtio-win.iso,media=cdrom,size=709474K
machine: pc-q35-9.2+pve1
memory: 786432
meta: creation-qemu=9.2.0,ctime=1747692088
name: MaciejVM2
net0: e1000=BC:24:11:13:80:31,bridge=vmbr0,firewall=1
numa: 1
numa0: cpus=0-126,mem=393216
numa1: cpus=127-253,mem=393216
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=1af339b5-a15d-42a0-8462-28244359ff1b
sockets: 2
tpmstate0: local-lvm:vm-108-disk-2,size=4M,version=v2.0
vga: virtio
vmgenid: 90a29539-fdaa-4f27-979a-6931e932fe8d
(Previously attempted to add args: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=0x200000000 – causes QEMU to exit with Invalid parameter 'value')
What I’ve Confirmed:
Expected Behavior:
A Windows 11 VM should boot successfully with PCI passthrough (RTX 5090) and high RAM (768 GB – 1 TB), assuming memory and PCI layout are valid. This may require increased MMIO space in the guest firmware.
Questions:
Please let me know if additional logs or test configs would be helpful. Thanks for your support.
- Host CPUs: Dual-socket AMD EPYC 9554 (256 cores total)
- Total RAM: 1 TB ECC RDIMM
- GPU: NVIDIA RTX 5090 (passed through via VFIO)
- Proxmox Version: pve-manager/8.4.0/ec58e45e1bcdf2ac
- Kernel: 6.8.12-9-pve
- QEMU Version: (from /usr/bin/qemu-system-x86_64 --version – please confirm exact version if needed)
Problem Description:
I’m experiencing a critical issue when assigning high RAM (>512 GB) to a Windows 11 Pro VM with GPU passthrough enabled. The VM hangs or fails to boot properly when the following conditions are met:
- RAM assigned > 512 GB
- PCI passthrough (hostpci0) enabled (for RTX 5090)
When GPU passthrough is disabled, the VM can boot successfully with up to 1 TB RAM. So the issue only occurs when both high memory and PCI passthrough are enabled together.
Attempting to increase the MMIO reservation using this line in the config:
args: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=0x200000000
fails with:
kvm: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=...: Invalid parameter 'value'
This seems to be due to a limitation in the default OVMF/QEMU build on Proxmox 8.4, where setting this parameter is not supported.
VM Configuration (/etc/pve/qemu-server/108.conf):
agent: enabled=1
balloon: 0
bios: ovmf
boot: order=ide0;ide2;net0
cores: 127
cpu: EPYC-Genoa
efidisk0: local-lvm:vm-108-disk-3,efitype=4m,pre-enrolled-keys=1,size=128M
hostpci0: 0000:c1:00,pcie=1
ide0: local-lvm:vm-108-disk-1,size=62G
ide2: local:iso/virtio-win.iso,media=cdrom,size=709474K
machine: pc-q35-9.2+pve1
memory: 786432
meta: creation-qemu=9.2.0,ctime=1747692088
name: MaciejVM2
net0: e1000=BC:24:11:13:80:31,bridge=vmbr0,firewall=1
numa: 1
numa0: cpus=0-126,mem=393216
numa1: cpus=127-253,mem=393216
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=1af339b5-a15d-42a0-8462-28244359ff1b
sockets: 2
tpmstate0: local-lvm:vm-108-disk-2,size=4M,version=v2.0
vga: virtio
vmgenid: 90a29539-fdaa-4f27-979a-6931e932fe8d
(Previously attempted to add args: -fw_cfg name=opt/ovmf/PcdPciMmio64Size,value=0x200000000 – causes QEMU to exit with Invalid parameter 'value')
What I’ve Confirmed:
- VM boots fine with ≤512 GB RAM and PCI passthrough enabled
- VM boots fine with 1 TB RAM if PCI passthrough is disabled
- Enabling both causes boot failure
- Hugepages enabled and NUMA properly balanced across two nodes
- GPU works under passthrough in other VMs with less memory
Expected Behavior:
A Windows 11 VM should boot successfully with PCI passthrough (RTX 5090) and high RAM (768 GB – 1 TB), assuming memory and PCI layout are valid. This may require increased MMIO space in the guest firmware.
Questions:
- Is there a supported way in Proxmox 8.4 to run Windows system on Proxmox with passthrough RTX5090 and RAM memory >512GB?
Please let me know if additional logs or test configs would be helpful. Thanks for your support.
Last edited: