Hello,
I've installed Proxmox on my server to do VMs anc LXC containers easily. I've also planned to create a Windows 11 VM with GPU passthrough to use the capacities of my dedicated GPU to transcode H264 videos in HEVC. So I followed this guide in a first time and at the end of the installation of Windows 11 I tried to install my GPU drivers and the VM crashes. Not only the VM crashes since the entire Proxmox has crashed. I search on the internet, on a lot of forums threads on the Proxmox forums & on Reddit also but I didn't find any solution.
Here are hardware/BIOS information:
Here are some useful configuration information:
Here is my Windows 11 virtual machine configuration:
What I tried:
I also provide 2 screenshots of the errors I have encountered respectively on Windows and at Ubuntu boot.
thanks in advance to those who will help me.
I've installed Proxmox on my server to do VMs anc LXC containers easily. I've also planned to create a Windows 11 VM with GPU passthrough to use the capacities of my dedicated GPU to transcode H264 videos in HEVC. So I followed this guide in a first time and at the end of the installation of Windows 11 I tried to install my GPU drivers and the VM crashes. Not only the VM crashes since the entire Proxmox has crashed. I search on the internet, on a lot of forums threads on the Proxmox forums & on Reddit also but I didn't find any solution.
Here are hardware/BIOS information:
- Motherboard: Atermitter x79 ATX socket LGA 2011
- CPU: Intel Xeon E5-2630 v2 2.6 GHz
- RAM: 32 GB (4x8GB) ECC registered 1333 MHz (quad channel)
- GPU: Sapphire RX 550 2GB GDDR5 OC
- BIOS: AMI UEFI BIOS
- BIOS mode: UEFI only, legacy/CSM totally disabled
- Intel virtualization technology: enabled
- Intel VT-d: enabled
Here are some useful configuration information:
- PVE version:
Code:root@radix:~# pveversion --verbose proxmox-ve: 7.4-1 (running kernel: 5.15.102-1-pve) pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a) pve-kernel-5.15: 7.3-3 pve-kernel-5.15.102-1-pve: 5.15.102-1 ceph-fuse: 15.2.17-pve1 corosync: 3.1.7-pve1 criu: 3.15-1+pve-1 glusterfs-client: 9.2-1 ifupdown2: 3.1.0-1+pmx3 ksm-control-daemon: 1.4-1 libjs-extjs: 7.0.0-1 libknet1: 1.24-pve2 libproxmox-acme-perl: 1.4.4 libproxmox-backup-qemu0: 1.3.1-1 libproxmox-rs-perl: 0.2.1 libpve-access-control: 7.4-1 libpve-apiclient-perl: 3.2-1 libpve-common-perl: 7.3-3 libpve-guest-common-perl: 4.2-4 libpve-http-server-perl: 4.2-1 libpve-rs-perl: 0.7.5 libpve-storage-perl: 7.4-2 libspice-server1: 0.14.3-2.1 lvm2: 2.03.11-2.1 lxc-pve: 5.0.2-2 lxcfs: 5.0.3-pve1 novnc-pve: 1.4.0-1 proxmox-backup-client: 2.3.3-1 proxmox-backup-file-restore: 2.3.3-1 proxmox-kernel-helper: 7.4-1 proxmox-mail-forward: 0.1.1-1 proxmox-mini-journalreader: 1.3-1 proxmox-widget-toolkit: 3.6.3 pve-cluster: 7.3-3 pve-container: 4.4-3 pve-docs: 7.4-2 pve-edk2-firmware: 3.20221111-1 pve-firewall: 4.3-1 pve-firmware: 3.6-4 pve-ha-manager: 3.6.0 pve-i18n: 2.11-1 pve-qemu-kvm: 7.2.0-8 pve-xtermjs: 4.16.0-1 qemu-server: 7.4-2 smartmontools: 7.2-pve3 spiceterm: 3.2-2 swtpm: 0.8.0~bpo11+3 vncterm: 1.7-1 zfsutils-linux: 2.1.9-pve1 - Bootloader kernel command line config file:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt initcall_blacklist=sysfb_init" - Command
dmesg | grep -e DMAR -e IOMMUreturned:
Code:root@radix:~# dmesg | grep -e DMAR -e IOMMU [ 0.015152] ACPI: DMAR 0x00000000BACE4AB8 0000BC (v01 A M I OEMDMAR 00000001 INTL 00000001) [ 0.015174] ACPI: Reserving DMAR table memory at [mem 0xbace4ab8-0xbace4b73] [ 0.079065] DMAR: IOMMU enabled [ 0.219925] DMAR: Host address width 46 [ 0.219926] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1 [ 0.219932] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0466 ecap f020de [ 0.219935] DMAR: RMRR base: 0x000000bb747000 end: 0x000000bb755fff [ 0.219937] DMAR: ATSR flags: 0x0 [ 0.219939] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0 [ 0.219942] DMAR-IR: IOAPIC id 0 under DRHD base 0xfbffc000 IOMMU 0 [ 0.219944] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0 [ 0.219945] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000 [ 0.219946] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 0.220315] DMAR-IR: Enabled IRQ remapping in x2apic mode [ 0.600369] DMAR: No SATC found [ 0.600372] DMAR: dmar0: Using Queued invalidation [ 0.601811] DMAR: Intel(R) Virtualization Technology for Directed I/O - Command
dmesg | grep 'remapping'returned:
Code:root@radix:~# dmesg | grep 'remapping' [ 0.219946] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 0.220315] DMAR-IR: Enabled IRQ remapping in x2apic mode - Content of the file
/etc/module:
Code:# /etc/modules: kernel modules to load at boot time.# # This file contains the names of kernel modules that should be loaded # at boot time, one per line. Lines beginning with "#" are ignored. vfio vfio_iommu_type1 vfio_pci vfio_virqfd - Content of the file
/etc/modprobe.d/blacklist.conf:
Code:blacklist amdgpublacklist radeon blacklist nouveau blacklist nvidia - Content of the file
/etc/modprobe.d/vfio.conf:
Code:options vfio-pci ids=1002:699f,1002:aae0 disable_vga=1 - If I do
lspci -vagain:
Code:03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X] (rev c7) (prog-if 00 [VGA controller]) Subsystem: Sapphire Technology Limited Lexa PRO [Radeon RX 550] Flags: bus master, fast devsel, latency 0, IRQ 255, IOMMU group 25 Memory at e0000000 (64-bit, prefetchable) [size=256M] Memory at f0000000 (64-bit, prefetchable) [size=2M] I/O ports at e000 [disabled] [size=256] Memory at fbd00000 (32-bit, non-prefetchable) [size=256K] Expansion ROM at fbd40000 [disabled] [size=128K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [58] Express Legacy Endpoint, MSI 00 Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [200] Physical Resizable BAR Capabilities: [270] Secondary PCI Express Capabilities: [2b0] Address Translation Service (ATS) Capabilities: [2c0] Page Request Interface (PRI) Capabilities: [2d0] Process Address Space ID (PASID) Capabilities: [320] Latency Tolerance Reporting Capabilities: [328] Alternative Routing-ID Interpretation (ARI) Capabilities: [370] L1 PM Substates Kernel driver in use: vfio-pci Kernel modules: amdgpu 03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] Subsystem: Sapphire Technology Limited Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] Flags: fast devsel, IRQ 255, IOMMU group 25 Memory at fbd60000 (64-bit, non-prefetchable) [disabled] [size=16K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [58] Express Legacy Endpoint, MSI 00 Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [328] Alternative Routing-ID Interpretation (ARI) Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel
Here is my Windows 11 virtual machine configuration:
Code:
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=virtio0;ide2;net0
cores: 6
cpu: host,hidden=1,flags=+pcid
efidisk0: local-lvm:vm-202-disk-0,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:04:00,pcie=1
ide2: local:iso/ubuntu-23.04-desktop-amd64.iso,media=cdrom,size=4816804K
machine: pc-q35-7.2
memory: 16384
meta: creation-qemu=7.2.0,ctime=1685816875
name: WIN-11
net0: virtio=AE:1A:06:02:97:49,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsihw: virtio-scsi-single
smbios1: uuid=70c306af-2628-413f-a244-b6a23053e0e5
sockets: 1
tpmstate0: local-lvm:vm-202-disk-1,size=4M,version=v2.0
virtio0: local-lvm:vm-202-disk-2,iothread=1,size=240G
vmgenid: f35ec065-880d-4d51-a18f-641e419e1a45What I tried:
- Install another OS:
- With Windows 10 it is the same
- With Ubuntu 23.04, the VM doesn't boot and display errors about PCI devices - Remove all my configurations and follow the official Proxmox wiki
- Install my GPU in another PCIe port
- Install the GPU in a physical machine on Windows 11 (the GPU is working, AMD drivers installed successfully)
I also provide 2 screenshots of the errors I have encountered respectively on Windows and at Ubuntu boot.
thanks in advance to those who will help me.
