Hello,
I am trying to do GPU pass through on guest#2
CPU: AMD Threadripper 3970x
GPU: NVIDIA 2080 Ti Founders Edition Blower Style
MB: MSI TRX40 Pro Wifi
BIOS is set to be CSM+UEFI and install was successful. IOMMU and SVM set enable
Because booting from Proxmox install flash drive, pure UEFI will stuck after some command line prompts then the display goes fuzzy boken lines. (white background won't show up)
My SeaBIOS Win10 guest (guest#1) works fine, but my OVMF (q35) Win10 guest (guest#2) doesn't start at all.
The OVMF (guest#2) can boot once I change machine to i440fx from q35.
But starting VM with q35 will produce status "Error: start failed: QEMU exited with code 1"
At this time, PCI device is not added in Hardware Tab yet. (PS: adding PCI device also won't start VM, same error message)
I followed the steps described in this post (it has steps, easier to follow) and cross checked with Proxmox official guide.
https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/
https://pve.proxmox.com/wiki/Pci_passthrough
I did every steps EXCEPT this optional steps:
(steps that I didn't do)
1. GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb
ff,efifbff"
2.
A. Disabling the Framebuffer: video=vesafbff,efifbff
B. ACS Override for IOMMU groups: pcie_acs_override=downstream,multifunction
3.
Step 3: IOMMU interrupt remapping
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
Reason is that my "dmesg | grep 'remapping' output seems to be fine and Proxmox official guide seems to suggest "unsafe interrupts" is not necessary if the output is current.
I DID put all 4 ids of my GPU in there. the first two are GPU and Audio, the later two are NVIDIA USB.
echo "options vfio-pci ids=10de:1e04,10de:10f7,10de:1ad6,10de:1ad7 disable_vga=1"> /etc/modprobe.d/vfio.conf
Conf File
Shell
lspci -nn output attached (because it exceed 10000 character limit on this web page)
I am trying to do GPU pass through on guest#2
CPU: AMD Threadripper 3970x
GPU: NVIDIA 2080 Ti Founders Edition Blower Style
MB: MSI TRX40 Pro Wifi
BIOS is set to be CSM+UEFI and install was successful. IOMMU and SVM set enable
Because booting from Proxmox install flash drive, pure UEFI will stuck after some command line prompts then the display goes fuzzy boken lines. (white background won't show up)
My SeaBIOS Win10 guest (guest#1) works fine, but my OVMF (q35) Win10 guest (guest#2) doesn't start at all.
The OVMF (guest#2) can boot once I change machine to i440fx from q35.
But starting VM with q35 will produce status "Error: start failed: QEMU exited with code 1"
At this time, PCI device is not added in Hardware Tab yet. (PS: adding PCI device also won't start VM, same error message)
I followed the steps described in this post (it has steps, easier to follow) and cross checked with Proxmox official guide.
https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/
https://pve.proxmox.com/wiki/Pci_passthrough
I did every steps EXCEPT this optional steps:
(steps that I didn't do)
1. GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb
ff,efifbff"2.
A. Disabling the Framebuffer: video=vesafb
ff,efifbffB. ACS Override for IOMMU groups: pcie_acs_override=downstream,multifunction
3.
Step 3: IOMMU interrupt remapping
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
Reason is that my "dmesg | grep 'remapping' output seems to be fine and Proxmox official guide seems to suggest "unsafe interrupts" is not necessary if the output is current.
I DID put all 4 ids of my GPU in there. the first two are GPU and Audio, the later two are NVIDIA USB.
echo "options vfio-pci ids=10de:1e04,10de:10f7,10de:1ad6,10de:1ad7 disable_vga=1"> /etc/modprobe.d/vfio.conf
Code:
~/rom-parser# ./rom-parser /tmp/image.rom
Valid ROM signature found @0h, PCIR offset 170h
PCIR: type 0 (x86 PC-AT), vendor: 10de, device: 1e04, class: 030000
PCIR: revision 0, vendor revision: 1
Error, ran off the end
Code:
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
GRUB_CMDLINE_LINUX="root=ZFS=rpool/ROOT/pve-1 boot=zfs"
# Disable os-prober, it might add menu entries for each guest
GRUB_DISABLE_OS_PROBER=true
# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"
# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console
# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480
# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true
# Disable generation of recovery mode menu entries
GRUB_DISABLE_RECOVERY="true"
# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"Conf File
Code:
agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
balloon: 0
bios: ovmf
bootdisk: scsi0
cores: 64
cpu: host,hidden=1,flags=+pcid
efidisk0: local-nvme:vm-500-disk-1,size=1M
ide2: network-proxmox:iso/en_windows_10_consumer_editions_version_1909_x64_dvd_be09950e.iso,media=cdrom
ide3: network-proxmox:iso/virtio-win-0.1.185.iso,media=cdrom,size=402812K
machine: q35
memory: 32768
name: Win10Edu-1909-2080Ti
net0: virtio=3E:80:D5:FD:81:D2,bridge=vmbr0,firewall=1
numa: 1
ostype: win10
scsi0: local-nvme:vm-500-disk-0,cache=writeback,discard=on,size=501G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=51b7e8ff-ae21-45e2-942a-c236379bbd62
sockets: 1
vmgenid: c1d954f9-87b9-47cb-83f4-7a5505137199Shell
lspci -nn output attached (because it exceed 10000 character limit on this web page)
Code:
dmesg | grep 'remapping'
[ 0.851717] AMD-Vi: Interrupt remapping enabled
Code:
dmesg | grep -e DMAR -e IOMMU
[ 0.816040] pci 0000:60:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.816082] pci 0000:40:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.816106] pci 0000:20:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.816122] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.851700] pci 0000:60:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.851706] pci 0000:40:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.851710] pci 0000:20:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.851714] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.855517] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 0.855530] perf/amd_iommu: Detected AMD IOMMU #1 (2 banks, 4 counters/bank).
[ 0.855546] perf/amd_iommu: Detected AMD IOMMU #2 (2 banks, 4 counters/bank).
[ 0.855560] perf/amd_iommu: Detected AMD IOMMU #3 (2 banks, 4 counters/bank).
Last edited:
