Hello,
I've followed the wiki (https://pve.proxmox.com/wiki/Pci_passthrough). Everything checks out (Virtualization and VT-d) on in BIOS. But when I add, x-vga=on to the .conf, the guest machine fails to boot, but there are no errors that show up in the PVE logs and the physical monitor is just black. Without x-vga, I can the VNC console, but again the physical monitor is black. On boot of the physical host, the default Proxmox startup page is shown until the guest OS is activated, then it is just black.
I have tried both Nvidia Quadro 6000 and Quadro 4000 in both an HP Z800 and Z600, all workstation gear.
Proxmox 4, no-subscription repo, all updates
Shows up in qm monitor:
Bus 1, device 0, function 0:
VGA controller: PCI device 10de:06dd
IRQ 10.
BAR0: 32 bit memory at 0xfb000000 [0xfbffffff].
BAR1: 64 bit prefetchable memory at 0xe8000000 [0xefffffff].
BAR3: 64 bit prefetchable memory at 0xf0000000 [0xf1ffffff].
BAR5: I/O at 0x7000 [0x707f].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0007fffe].
id "hostpci0.0"
Bus 1, device 0, function 1:
Audio controller: PCI device 10de:0e09
IRQ 10.
BAR0: 32 bit memory at 0xfc080000 [0xfc083fff].
id "hostpci0.1"
Grub: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pci_stub ids=10de:06dd,10de:0e09"
initramfs-tools/modules: pci_stub ids=10de:06dd,10de:0e09
root@goblin:~# lspci -nn | grep NVIDIA
0f:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF100GL [Quadro 4000] [10de:06dd] (rev a1)
0f:00.1 Audio device [0403]: NVIDIA Corporation GF110 High Definition Audio Controller [10de:0e09] (rev a1)
Entry in my .conf:
hostpci0: 0f:00,pcie=1,multifunction=on,x-vga=on
machine: q35
I really really do not want to have to use ESXi for this.
root@goblin:/etc/pve/qemu-server# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x00000000DEFB8F83 000170 (v01 HPQOEM TYLSBURG 00000001 00000000)
[ 0.000000] DMAR: IOMMU enabled
[ 0.025459] DMAR-IR: This system BIOS has enabled interrupt remapping
[ 0.643886] DMAR: Host address width 40
[ 0.643888] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[ 0.643903] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f02076
[ 0.643905] DMAR: RMRR base: 0x000000defd0000 end: 0x000000defd0fff
[ 0.643906] DMAR: RMRR base: 0x000000defd1000 end: 0x000000defd1fff
[ 0.643907] DMAR: RMRR base: 0x000000defd2000 end: 0x000000defd2fff
[ 0.643908] DMAR: RMRR base: 0x000000defd3000 end: 0x000000defd3fff
[ 0.643909] DMAR: RMRR base: 0x000000defd4000 end: 0x000000defd4fff
[ 0.643911] DMAR: RMRR base: 0x000000defd5000 end: 0x000000defd5fff
[ 0.643912] DMAR: RMRR base: 0x000000defd6000 end: 0x000000defd6fff
[ 0.643913] DMAR: RMRR base: 0x000000defd7000 end: 0x000000defd7fff
[ 0.643914] DMAR: ATSR flags: 0x0
[ 0.644087] DMAR: dmar0: Using Queued invalidation
[ 0.644101] DMAR: Setting RMRR:
[ 0.644112] DMAR: Setting identity map for device 0000:00:1a.2 [0xdefd7000 - 0xdefd7fff]
[ 0.644143] DMAR: Setting identity map for device 0000:00:1a.1 [0xdefd6000 - 0xdefd6fff]
[ 0.644166] DMAR: Setting identity map for device 0000:00:1a.0 [0xdefd5000 - 0xdefd5fff]
[ 0.644190] DMAR: Setting identity map for device 0000:00:1d.2 [0xdefd4000 - 0xdefd4fff]
[ 0.644213] DMAR: Setting identity map for device 0000:00:1d.1 [0xdefd3000 - 0xdefd3fff]
[ 0.644235] DMAR: Setting identity map for device 0000:00:1d.0 [0xdefd2000 - 0xdefd2fff]
[ 0.644258] DMAR: Setting identity map for device 0000:00:1a.7 [0xdefd1000 - 0xdefd1fff]
[ 0.644281] DMAR: Setting identity map for device 0000:00:1d.7 [0xdefd0000 - 0xdefd0fff]
[ 0.644299] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.644307] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.644445] DMAR: Intel(R) Virtualization Technology for Directed I/O
I've followed the wiki (https://pve.proxmox.com/wiki/Pci_passthrough). Everything checks out (Virtualization and VT-d) on in BIOS. But when I add, x-vga=on to the .conf, the guest machine fails to boot, but there are no errors that show up in the PVE logs and the physical monitor is just black. Without x-vga, I can the VNC console, but again the physical monitor is black. On boot of the physical host, the default Proxmox startup page is shown until the guest OS is activated, then it is just black.
I have tried both Nvidia Quadro 6000 and Quadro 4000 in both an HP Z800 and Z600, all workstation gear.
Proxmox 4, no-subscription repo, all updates
Shows up in qm monitor:
Bus 1, device 0, function 0:
VGA controller: PCI device 10de:06dd
IRQ 10.
BAR0: 32 bit memory at 0xfb000000 [0xfbffffff].
BAR1: 64 bit prefetchable memory at 0xe8000000 [0xefffffff].
BAR3: 64 bit prefetchable memory at 0xf0000000 [0xf1ffffff].
BAR5: I/O at 0x7000 [0x707f].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0007fffe].
id "hostpci0.0"
Bus 1, device 0, function 1:
Audio controller: PCI device 10de:0e09
IRQ 10.
BAR0: 32 bit memory at 0xfc080000 [0xfc083fff].
id "hostpci0.1"
Grub: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pci_stub ids=10de:06dd,10de:0e09"
initramfs-tools/modules: pci_stub ids=10de:06dd,10de:0e09
root@goblin:~# lspci -nn | grep NVIDIA
0f:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF100GL [Quadro 4000] [10de:06dd] (rev a1)
0f:00.1 Audio device [0403]: NVIDIA Corporation GF110 High Definition Audio Controller [10de:0e09] (rev a1)
Entry in my .conf:
hostpci0: 0f:00,pcie=1,multifunction=on,x-vga=on
machine: q35
I really really do not want to have to use ESXi for this.
root@goblin:/etc/pve/qemu-server# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x00000000DEFB8F83 000170 (v01 HPQOEM TYLSBURG 00000001 00000000)
[ 0.000000] DMAR: IOMMU enabled
[ 0.025459] DMAR-IR: This system BIOS has enabled interrupt remapping
[ 0.643886] DMAR: Host address width 40
[ 0.643888] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[ 0.643903] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f02076
[ 0.643905] DMAR: RMRR base: 0x000000defd0000 end: 0x000000defd0fff
[ 0.643906] DMAR: RMRR base: 0x000000defd1000 end: 0x000000defd1fff
[ 0.643907] DMAR: RMRR base: 0x000000defd2000 end: 0x000000defd2fff
[ 0.643908] DMAR: RMRR base: 0x000000defd3000 end: 0x000000defd3fff
[ 0.643909] DMAR: RMRR base: 0x000000defd4000 end: 0x000000defd4fff
[ 0.643911] DMAR: RMRR base: 0x000000defd5000 end: 0x000000defd5fff
[ 0.643912] DMAR: RMRR base: 0x000000defd6000 end: 0x000000defd6fff
[ 0.643913] DMAR: RMRR base: 0x000000defd7000 end: 0x000000defd7fff
[ 0.643914] DMAR: ATSR flags: 0x0
[ 0.644087] DMAR: dmar0: Using Queued invalidation
[ 0.644101] DMAR: Setting RMRR:
[ 0.644112] DMAR: Setting identity map for device 0000:00:1a.2 [0xdefd7000 - 0xdefd7fff]
[ 0.644143] DMAR: Setting identity map for device 0000:00:1a.1 [0xdefd6000 - 0xdefd6fff]
[ 0.644166] DMAR: Setting identity map for device 0000:00:1a.0 [0xdefd5000 - 0xdefd5fff]
[ 0.644190] DMAR: Setting identity map for device 0000:00:1d.2 [0xdefd4000 - 0xdefd4fff]
[ 0.644213] DMAR: Setting identity map for device 0000:00:1d.1 [0xdefd3000 - 0xdefd3fff]
[ 0.644235] DMAR: Setting identity map for device 0000:00:1d.0 [0xdefd2000 - 0xdefd2fff]
[ 0.644258] DMAR: Setting identity map for device 0000:00:1a.7 [0xdefd1000 - 0xdefd1fff]
[ 0.644281] DMAR: Setting identity map for device 0000:00:1d.7 [0xdefd0000 - 0xdefd0fff]
[ 0.644299] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.644307] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.644445] DMAR: Intel(R) Virtualization Technology for Directed I/O