PCIe passthrough does not work

E

efeu

Renowned Member
Proxmox Subscriber
Nov 6, 2015
91
10
73
Workaround:

  1. Install Debian Jessie
  2. Add Proxmox Repositories as per wikihttps://pve.proxmox.com/wiki/Install..._Debian_Jessie
  3. Set Debian Kernel 3.16 as default (or select it during boot in grub menu)
  4. Enable PCIE Passthrough as per wikihttps://pve.proxmox.com/wiki/Pci_passthrough
  5. Use OMVF instead of SeaBios on VM
  6. PCIE-Passthrough works (also with GPUS)
Old Post:
Hi,
I try to switch from Ubuntu Server 14.04 to proxmox, but I cant get pcie vga passthrough to work.
Just to mention again it runs well under ubuntu, so it should not be a hardware issue.
my system:
cpu: i7-4970k
proxmox 4.2.2-1-pve
host graphics: intel hd4600
guest graphics: nvidia gtx970
Code: 
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 rootdelay=10 scsi_mod.scan=sync"
And yes I did update-grub
Code: 
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
pci_stub
vfio
vfio_iommu_type1
vfio_pci
kvm
kvm_intel
/etc/initramfs-tools/modules
Code: 
# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod
pci_stub ids=10de:13c2,10de:0fbb
100.conf
Code: 
bootdisk: virtio0
cores: 4
cpu: host
hostpci1: 01:00,x-vga=on,pcie=1,driver=vfio
ide2: local:iso/de_windows_10_multiple_editions_x64_dvd_6846954.iso,media=cdrom
kvm: 1
machine: q35
memory: 10000
name: win10
net0: e1000=C2:9D:85:B8:AA:47,bridge=vmbr0
numa: 0
ostype: other
smbios1: uuid=6ee35a46-4266-44a8-8ac2-478ecc811cbe
sockets: 1
usb0: host=046d:c051
usb1: host=413c:2005
vga: none
virtio0: local:100/vm-100-disk-1.raw,discard=on,size=250G
dmesg | grep -e DMAR -e IOMMU
Code: 
[    0.000000] ACPI: DMAR 0x00000000BD5F85E0 0000B8 (v01 INTEL  BDW      00000001 INTL 00000001)
[    0.000000] DMAR: IOMMU enabled
[    0.032062] DMAR: Host address width 39
[    0.032064] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.032070] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.032071] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.032074] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da
[    0.032074] DMAR: RMRR base: 0x000000bdea8000 end: 0x000000bdeb6fff
[    0.032075] DMAR: RMRR base: 0x000000bf000000 end: 0x000000cf1fffff
[    0.032077] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
[    0.032077] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.032078] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.032078] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.032280] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.421076] DMAR: No ATSR found
[    0.421140] DMAR: dmar0: Using Queued invalidation
[    0.421184] DMAR: dmar1: Using Queued invalidation
[    0.421190] DMAR: Setting RMRR:
[    0.421197] DMAR: Setting identity map for device 0000:00:02.0 [0xbf000000 - 0xcf1fffff]
[    0.422231] DMAR: Setting identity map for device 0000:00:14.0 [0xbdea8000 - 0xbdeb6fff]
[    0.422249] DMAR: Setting identity map for device 0000:00:1a.0 [0xbdea8000 - 0xbdeb6fff]
[    0.422264] DMAR: Setting identity map for device 0000:00:1d.0 [0xbdea8000 - 0xbdeb6fff]
[    0.422275] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.422280] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.422325] DMAR: Intel(R) Virtualization Technology for Directed I/O
[   11.399914] [drm] DMAR active, disabling use of stolen memory
lspci -nnk
Code: 
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
    Subsystem: Gigabyte Technology Co., Ltd Device [1458:367a]
    Kernel driver in use: vfio-pci
01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
    Subsystem: Gigabyte Technology Co., Ltd Device [1458:367a]
    Kernel driver in use: vfio-pci
/etc/modprobe.d/iommu_unsafe_interrupts.conf
Code: 
options vfio_iommu_type1 allow_unsafe_interrupts=1
when I run 'qm start 100' I get:
Code: 
proxmox:~# qm start 100
unknown hostpci setting 'driver=vfio'
unknown hostpci setting 'driver=vfio'
unknown hostpci setting 'driver=vfio'
Running as unit 100.scope.
unknown hostpci setting 'driver=vfio'
Status in webfrontend says OK, but forsure nothing happens.

On my ubuntu server I additionaly use this grub flag, and use OVMF for the VM:
Code: 
i915.enable_hd_vgaarb=1
But it does not change anything in proxmox with or w/o this flag.

I know there are two kernel patches mentioned in the arch-forum, so called "ACS override patch" and "i915 VGA arbiter patch", do I need this patches?, if yes how do I apply them to the proxmox kernel, sorry I'm not familiar in patching kernels.

Any sugestions?

Regards!

Edit:
Code: 
pveversion -v
proxmox-ve: 4.0-16 (running kernel: 4.2.2-1-pve)
pve-manager: 4.0-57 (running version: 4.0-57/cc7c2b53)
pve-kernel-4.2.2-1-pve: 4.2.2-16
lvm2: 2.02.116-pve1
corosync-pve: 2.3.5-1
libqb0: 0.17.2-1
pve-cluster: 4.0-24
qemu-server: 4.0-35
pve-firmware: 1.1-7
libpve-common-perl: 4.0-36
libpve-access-control: 4.0-9
libpve-storage-perl: 4.0-29
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.4-12
pve-container: 1.0-21
pve-firewall: 2.0-13
pve-ha-manager: 1.0-13
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.4-3
lxcfs: 0.10-pve2
cgmanager: 0.39-pve1
criu: 1.6.0-1
zfsutils: 0.6.5-pve6~jessie
 
Last edited:
I was going to write the exact same post.

I would just add that console show "This VM has no graphic display device."
 
OK, I'm on my way at this problem. It's related to the bios proxmox is using. With the GTX970 as vga device the VM does not start. I'm using current OVMF EFI build and it works.

Installation and booting into Windows needs a lot of "VM reboots" until it finaly boots up correctly. Current OVMF build also has a lot of problems with the proxmox implementation of I/O devices. The only one worked for me was SATA...

Does proxmox serve the capability to use some implemented efi boot instead of seabios (or whatever proxmox uses)?? Or is there any save method of using OVMF in proxmox, I'm not verry happy with the way I'm using it.

For those who have similar problems here my config:
Code: 
args: -cpu host,kvm=off -bios /var/lib/vz/images/100/OVMF.fd -device virtio-scsi-pci,id=scsi -drive file=/var/lib/vz/template/iso/de_windows_10_multiple_editions_x64_dvd_6846954.iso,id=iso_install,if=none -device scsi-cd,drive=iso_install -cdrom /var/lib/vz/template/iso/virtio-win.iso
cores: 8
cpu: host
hostpci0: 01:00,x-vga=on,pcie=1
hostpci1: 00:14.0,pcie=2
machine: q35
memory: 10000
name: win10
net0: virtio=26:F5:9A:85:25:61,bridge=vmbr0
numa: 0
ostype: win8
sata0: local:100/vm-100-disk-1.raw,cache=writeback,discard=on,size=250G
smbios1: uuid=76521a3b-440f-4ccb-980a-8f85d6043e98
sockets: 1
usb0: host=046d:c051
usb1: host=413c:2005
usb2: host=1-2

But just to mention again, this is far away from beeing stable or solved.
 
I really hope that it's possible to -- and it was my understanding that it was before -- do gpu passthrough with SeaBios in Proxmox because in my case I will need it instead of OVMF.

Envoyé de mon TouchPad en utilisant Tapatalk
 
jpbaril said:
I really hope that it's possible to -- and it was my understanding that it was before -- do gpu passthrough with SeaBios in Proxmox because in my case I will need it instead of OVMF.

Envoyé de mon TouchPad en utilisant Tapatalk
Click to expand...


Why?

Also I use PCIE passthrough and noticed that on Proxmox it didn't work, would using QVMP fix the issue? I really would like to get it working ASAP.
 
haxxa said:
Why?
Click to expand...

Because my ultimate goal would be to run a virtualized OS X desktop instead of Windows and for now virtualized OS X does not work with OVMF, only SeaBios.

I don't understand why GPU Passthrough is not working on Proxmox 4.0 as it seamed to work before as per the Proxmox forum threads and wiki I read. There are also numerous articles on the web indicating that passing though GPU with KVM is now common.
 
jpbaril said:
Because my ultimate goal would be to run a virtualized OS X desktop instead of Windows and for now virtualized OS X does not work with OVMF, only SeaBios.

I don't understand why GPU Passthrough is not working on Proxmox 4.0 as it seamed to work before as per the Proxmox forum threads and wiki I read. There are also numerous articles on the web indicating that passing though GPU with KVM is now common.
Click to expand...

Wow your really doing something outside the books by visualizing OSX with PCIE Passthrough, but fair enough - hopefully passthrough can get fixed. I really need it too.
 
spirit said:
We have already applied this fix (it revert b8eb5512fd8a115f164edbbe897cdf8884920ccb) on current proxmox 4.0 qemu
Click to expand...

Ok my apologies, similar issue so I thought might be the reason, obviously not then :/
 
With the mentioned Kernel it's even more broke. Booting with Seabios results in black screen, hanging VM with permanent 25% CPU load, like in the 4.2 kernel. OVMF UEFI instantly crashes, and VM is freezed also with permanent 25% CPU load. At least it shows OVMF first screen, but forsure hanging.
GPU Passthrough seems to be realy hard broken on proxmox at this moment.
 
You mentioned using IGP + Nvidia GPU.

Have you tried with e.g. No IGP + Nvidia or maybe even ATI only ?

I don't have the time right now to get this working on my personal Homelab. Might help to narrow down wether this is an issue related to the used GPU's
 
jpbaril said:
Because my ultimate goal would be to run a virtualized OS X desktop instead of Windows and for now virtualized OS X does not work with OVMF, only SeaBios.

I don't understand why GPU Passthrough is not working on Proxmox 4.0 as it seamed to work before as per the Proxmox forum threads and wiki I read. There are also numerous articles on the web indicating that passing though GPU with KVM is now common.
Click to expand...

I know this is a bit off topic, but have you ever had success with recent versions of OS X running under proxmox (yosemite/el capitan)? I tried about a year ago and failed miserably. I've since resorted to running my os x VM under vmware workstation which really sucks. Any info would be greatly appreciated!
 
bleomycin said:
I know this is a bit off topic, but have you ever had success with recent versions of OS X running under proxmox (yosemite/el capitan)? I tried about a year ago and failed miserably. I've since resorted to running my os x VM under vmware workstation which really sucks. Any info would be greatly appreciated!
Click to expand...

Hint: It's KVM so follow one of the guides online and set a few settings in config pertaining to the CPU. Finally Vanilla OSX is a pain to deal with using a distribution helps vastly - no need for fakesmc and annoying config settings.

But back to VM passthrough, if QVFM works how can I get it setup now?
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom