VGA passthrough on consumer hardware

S

SeerOfVoid

New Member
Aug 4, 2014
8
0
1
I'm running a PVE host on consumer grade hardware and can't get VFIO VGA passthrough to work right.

CPU: Core i7-950
Mobo: Asus P6X58D-Premium
RAM: 10GB non ECC
GPU: Nvidia Quadro FX 1800

Here is the Guest OS hardware

Code: 
 cat /etc/pve/nodes/VMnode0/qemu-server/100.confballoon: 64
bootdisk: virtio0
cores: 2
cpu: host
memory: 3072
name: devmachine
net0: virtio=AA:1E:18:73:AB:80,bridge=vmbr0
ostype: l26
sockets: 1
virtio0: lvm0:vm-100-disk-1,size=60G
machine: q35
hostpci0: 02:00.0,pcie=1,driver=vfio

Here's the error I get when trying to start the VM

Code: 
[COLOR=#000000][FONT=tahoma]kvm: -device vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio: failed to set iommu for container: Operation not permitted[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]kvm: -device vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio: failed to setup container for group 16[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]kvm: -device vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: vfio: failed to get group 16[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]kvm: -device vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Device initialization failed.[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]kvm: -device vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0: Device 'vfio-pci' could not be initialized[/FONT][/COLOR]
[COLOR=#000000][FONT=tahoma]TASK ERROR: start failed: command '/usr/bin/kvm -id 100 -chardev 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' -mon 'chardev=qmp,mode=control' -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -name devmachine -smp 'sockets=1,cores=2' -nodefaults -boot 'menu=on' -vga cirrus -cpu host,+x2apic -k en-us -m 3072 -readconfig /usr/share/qemu-server/pve-q35.cfg -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' -device 'vfio-pci,host=04:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0' -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' -iscsi 'initiator-name=iqn.1993-08.org.debian:01:804b6a81ca7e' -drive 'file=/dev/lvm0/vm-100-disk-1,if=none,id=drive-virtio0,aio=native,cache=none' -device 'virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100' -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' -device 'virtio-net-pci,mac=AA:1E:18:73:AB:80,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' -machine 'type=q35'' failed: exit code 1[/FONT][/COLOR]

Any idea what I'm doing wrong?
 
dietmar said:
Please test with latest kernel from pvetest:

http://download1.proxmox.com/debian...4/pve-kernel-3.10.0-3-pve_3.10.0-13_amd64.deb
Click to expand...

I noticed a contradiction here:

- according to http://ark.intel.com/compare/52583,37150 i7-950 does not support VT-d

- The motherboard (obviously) supports VT-d

- the reference list http://support.asus.com/cpusupport/list.aspx?SLanguage=en&m=P6X58D Premium&p=1 shows only CPUs without VT-d support (according to http://ark.intel.com/Search/Advanced?s=t&ProcessorNumber=i7&SocketsSupported=FCLGA1366&VTD=true )

Maybe I misunderstand something, is it possible that

- VT-d works without "CPU support" if you have the proper kernel?

- the data in ark are wrong/incomplete?
 
Last edited:
I just updated to kernel 3.10.0-4-pve from the pvetest repo and am going to give VGA passthrough another shot, but between posts I did successfully get it working on some older 3.10 kernel. This time I manually downloaded all packages and dependencies until apt was satisfied and also fixed the problem in my other thread, which was accessing the web gui.
 
MY attempt at VGA passthrough with Windows 7 as guest os and an AMD Radeon card was completely successful, I'm now waiting for updates to install and then the GPU driver. Hopefully I won't get a BSOD :p

Here's some more info for future reference. It might help someone else later.

My <vmid>.conf
Code: 
balloon: 2bootdisk: virtio0
cores: 3
cpu: host
memory: 2048
name: windoge7
net0: virtio=A6:C9:F0:0B:D4:64,bridge=vmbr0
ostype: win7
scsihw: virtio-scsi-pci
smbios1: uuid=58446416-70a8-4df1-b027-750787c1e709
sockets: 1
virtio0: lvm0:vm-100-disk-1,size=54G
machine: q35
hostpci0: 04:00.0,x-vga=on,pcie=1,driver=vfio
usb0: host=8-1
usb1: host=8-2

My /etc/default/grub

Code: 
# If you change this file, run 'update-grub' afterwards to update# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'


GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pci-stub.ids=1002:68f9"
GRUB_CMDLINE_LINUX=""


# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"


# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console


# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480


# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true


# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"


# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

And where 04:00.0 is, you replace that with the corresponding ID for your GPU from lspci. For the "pci-stub.ids" you need to do lspci -n, and find your GPU's ID, and grab the "xxxx:xxxx" from the end of the line.

I'm overall very satisfied with Proxmox now that I got everything working! I even went and set up an LACP bond for the double-NICs on my mobo. I've had no other issues with the new kernel or any of the latest packages.


TL;DR
It works great!
 
Mr.Holmes said:
I noticed a contradiction here:

- according to http://ark.intel.com/compare/52583,37150 i7-950 does not support VT-d

- The motherboard (obviously) supports VT-d

- the reference list http://support.asus.com/cpusupport/list.aspx?SLanguage=en&m=P6X58D Premium&p=1 shows only CPUs without VT-d support (according to http://ark.intel.com/Search/Advanced?s=t&ProcessorNumber=i7&SocketsSupported=FCLGA1366&VTD=true )

Maybe I misunderstand something, is it possible that

- VT-d works without "CPU support" if you have the proper kernel?

- the data in ark are wrong/incomplete?
Click to expand...

You are right about the ARK being wrong. The entire 1st gen Core i series CPUs don't have a VT-d listing (rather than having one and saying yes or no), yet here I am passing through VGA on one. Also, the X58 Asus motherboards have incorrect DMAR tables, but the latest kernel seems to not throw those errors like the 2.6 one did, or I did something differently. Nevertheless, it all works, so it's irrelevant to me. ;)
 
SeerOfVoid said:
MY attempt at VGA passthrough with Windows 7 as guest os and an AMD Radeon card was completely successful, I'm now waiting for updates to install and then the GPU driver. Hopefully I won't get a BSOD :p

Here's some more info for future reference. It might help someone else later.

My <vmid>.conf
Code: 
balloon: 2bootdisk: virtio0
cores: 3
cpu: host
memory: 2048
name: windoge7
net0: virtio=A6:C9:F0:0B:D4:64,bridge=vmbr0
ostype: win7
scsihw: virtio-scsi-pci
smbios1: uuid=58446416-70a8-4df1-b027-750787c1e709
sockets: 1
virtio0: lvm0:vm-100-disk-1,size=54G
machine: q35
hostpci0: 04:00.0,x-vga=on,pcie=1,driver=vfio
usb0: host=8-1
usb1: host=8-2

My /etc/default/grub

Code: 
# If you change this file, run 'update-grub' afterwards to update# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'


GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Proxmox Virtual Environment"
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pci-stub.ids=1002:68f9"
GRUB_CMDLINE_LINUX=""


# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"


# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console


# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480


# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true


# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"


# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

And where 04:00.0 is, you replace that with the corresponding ID for your GPU from lspci. For the "pci-stub.ids" you need to do lspci -n, and find your GPU's ID, and grab the "xxxx:xxxx" from the end of the line.

I'm overall very satisfied with Proxmox now that I got everything working! I even went and set up an LACP bond for the double-NICs on my mobo. I've had no other issues with the new kernel or any of the latest packages.


TL;DR
It works great!
Click to expand...

Please report if it works with your Radeon because i had a BSOD with my HD 7770 on catalyst drivers....

will try a NVIDIA 750TI right now
 
mcflym said:
Please report if it works with your Radeon because i had a BSOD with my HD 7770 on catalyst drivers....

will try a NVIDIA 750TI right now
Click to expand...


During the Catalyst Installer the "AMD Drag and Drop Transcoding" failed but the Display Drivers and Catalyst Control Center worked fine. The card is a Radeon HD 5450/6350 (The PCI Device ID is 5450 but the AMD program says its a 6350, and so does the sticker on it. :confused:).
I rebooted after the install and no errors or BSODs, and I'm going to try running Minecraft, but so far it seems to run fine.
 
You must log in or register to reply here.
Top Bottom