[SOLVED] GPU pass-through is working but not as multi-function device.

[StarkWiz]

I didn't find an exact solution for this in the forum and it seems some others have noticed this issues as well but the solution isn't clearly mentioned.
My guest OS is Windows 10 Pro (Build 10586) and I am using ASUS STRIX GTX 970 with pass-through.

When I add below to the config only my GPU is added to the VM but it's not detected as multi-function device and I don't see it in the running qemu command line when I verified the command line using ps.

hostpci0: 02:00,pcie=1,x-vga=on

If I add the 2 lines below, I do get to use the HDMI audio but if you noticed it's added separately and not as multi-function which ends up using available pcie pass-through slots which is max 4 in default PVE configuration.

hostpci0: 02:00.0,pcie=1,x-vga=on
hostpci1: 02:00.1,pcie=1

Below is my config file.

agent: 1
balloon: 0
bios: ovmf
boot: c
bootdisk: scsi0
cores: 12
cpu: host,hidden=1
hostpci0: 02:00,pcie=1,x-vga=on
hostpci3: 00:14.0,pcie=1
hostpci2: 00:1f.2,pcie=1
hotplug: disk,network,usb
hugepages: 2
ide2: local:iso/virtio-win-0.1.118.iso,media=cdrom,size=55632K
machine: q35
memory: 16384
name: W10PRO
net0: bridge=vmbr0,virtio=32:35:35:65:31:32
numa: 1
ostype: win8
scsi0: /dev/disk/by-id/ata-ST95005620AS_5YX0VE3K,cache=writeback,size=488386584K
scsihw: virtio-scsi-pci
shares: 0
smbios1: uuid=eeea7cb7-5855-4095-80aa-9bfb79e20db3
sockets: 1
tablet: 0

I am using PVE 4.2 with latest updates from Non-Subscription repository.

Can someone please help me add GTX 970 as multi-function device with Q35 machine type ?

Thanks.

 

[StarkWiz]

Bump.. I hope someone can help me. Thanks.

 

[dcsapak]

hi, try the following line:

hostpci0: 02:00.0;02:00.1,pcie=1,x-vga=on

 

[StarkWiz]

Hi Dominik,

Thanks and yes that works I figured it when I went through the qemuserver.pm perl code while ago.
It's strange that this is not mentioned anywhere.

I'll mark this thread as solved.

 

[dcsapak]

well it is mentioned in the man page:

man qm

and http://pve.proxmox.com/wiki/Manual:_vm.conf

 

[StarkWiz]

Thanks probably I was looking too hard for examples and use of word "multi-function". So, I didn't try to understand the syntax.

 

[dcsapak]

yes maybe we should include an example for this...

 

[spirit]

hi, try the following line:

hostpci0: 02:00.0;02:00.1,pcie=1,x-vga=on

I thinked that proxmox code will include all functions with a simple

hostpci0: 02:00,pcie=1,x-vga=on

see my original commit from 2014
https://git.proxmox.com/?p=qemu-ser...;hpb=2e3b7e2a3a9226b7942668cb9ebeecb6c9f289fd

Maybe it's a regression ?
the wiki is not ok :
http://pve.proxmox.com/wiki/Pci_passthrough

 

[StarkWiz]

Hi Spirit,

I was hoping it would work like that but it doesn't, I am not sure if this is related to pcie and q35 machine type.
I can confirm it's working correctly, when we specify all the multi-function devices using semi-colon.

Is the code from 2014 you mentioned part of latest updates from Non-Subscription repository ?

I'll compare with the existing code when I have some free time and let you know.

 

[spirit]

Hi Spirit,

I was hoping it would work like that but it doesn't, I am not sure if this is related to pcie and q35 machine type.
I can confirm it's working correctly, when we specify all the multi-function devices using semi-colon.

Is the code from 2014 you mentioned part of latest updates from Non-Subscription repository ?

I'll compare with the existing code when I have some free time and let you know.

I think the change is here:
https://git.proxmox.com/?p=qemu-ser...;hpb=cd9c34d1866bd7c3a942b73f2dcae46df5f24c24

- if(defined($4)){
- push @{$res->{pciid}}, { id => $2 , function => $4};
-
- }else{
- my $pcidevices = lspci($2);
- $res->{pciid} = $pcidevices->{$2};
- }

They was a lspci which try to find functions if they was not defined

 

[StarkWiz]

Yes, they changed the code in April 2016.
property string update: hostpci*
This commit changes the listing of virtual functions from multiple host= entries to one semicolon-separated host list.

It seems the PCI Passthrough wiki page hasn't been updated with latest syntax and multi-function examples.

 

[spirit]

Yes, they changed the code in April 2016.
property string update: hostpci*
This commit changes the listing of virtual functions from multiple host= entries to one semicolon-separated host list.

It seems the PCI Passthrough wiki page hasn't been updated with latest syntax and multi-function examples.

I have updated the wiki with new syntax

 

[wbumiller]

Should we revert to the old behavior? The behavioral change was not intentional.

 

[spirit]

Should we revert to the old behavior? The behavioral change was not intentional.

maybe we can readd the old behavior, it's a simply lspci which try to find functions, if no function is defined.
(I have seen some forum questions related to this change last months)

 

[StarkWiz]

Thanks @spirit for updating the wiki.

I would really like if it works like before as it's more easier, unless it's known to cause any issues with latest qemu updates or anything else.
I have one query about the old behavior though. I am not quite sure how many virtual functions can one device have.
But what if there is a device with say 3 or 4 virtual functions and I want to passthrough only 2 of them say .0 and .3
Is this possible with old behavior ? If not, then this recent code update does provide more flexibility.
It would be also great if previous behavior can be retained alongwith this one instead of reverting.

Just FYI, this is how the arguments in qemu command line look with multifunction.

-device vfio-pci,host=02:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on -device vfio-pci,host=02:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1