GPU Passthrough CPU Options

[ilikepizza]

Im using Proxmox 4.3-71 on a Dell R730. Besides a lot of CPU cores we also have 1x Asus 970 Strix GPU in the system and are planning on upgrading to 2x bigger GPU for more computation power. The GPU is passed through to a VM using PCI Passthrough.

The system was set up one year ago with GPU passthrough working nicely (both Windows and Linux). I came back a couple of days ago to see the GPU was no longer working properly inside the VM after updating to the newest version (don't remember the old version). The system would randomly crash the driver but 'nvidia-smi' was still working. Most CUDA samples would either run through a single time or block indefinitely till the VM was restarted. Driver version 370.28 with CUDA 8.

So first off, I managed to fix the problem. The error was in the CPU options:

Generated by Proxmox: -cpu host,+kvm_pv_unhalt,+kvm_pv_eoi,kvm=off (not working)

Working: -cpu host,kvm=off

Is there a way to disable the +kvm_pv_unhalt and +kvm_pv_eoi option in Proxmox for this single vm using the conf file or an interface provided by proxmox?

 

[dcsapak]

you can set this via the 'args' parameter, like so:

qm set VMID -args '-cpu host,kvm=off'

this overwrites then the cpu flag we generate

but could you post the ouptut of

pveversion -v

and the vm config?

 

[ilikepizza]

pveversion -v

proxmox-ve: 4.3-71 (running kernel: 4.4.21-1-pve)
pve-manager: 4.3-9 (running version: 4.3-9/f7c6f0cd)
pve-kernel-4.4.8-1-pve: 4.4.8-52
pve-kernel-4.4.21-1-pve: 4.4.21-71
pve-kernel-4.2.8-1-pve: 4.2.8-41
pve-kernel-4.2.2-1-pve: 4.2.2-16
pve-kernel-4.4.10-1-pve: 4.4.10-54
lvm2: 2.02.116-pve3
corosync-pve: 2.4.0-1
libqb0: 1.0-1
pve-cluster: 4.0-46
qemu-server: 4.0-92
pve-firmware: 1.1-10
libpve-common-perl: 4.0-79
libpve-access-control: 4.0-19
libpve-storage-perl: 4.0-68
pve-libspice-server1: 0.12.8-1
vncterm: 1.2-1
pve-docs: 4.3-12
pve-qemu-kvm: 2.7.0-4
pve-container: 1.0-80
pve-firewall: 2.0-31
pve-ha-manager: 1.0-35
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u2
lxc-pve: 2.0.5-1
lxcfs: 2.0.4-pve2
criu: 1.6.0-1
novnc-pve: 0.5-8
smartmontools: 6.5+svn4324-1~pve80
zfsutils: 0.6.5.8-pve13~bpo80

vm conf

args: -cpu host,kvm=off
balloon: 2048
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 8
cpu: host
hostpci0: 04:00,x-vga=on
ide2: file=none,media=cdrom
keyboard: en-us
machine: q35,max-ram-below-4g=1G
memory: 32768
name: Ubuntu-GPU
net0: virtio=CA:E3:1B:99:0D:2B,bridge=vmbr0
numa: 0
ostype: l26
smbios1: uuid=6523c594-bc98-4428-b1da-013d9e7eae7d
sockets: 1
virtio0: local:109/vm-109-disk-1.qcow2,size=50G

qm set VMID -args '-cpu host,kvm=off'

Results in

-cpu host,+kvm_pv_unhalt,+kvm_pv_eoi,kvm=off -cpu host,kvm=off

Which again does not work...

 

[dcsapak]

mhmm ok you could try the following:

qm set VMID -args '-cpu host,-kvm_pv_unhalt,-kvm_pv_eoi,kvm=off'

altough i could not verify if this works

could you alternatively try to change the line:

machine: q35,max-ram-below-4g=1G

to

machine: pc-q35-2.6,max-ram-below-4g=1G

?

edit: typo

 

[ilikepizza]

So this is fun.

qm set VMID -args '-cpu host,-kvm_pv_unhalt,-kvm_pv_eoi,kvm=off'

When starting:

kvm: CPU feature kvm_pv_eo not found

Also the machine settings don't have any influence. I played around with the parameters a bit. With only -kvm_pv_unhalt the VM starts but the GPU isn't working. The only way I can get it running is if I remove the generated cpu string and replace it with -cpu host,kvm=off

Thanks for your help!

 

[dcsapak]

When starting:

kvm: CPU feature kvm_pv_eo not found

please make sure the correct thing is in the config, i had a typo but edited it

 

[ilikepizza]

please make sure the correct thing is in the config, i had a typo but edited it

Sorry I missed that, but it doesn't seem to have an influence.. I guess I can live with writing a script to start the vm manually. But if the problem is the Nvidia driver detecting the virtualized system like the windows driver does, it might effect more people soon.

 

[eminempt]

hi guys, how are you in your lifes after 5 years? lol

what does this two arguments do anyway?
kvm_pv_unhalt,+kvm_pv_eoi

 

[yomi]

hi guys, how are you in your lifes after 5 years? lol

what does this two arguments do anyway?
kvm_pv_unhalt,+kvm_pv_eoi

Haha. I'm curious to know too.

I just applied it to my Windows 11 VM and it seems to make it work with cpu as host.