VM hangs during boot if passthrough device is Nvidia Tesla K80

[vtrandal]

The subject says it all. We can speculate as to the cause: We think it is necessary to enable "above 4G decoding" in the OVMF bios. If that is the case then the question is: How can we enabled "above 4G decoding" or similar feature in the OVMF bios?

Otherwise, we believe the reason the VM cannot finish booting is because the K80 cannot get the PCI resources it needs unless "above 4G decoding" or similar feature is enabled in the OVMF bios.

Specifically, when attempting to start the VM using "qm start 112" there is no error message except on the console it says: "This VM has no graphic display device." That is the only message of any kind that we can see anywhere.

PCI passthrough works on this VM if the device is Nvidia Tesla M2090 as solved in previous thread:
https://forum.proxmox.com/threads/kvm-hardware-virtualization-affects-nvidia-smi.25873/

In the case of the M2090 we can use ssh to login into the guest operating system, but in the case of the K80 we get the following error message from ssh:
ssh: connect to host 192.168.1.65 port 22: No route to host

 

[spirit]

I don't known if it's related, but qemu have an option "max-ram-below-4G"

args: -machine pc,max-ram-below-4g=2G

http://git.qemu.org/?p=qemu.git;a=commit;h=c87b1520726f7ae1e698a41f07043d1b539ac88c

Also a reference here, about nvidia passthrough (max-ram-below-4g=1G)
https://www.altechnative.net/2015/0...rt-2-evolution-consolidation-and-move-to-kvm/

 

[vtrandal]

@spirit, you may have solved this problem for us with your reply. Thank you!
Your reply appears to address the issue and may lead to a solution or work around for us. Thanks again!
So then how would be go about setting those args in a Proxmox guest? Just add them to the config file in /etc/pve/<vmid>.conf?

 

[spirit]

@spirit,
So then how would be go about setting those args in a Proxmox guest? Just add them to the config file in /etc/pve/<vmid>.conf?

yes,

just put in vmid.conf

args: -machine pc,max-ram-below-4g=2G

 

[spirit]

If it's works, please send me the config in this post, I'll update the wiki

 

[spirit]

Hi, any news ? does it work ?

 

[T.Herrmann]

Was the problem solved?

args: -machine pc,max-ram-below-4g=2G

We are also looking for a solution for the Tesla K80. We transformed a HPC workstation with 4x Tesla K80 (8xK40) into a Proxmox Hypervisor but still need to use the Tesla K80 inside Win 10 Pro.

With OVMF we get just a half K80 (1xK40) online,
With SeaBIOS we get 2x K80 (4xK40) online

>> machine q35