[SOLVED] Nvidia Tesla T4 with pci passthrough

[AxelTwin]

Hi everybody,

I am trying to setup a windows server VM with a nvidia tesla T4 card using pci passthrough.
I followed a few tutorials and everything seems to be ok but I am not sure it is working as it shoud.

When I try to open nvidia control panel int the vm I get the error message
“NVIDIA display settings are not available” you are not currently using a display attached to an NVIDIA GPU

On host server I get the following

root@hyperviser:~# modprobe nvidia -vv
modprobe: INFO: ../libkmod/libkmod.c:365 kmod_set_log_fn() custom logging function 0x55fbc6bb26f0 registered
modprobe: FATAL: Module nvidia not found in directory /lib/modules/5.13.19-2-pve

Thanks

 

[AxelTwin]

anyone ?

 

[NerdyNigel]

Have you followed the wiki?
https://pve.proxmox.com/wiki/Pci_passthrough

 

[leesteken]

If you do PCI passthrough, then the host Proxmox loses the connection to the device, so why are you trying to load a nvidia driver on the host?
Do you have a picture on a monitor connected to the GPU? Is there a physical monitor connected to the GPU? Or are you looking at Windows inside the VM using noVNC?
It might help if you share the VM configuration file from the /etc/pve/qemu-server/ directory on the Proxmox host.

 

[AxelTwin]

Thanks for your reply,
- So from what I understand, I don't need to load driver on the host and this is normal behavior
- There is no monitor connected to the GPU, I use RDP connexion directly to windows vm

I used those 2 tutos:
https://pve.proxmox.com/wiki/Pci_passthrough
and
https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

here is my vm config:

root@hyperviser:~# cat /etc/pve/qemu-server/100.conf
agent: 1
args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
bios: ovmf
boot: order=scsi0;ide2;net0;ide0
cores: 4
cpu: host,hidden=1,flags=+pcid
efidisk0: data1:vm-100-disk-1,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:5e:00,pcie=1,x-vga=1
ide0: local:iso/virtio-win-0.1.215.iso,media=cdrom,size=528322K
ide2: local:iso/SRV2K12.ISO,media=cdrom
machine: pc-q35-6.1
memory: 24576
meta: creation-qemu=6.1.0,ctime=1642087468
name: attic.eec31.local
net0: virtio=0A:C0:C8:75:50:A6,bridge=vmbr1,firewall=1
numa: 1
onboot: 1
ostype: win8
scsi0: data1:vm-100-disk-0,size=100G
scsihw: virtio-scsi-pci
smbios1: uuid=0bd28923-b705-40da-9544-9d52b5acc947
sockets: 4
vmgenid: a236e5a6-85ec-43ef-80dd-97d137940e04

How can I make sure the VM is using the TESLA4 GPU ?
There will be 3 users connected to windows VM through RDP connexion using 3D software, is that setup coherent ?

 

[leesteken]

I don't think you need args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off' because you already use x-vga=1.
Sorry, I have no experience with that GPU (or multi-user 3D software) but maybe the error message is correct because you have no display connected to the GPU?

 

[AxelTwin]

Ok, I disabled the options.
That's my guess, the message might be correct but I am unsure.
It would be great to be able to monitor if the GPU is correctly used while RDP users are using it

 

[kobemtl]

Ok, I disabled the options.
That's my guess, the message might be correct but I am unsure.
It would be great to be able to monitor if the GPU is correctly used while RDP users are using it

Hi, are you able to finnaly get T4 passthrough? What driver you using in the VM? Thanks.

 

[AxelTwin]

Hi,
Nope.
I had it mounted with pci passthrough but never managed to get it working with my 3D software.
I even tried with nvidia vws licence but nothing worked

 

[dorinxev]

There are 2 different things:

1. the PCI Passthrough - which in your case won't help much since this card is primary designed for "Virtual GPU" and not as a PCI Passthrough, so even if you'll pass it to a Windows VM, you'll need different drivers and vGPU license to make it work.
2. the vGPU - this is the recommended way with this card (see here how to make it work, you might need the nvidia docs in order to install the "server" part), and you'll need to purchase some licenses (depending how many VMs you'll use) from nVidia. There is a 90 days trial, and after installing the vGPU, you'll need to install on any VM the "client" driver.

All in all, if you just want to have a single VM with a GPU Passthrough, get yourself an RTX (or any Radeon) card and do a PCI passthrough this card. The T4 is clearly not meant for that.