Proxmox 7.1 and vGPU server

[azim.uiu]

I have a server with NVIDIA GPU and I tried to create a remote desktop with vGPU. I using NVIDIA Corporation GA102 [GeForce RTX 3090] GPU. But if I run this command it show some error,

root@ds1:~# ./NVIDIA-Linux-x86_64-470.82-vgpu-kvm.run -dkms


ERROR: Failed to run `/usr/sbin/dkms build -m nvidia -v 470.82 -k 5.15.39-3-pve`:
Kernel preparation unnecessary for this kernel. Skipping...

Building module:
cleaning build area...
'make' -j24 NV_EXCLUDE_BUILD_MODULES='' KERNEL_UNAME=5.15.39-3-pve IGNORE_CC_MISMATCH='' modules....(bad
exit status: 2)
Error! Bad return status for module build on kernel: 5.15.39-3-pve (x86_64)
Consult /var/lib/dkms/nvidia/470.82/build/make.log for more information.

ERROR: Failed to install the kernel module through DKMS. No kernel module was installed; please try installing
again without DKMS, or check the DKMS logs for more information.

ERROR: Installation has failed. Please see the file '/var/log/nvidia-installer.log' for details. You may find
suggestions on fixing installation problems in the README available on the Linux driver download page at
www.nvidia.com.


Please, anyone, help me to solve this problem?

 

[kenneth104]

I found this

https://www.reddit.com/r/Proxmox/comments/rxhx2e/deskpool_vdi_for_proxmox_with_vgpu/

 

[azim.uiu]

I already try to this way.. but the same error was found.

I found this

https://www.reddit.com/r/Proxmox/comments/rxhx2e/deskpool_vdi_for_proxmox_with_vgpu/

 

[1liminal1]

Did you get anywhere with this?

 

[trio198]

Same case.....
Is it even possible to share RTX 3090 to multiple VM's through Proxmox and vgpu?

Thanks.

 

[spirit]

Hi,

try:
https://gitlab.com/polloloco/vgpu-proxmox

 

[trio198]

Hi,

try:
https://gitlab.com/polloloco/vgpu-proxmox

Hi...thanks...will surely try this...

 

[trio198]

Please pardon my English. On the guide, says "
A few days ago, NVIDIA released their latest enterprise GRID driver. I created a patch that allows the use of most consumer GPUs for vGPU. One notable exception from that list is every officially unsupported Ampere GPU."

Does this mean that it will not work on Ampere (RTX 30 series) GPU?

 

[trio198]

Hi,

About my system :
Asus WRX80e-SAGE WIFI + AMD TR Pro + Dual Gigabyte RTX3090Ti

Followed above guide, managed to install Nvidia 510.85.03 GRID Driver on Proxmox VE.





But when installing guest Nvidia Driver, stuck in 5%.


mdevctl output:



Guest systems are ubuntu Desktop 22.04 and Ubuntu Server 22.04.

Another notes, every time restarting VMs, got "[nvidia-vgpu-vfio] 00000000-0000-0000-0000-000000000103: vGPU migration disabled".

Thank you.

 

[dcsapak]

But when installing guest Nvidia Driver, stuck in 5%.

did you wait enough ? it must compile some things, that can take a while (depending on the vm config). check e.g. 'ps ax' or 'top' output what is running/hanging.

Another notes, every time restarting VMs, got "[nvidia-vgpu-vfio] 00000000-0000-0000-0000-000000000103: vGPU migration disabled".

yes that's normal, live migration for linux kvm is currently not supported by nvidia it seems

 

[trio198]

did you wait enough ? it must compile some things, that can take a while (depending on the vm config). check e.g. 'ps ax' or 'top' output what is running/hanging.


yes that's normal, live migration for linux kvm is currently not supported by nvidia it seems

Hi,

Thanks for the response.
I've waited about 10 - 15 minutes. Will try to wait longer next time.

Thanks.

 

[trio198]

A little update.

Managed to install Nvidia regular quadro/rtx driver with GPU Spoofed to RTX 6000

Screenshot with GPU-Z...


But memory size and GPU Clock is 0.

I'm using nvidia-529, which should give 4GB of VRAM right?

 

[dcsapak]

I'm using nvidia-529, which should give 4GB of VRAM right?

according to the screenshot, 'framebuffer=8192M' which means 8GiB

 

[trio198]

according to the screenshot, 'framebuffer=8192M' which means 8GiB

Ow....I see.....thanks for the info. Should've looked at "nvidia-smi" output first.

But why in Windows VM, GPU-Z unable to read the memory amount and GPU clock?

BTW, in my Ubuntu VM, I've waited for more than 1 hour, but GRID Driver setup still stuck at 5% when building module.

ps-ef output.



Thank you.

 

[dcsapak]

But why in Windows VM, GPU-Z unable to read the memory amount and GPU clock?

i don't know, maybe it's related to the vgpu unlock (AFAIR this is only really tested with 20xx and 10xx cards?)

BTW, in my Ubuntu VM, I've waited for more than 1 hour, but GRID Driver setup still stuck at 5% when building module.

mhmm.. can you post a full 'ps ax' output?

 

[trio198]

i don't know, maybe it's related to the vgpu unlock (AFAIR this is only really tested with 20xx and 10xx cards?)


mhmm.. can you post a full 'ps ax' output?

Hi @dcsapak ,

Tried to do clean install of proxmox 7.2 and focusing on Windows 10 guest VM first.

Followed this guide :

https://gitlab.com/polloloco/vgpu-proxmox

And able to install patched 510.85.03 Nvidia driver on host (proxmox).
GRUB options :



On mainboard BIOS, set the default display to Internal VGA Adapter.


Windows VM setting


But cannot enable All-Functions options since it was greyed out.

In Windows Guest VM, recognized as RTX A6000-8Q (profile 529 in mdev).
And able to install using GRID driver 513.46.


After VM restart (as requested by NVidia GRID Driver), got Code 43.

Am I missing something during installation?

Thanks.

 

[dcsapak]

After VM restart (as requested by NVidia GRID Driver), got Code 43.

Am I missing something during installation?

not that i could see, but as i said is it verified that the unlock script works with 3000 series gpus?
e.g. the readme of your guide says:

I created a patch that allows the use of most consumer GPUs for vGPU. One notable exception from that list is every officially unsupported Ampere GPU.

 

[trio198]

not that i could see, but as i said is it verified that the unlock script works with 3000 series gpus?
e.g. the readme of your guide says:

Hi....yes, that's why I'm wondering, does it even works with Ampere since "nvidia-smi" and "mdevctl" manage to display vgpu profile.
So I'm pushing my luck here...:-D

Other thread saying that they are successfully use RTX 30 series with Proxmox, is it only for PCIE Passthrough and not vgpu?

Thanks.

 

[dcsapak]

Other thread saying that they are successfully use RTX 30 series with Proxmox, is it only for PCIE Passthrough and not vgpu?

which thread do you mean?

 

[trio198]

which thread do you mean?

Thread like this

https://forum.proxmox.com/threads/gpu-passthrough-throws-driver-error-43.109076/#post-494956

But then again, it's GPU Passthrough, not vgpu on 3090.

BTW, managed to do GPU passthrough with Proxmox 6.2, and will try to reinstall with 7.2.
Turns out, in my system BIOS, need to set CSM to Enable, and select "UEFI Only" on every options in CSM menus.
Previously, it was Disabled, and when creating new VM using OVMF Bios, always says "Guest has not initialized the display (yet)".

Thanks.