Does pve 8.3 support rtx5090 passthrough?

[simon11118]

Does anyone know about this or tried successfully?
We are building a small AI private cloud platform for a school, and they want to use rtx5090 in the VM.
Please advice.

 

[Curt Hall]

Its generally considered not advisable to use a "gamer" GPU card for a passthrough-to-vm AI card. Even it it connects to the VM, its still just a 5090, your AI will be the slowest thing you ever saw. We have done this trying to use various cards, some Nvidia (gamer) style and some Tesla cards (enterprise) AI ready ones. Long story short: you can do it but don't be surprised if its nothing but a headache (like when the VM just happens to lose connection to the pass through for no good reason, sometimes resulting in needing to reboot Proxmox host to ultimately correct). NOTE TO ADMINS: There should be a banner at the top of this forum just telling everyone using Proxmox to "NOT use consumer grade hardware for enterprise level processes just because some gamer said it got 10 more FPS in his favorite game." Just saying.

 

[alexskysilk]

Its generally considered not advisable to use a "gamer" GPU card for a passthrough-to-vm AI card. Even it it connects to the VM, its still just a 5090, your AI will be the slowest thing you ever saw.

The admonition against using gamer cards is both because they are usually very poorly provisioned with VRAM, AND mechanically not suitable for cooling in a server chassis. As long as OPs models fit in the vram (which on a 5090 is a not inconsiderable 32GB) I dont see why not.

Having said that- without very careful cpu pinning you WILL get better performance running you AI load on the HOST, not the guest just due to cpu/ram not having direct access to the gpu. You can still run virtualization loads besides, just not using the GPU.

 

[Curt Hall]

Does anyone know about this or tried successfully?
We are building a small AI private cloud platform for a school, and they want to use rtx5090 in the VM.
Please advice.

Sorry @simon11118 if not helpful, by steering you away from passing through non-vGPU's as if they were vGPU's, but @alexskysilk is right, it would be better on physical box for sure.

 

[simon11118]

Sorry @simon11118 if not helpful, by steering you away from passing through non-vGPU's as if they were vGPU's, but @alexskysilk is right, it would be better on physical box for sure.

Thanks for your reply, let me check it with client.

 

[frankmanzhu]

There is nothing wrong with using consumer hardware for Proxmox. I've been using consumer hardware on proxmox for years (my workstation with multiple GPUs)
It has been rock solid. Without ECC ram and vram, you'll basically just need to put your exception right. (accept once a week or month restart)

Generally Nvidia GPU works out of the box to passthrough (even proxmox doesn't have a valid driver). You just need to disable loading on the host and directly passthrough to the VM for driver. However, I normally use 3 years old hardware for better support.

 

[simon11118]

There is nothing wrong with using consumer hardware for Proxmox. I've been using consumer hardware on proxmox for years (my workstation with multiple GPUs)
It has been rock solid. Without ECC ram and vram, you'll basically just need to put your exception right. (accept once a week or month restart)

Generally Nvidia GPU works out of the box to passthrough (even proxmox doesn't have a valid driver). You just need to disable loading on the host and directly passthrough to the VM for driver. However, I normally use 3 years old hardware for better support.

Thanks, you gave me strong confidence to explore it deeply.

 

[Curt Hall]

Thanks, you gave me strong confidence to explore it deeply.

@frankmanzhu not wrong per se, so don't confuse my point of view with his, I maintain multiple VM's per host (60-100) and cannot afford to have vGPU tripping my host out. but if @simon11118 has only the one or very few other VM's on that host it could be worth a shot, my environment just can't tolerate once a week reboots or the random weirdness it brings, I have tested with cheap $200 cards all the way up to $5500 enterprise TESLA A16 card.