iadityaharsh

New Member
Jun 21, 2022
9
0
1
Can something like this be achieved with Proxmox?

Basic Idea:
1) One Mirror Pool with 3 ZVols, each containing a Windows 11 VM

2) Pass through 3 Nvidia A4000 to these three VMs respectively as the GPU

3) Output the VM i.e., Windows on an external Monitor connected to these GPUs

Plan to run 3 Windows machine on a single Proxmox Server.

Specs:
AMD Epyc 7402P (24C/48T)
Supermicro H12SSL-I
128GB DDR4 3200 MT/s ECC RDIMM

https://drive.google.com/file/d/1-NTLgBeDehSY-QO1EMLpy_1vDoohGqFu/view?usp=drivesdk
 
Best would be to get three USB controller PCIe cards and passthrough one of them to each VM too. That way USB is more reliable, faster and anything plugged into these cards will automatically end up as a device inside the VM without needing to passthrough individual USB devices.
But simple USB passthrough might be fine if you just want to use a mouse/keyboard that doesn't need great latency or throughput.
 
  • Like
Reactions: iadityaharsh
Best would be to get three USB controller PCIe cards and passthrough one of them to each VM too. That way USB is more reliable, faster and anything plugged into these cards will automatically end up as a device inside the VM without needing to passthrough individual USB devices.
But simple USB passthrough might be fine if you just want to use a mouse/keyboard that doesn't need great latency or throughput.
Ideally I'd do that but even though my Motherboard has 5 PCIe Gen4 x16 Slots and 2 PCIe Gen4 x8 slots, I'll be adding 3 Nvidia A4000 which will each consume 3 x16 slots, 1 Asus Hyper M.2 Gen4 Card for 4 M.2 NVMe drives (for OS and Applications), leaving me with 3 slots for which 1 x8 will be used by a Intel Dual 10 gig NIC and I donot want to waste precious x16 slot on some USB Expansion Card.

For GPU Passthrough of A4000 do I need to tweak some things inside Proxmox?
 
1 Asus Hyper M.2 Gen4 Card for 4 M.2 NVMe drives (for OS and Applications)
Make sure to buy enterprise grade SSDs in case you want to use ZFS. There aren't much M.2 options if you don't use M.2 to U.2 adapters.
If you want PCIe 4.0 there are basically only two options: Samsung PM9A3 and Micron 7400 PRO/MAX.
For GPU Passthrough of A4000 do I need to tweak some things inside Proxmox?
You should follow this guide: https://pve.proxmox.com/wiki/Pci_passthrough

And if you are lucky the onboard USB controllers got their own IOMMU group. In that case you could also passthrough some of them.
 
Last edited:
You should follow this guide: https://pve.proxmox.com/wiki/Pci_passthrough

And if you are lucky the onboard USB controllers got their own IOMMU group. In that case you could also passthrough some of them.
I've been using TrueNAS Core for a long time and I've seen the guides for TrueNAS Scale, there GPU passthrough is as simple as isolating the GPU from settings and bam I've a new VM with allocated GPU.

The only issue is, that there is no other way to passthrough USB except for using a PCIEe Expansion Card for each VM and that is the reason why I leaned towards Proxmox, but I think it'll some extra work to do for proper configuration.
 
Big difference between PVE and TrueNAS is that PVE is a full Linux distribution and TrueNAS is just an appliance. So you got much more freedom with PVE because you are not just limited to stuff that the GUI allows you to do. But this also comes with more complexity so its harder to setup.
 
I've been using TrueNAS Core for a long time and I've seen the guides for TrueNAS Scale, there GPU passthrough is as simple as isolating the GPU from settings and bam I've a new VM with allocated GPU.
With proper hardware, it's just selecting passthrough via the GUI and "bam" it works :-D
 
Any hrdw do. Disconnect internet, install proxmox 7.1 , and all passthrough do work fine. i run 3 quadro over a nuc9 and one in a tb case. Mouse is ok via gui of prox, but anything else, get an highpoint usb card.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!