[SOLVED] True Headless PVE with single GPU passthrough

[krikey]

I have a home lab setup using a relatively low specced Ryzen5 2600x (no integrated graphics). My Mobo has VGA and DV ports, but presumably these will only work if the CPU has a built-in GPU.

I also have a nvidia 1080ti mounted in a PCI-e slot which I'd like to use with PCI Passthrough, however, as the 1080ti is the ONLY GPU available, Proxmox allocates it to itself. This means that I cannot dedicate it to a VM.

Now that proxmox is installed, is there a way I can et proxmox to ignore the 1080 and essentially boot headless, so that my 1080 can be reallocated?

 

[krikey]

I should say that after the 1080ti is installed, theres no room left to install another PCI-e card (its quite a small mobo).

 

[krikey]

Im presuming from the absence of any responses that installing Proxmox with no GPU is not possible?

 

[leesteken]

Indeed, the display outputs on the motherboard (maybe you can share which motherboard exactly?) will only work with APUs (Ryzen CPU with graphics).

In short: install Proxmox with the NVidia GPU. Create and install a VM (without passthrough). Setup the VM with passthrough of the GPU (but don't start it!) and also setup Proxmox to not use the GPU (which will make troubleshooting harder if anything goes wrong). Reboot and try it remotely.

Make sure you have another computer that can SSH to Proxmox and preferably allows you to use the web GUI. Check your IOMMU groups, but the PCIe x16 slot closest to the CPU should be alone in a group and passthrough should work fine. Check this forum for post on how to do all this and what kind of problems you can expect from passthrough of a 1080.

 

[krikey]

Indeed, the display outputs on the motherboard (maybe you can share which motherboard exactly?) will only work with APUs (Ryzen CPU with graphics).

In short: install Proxmox with the NVidia GPU. Create and install a VM (without passthrough). Setup the VM with passthrough of the GPU (but don't start it!) and also setup Proxmox to not use the GPU (which will make troubleshooting harder if anything goes wrong). Reboot and try it remotely.

Make sure you have another computer that can SSH to Proxmox and preferably allows you to use the web GUI. Check your IOMMU groups, but the PCIe x16 slot closest to the CPU should be alone in a group and passthrough should work fine. Check this forum for post on how to do all this and what kind of problems you can expect from passthrough of a 1080.

Thanks AVW. So it is possible but not recommended for the reasons you mentioned?

The Mobo is a Gigabyte GA-A320M-DS2 and while it has 1 x PCIe x16 slot and 2 x PCIe x16 slots, the smaller slots are overlapped by the thick 1080Ti GPU.

 

[leesteken]

You would not be able to use the other two slots for passthrough anyway, with a A320 chipset.
Single GPU passthrough is indeed possible; this forum and the internet is full of examples. People have had success with GTX 1080 before. The x16 PCIe slot is most likely in its own IOMMU group (or update the BIOS if not). Passthrough would most likely be possible for your setup.
Downside is that it is more difficult to troubleshoot: make sure you can access Proxmox from another computer via a local network (because you lose the "physical" console).

 

[krikey]

Thanks AVW. I swapped out the Ryzen 5 2600x for an AMD a10-9700 which as its own built in GPU and was able to then passthrough the 1080Ti.
The downside is that the A10 is a less capable CPU in terms of speed and cores, but as this is only a testbed its not critical.

 

[krikey]

Just revisiting this again as I'm setting up another test box with 1 x nvidia 1060 GPU. This Mobo has no video out ports of any kind and the processor does not support it.

There was mention above of "and also setup Proxmox to not use the GPU".

How would I stop PVE from using the 1060 GPU? Is there a command I should add to /etc/default/grub ?

 

[leesteken]

How would I stop PVE from using the 1060 GPU? Is there a command I should add to /etc/default/grub ?

initcall_blacklist=sysfb_init and probably early binding to vfio-pci or preventing any drivers (nouveau) from touching it. Note that you won't see any boot messages and have not Proxmox host console anymore if things go wrong. Press e in the boot menu to add or remove kernel parameters for one boot only.

 

[krikey]

Thank you.