[SOLVED] True Headless PVE with single GPU passthrough

krikey

Active Member
Aug 15, 2018
86
14
28
UK
I have a home lab setup using a relatively low specced Ryzen5 2600x (no integrated graphics). My Mobo has VGA and DV ports, but presumably these will only work if the CPU has a built-in GPU.

I also have a nvidia 1080ti mounted in a PCI-e slot which I'd like to use with PCI Passthrough, however, as the 1080ti is the ONLY GPU available, Proxmox allocates it to itself. This means that I cannot dedicate it to a VM.

Now that proxmox is installed, is there a way I can et proxmox to ignore the 1080 and essentially boot headless, so that my 1080 can be reallocated?
 

krikey

Active Member
Aug 15, 2018
86
14
28
UK
I should say that after the 1080ti is installed, theres no room left to install another PCI-e card (its quite a small mobo).
 

krikey

Active Member
Aug 15, 2018
86
14
28
UK
Im presuming from the absence of any responses that installing Proxmox with no GPU is not possible?
 

leesteken

Famous Member
May 31, 2020
2,362
503
118
Indeed, the display outputs on the motherboard (maybe you can share which motherboard exactly?) will only work with APUs (Ryzen CPU with graphics).

In short: install Proxmox with the NVidia GPU. Create and install a VM (without passthrough). Setup the VM with passthrough of the GPU (but don't start it!) and also setup Proxmox to not use the GPU (which will make troubleshooting harder if anything goes wrong). Reboot and try it remotely.

Make sure you have another computer that can SSH to Proxmox and preferably allows you to use the web GUI. Check your IOMMU groups, but the PCIe x16 slot closest to the CPU should be alone in a group and passthrough should work fine. Check this forum for post on how to do all this and what kind of problems you can expect from passthrough of a 1080.
 

krikey

Active Member
Aug 15, 2018
86
14
28
UK
Indeed, the display outputs on the motherboard (maybe you can share which motherboard exactly?) will only work with APUs (Ryzen CPU with graphics).

In short: install Proxmox with the NVidia GPU. Create and install a VM (without passthrough). Setup the VM with passthrough of the GPU (but don't start it!) and also setup Proxmox to not use the GPU (which will make troubleshooting harder if anything goes wrong). Reboot and try it remotely.

Make sure you have another computer that can SSH to Proxmox and preferably allows you to use the web GUI. Check your IOMMU groups, but the PCIe x16 slot closest to the CPU should be alone in a group and passthrough should work fine. Check this forum for post on how to do all this and what kind of problems you can expect from passthrough of a 1080.
Thanks AVW. So it is possible but not recommended for the reasons you mentioned?

The Mobo is a Gigabyte GA-A320M-DS2 and while it has 1 x PCIe x16 slot and 2 x PCIe x16 slots, the smaller slots are overlapped by the thick 1080Ti GPU.
 

leesteken

Famous Member
May 31, 2020
2,362
503
118
You would not be able to use the other two slots for passthrough anyway, with a A320 chipset.
Single GPU passthrough is indeed possible; this forum and the internet is full of examples. People have had success with GTX 1080 before. The x16 PCIe slot is most likely in its own IOMMU group (or update the BIOS if not). Passthrough would most likely be possible for your setup.
Downside is that it is more difficult to troubleshoot: make sure you can access Proxmox from another computer via a local network (because you lose the "physical" console).
 
  • Like
Reactions: krikey

krikey

Active Member
Aug 15, 2018
86
14
28
UK
Thanks AVW. I swapped out the Ryzen 5 2600x for an AMD a10-9700 which as its own built in GPU and was able to then passthrough the 1080Ti.
The downside is that the A10 is a less capable CPU in terms of speed and cores, but as this is only a testbed its not critical.
 

krikey

Active Member
Aug 15, 2018
86
14
28
UK
Just revisiting this again as I'm setting up another test box with 1 x nvidia 1060 GPU. This Mobo has no video out ports of any kind and the processor does not support it.

There was mention above of "and also setup Proxmox to not use the GPU".

How would I stop PVE from using the 1060 GPU? Is there a command I should add to /etc/default/grub ?
 

leesteken

Famous Member
May 31, 2020
2,362
503
118
How would I stop PVE from using the 1060 GPU? Is there a command I should add to /etc/default/grub ?
initcall_blacklist=sysfb_init and probably early binding to vfio-pci or preventing any drivers (nouveau) from touching it. Note that you won't see any boot messages and have not Proxmox host console anymore if things go wrong. Press e in the boot menu to add or remove kernel parameters for one boot only.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!