Ubuntu VM with multiple GPU passthroughs wont boot

Schmerbert

New Member
Sep 12, 2023
4
0
1
I have a server with mutiple GPUs (NVIDIA A40) and want to create a VM using mutiple GPUs. I created a VM with Ubuntu-22.04, set up passtrough on the host and on the VM. A single GPU is detected and usable in the VM. If i add another GPU, the VM wont boot anymore. Any idea where I should start looking for the Error?
 
If the Task does not show any information, use journaltctl (on the Proxmox host console or SSH) and scroll with the arrow keys to the time of starting the VM.
I already did that and searched for any GPU related errors but could not find any.
 
I already did that and searched for any GPU related errors but could not find any.
I didn't realize, sorry. If there are no errors anywhere but just a timeout during VM start, try setting the VM memory (much) lower. This is not related to the VM (which needs all memory to be pinned in to host RAM because of passthrough) but maybe your Proxmox has less memory free now that before (when you tested with one GPU).
If i add another GPU, the VM wont boot anymore.
Can you be more specific about "wont boot"? Does it still boot with one GPU? Does it give a timeout in the Proxmox Task? Does it hang in a certain screen? Does it start from the console with qm start? What does journalctl show when starting the VM?
 
I didn't realize, sorry. If there are no errors anywhere but just a timeout during VM start, try setting the VM memory (much) lower. This is not related to the VM (which needs all memory to be pinned in to host RAM because of passthrough) but maybe your Proxmox has less memory free now that before (when you tested with one GPU).

Can you be more specific about "wont boot"? Does it still boot with one GPU? Does it give a timeout in the Proxmox Task? Does it hang in a certain screen? Does it start from the console with qm start? What does journalctl show when starting the VM?
Proxmox has enough memory. With just 1 GPU, the VM boots and works perfectly fine. With more than 1 GPU, the VM can be started via the GUi and qm start but wont respond (but does not timeout), Guest Agent and SSH server are also not starting. There is no way to access ist. The VM needs to be forcefully stopped as it does not respond to shutdown/reboot. There is actually no record in the journal in those cases, so journalctl only shows the successfull boot attemps using 1 GPU.
 
Last edited:
Hey I have a similar problem have you managed to get multiple GPUs get assigned to a single VM? I suspect that it might be that we both are trying to have 2 similarly IDed GPUs and then maybe there is an addressing collision?
 
Hey I have a similar problem have you managed to get multiple GPUs get assigned to a single VM? I suspect that it might be that we both are trying to have 2 similarly IDed GPUs and then maybe there is an addressing collision?
Not really, I deleted the VM and set it up againg and now it works
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!