Pass GPU back to Host on VM Shutdown

[paradigmapc]

Does anyone know a way to do either the following….
1. Reattach the gpu to the host when a VM shuts down

2. Automatically start a VM when a particular VM shuts down?

I’m attempting to give a gpu back to the host for console display when a be shuts down or start a vm when one shuts down. Option 1 would be my first choice.

I am assuming this has to be possible because I can start one by with gpu pass through, shut it down and then star my another vm with the same gpu pass through via the api. So somewhere the pci card is being in moved from one vm to another, so I am assuming it is possible to reattach it to the host instead of the second vm, but my logic is often wrong.

Thank you

 

[Dunuin]

For point 2 you could make use of hook scripts.
But I think attaching a GPU back to the host is problematic in general. Many GPUs got a problem with being switched between host and guests thats why the wiki is pointing to alot of ways how to disable that this GPU can be used with the host at all. Blacklisting drivers, setting iGPU as primary GPU in BIOS instead of PCIe GPUs, blacklisting GPUs with specific IDs...all workarounds to make sure that the PVE host will never try to use that GPU so it can be passed through to a VM.

 

[leesteken]

If you are lucky enough to have a GPU that resets properly and can do so multiple times, you can (also using hook-scripts) unbind the vfio-pci driver and bind the GPU driver on the host. This is the reverse of something like this and you need to do it in Perl. For me, it worked with a RX570 when using vendor-reset.
If you want to pass the same GPU from one VM to another, you don't need to pass it back to the Proxmox host (like I described above) and it just stays bound to the vfio-pci driver. But you still need a GPU that can reset properly multiple times. You can test this by fully stopping and starting (which not the same as a reboot from inside the VM) the first VM a dozen times via the Proxmox web GUI.

 

[paradigmapc]

If you are lucky enough to have a GPU that resets properly and can do so multiple times, you can (also using hook-scripts) unbind the vfio-pci driver and bind the GPU driver on the host. This is the reverse of something like this and you need to do it in Perl. For me, it worked with a RX570 when using vendor-reset.
If you want to pass the same GPU from one VM to another, you don't need to pass it back to the Proxmox host (like I described above) and it just stays bound to the vfio-pci driver. But you still need a GPU that can reset properly multiple times. You can test this by fully stopping and starting (which not the same as a reboot from inside the VM) the first VM a dozen times via the Proxmox web GUI.

For point 2 you could make use of hook scripts.
But I think attaching a GPU back to the host is problematic in general. Many GPUs got a problem with being switched between host and guests thats why the wiki is pointing to alot of ways how to disable that this GPU can be used with the host at all. Blacklisting drivers, setting iGPU as primary GPU in BIOS instead of PCIe GPUs, blacklisting GPUs with specific IDs...all workarounds to make sure that the PVE host will never try to use that GPU so it can be passed through to a VM.

Thank you for the replies. I had attempted to use a hookscript that executed attempted to start the second vm in the “post-stop” phase but this seemed to only get called if I stopped the vm from the web-gui and not shutdown the vm from within which I think the doc says is how it works too. I can’t find reference to any other phases than what is listed in the example script, but the example script does quote “shutdown” but I don’t see anything listed in logs referencing any other phases that occur that would make it past the elsif statements. For good measure I did try to execute the commands if a shutdown phase was found but that didn’t seem to work either.

My guess is I’m thinking of this completely wrong, wouldn’t be the first time.

 

[leesteken]

Hook script on shutdown from inside the VM is supposed to work but indeed some people are running into this issue.

 

[paradigmapc]

Hook script on shutdown from inside the VM is supposed to work but indeed some people are running into this issue.

That is what I figured. Reading that thread it sounds like the same issue as I am passing through a usb Logitech dongle. Not certain how it would be functional to remove that. Thank you for the response.