Running AI models on the host

[niko83]

I am running a Proxmox cluster at home with some old hardware, mostly to educate myself. My attempts at passing thru GPUs to VMs did not yield satisfactory results so the idea of running some AI models directly on the Proxmox host popped into my head.

Is this a bad idea? I have already botched a few installations so I am not a fan of mingling with the host configuration but this seems to be the only option I got left with.

 

[Dunuin]

Is this a bad idea?

For security, stability, maintainability, disaster recovery it is best practice to not run services directly on the PVE host that you could run inside a VM or LXC.
But PVE is Debian based with custom Ubuntu LTS kernel. So in theory you could do everything that you want that is possible with a normal Debian.