NVIDIA MIG (Multi-Instance GPU) on Proxmox

[willh32]

@dosmage any further update? I'm looking to run some LLM instances on VMs and using MIG in proxmox would be awesome.

 

[MasaMC]

Has anyone tried this recently?

 

[denormalize]

Someone. Please! I'm not smart enough to figure this out myself. Same for AMD SEV-SNP which I know works with this technology

 

[Dariusz]

I ended up buying Intel B50 and running SV-IOR and I now have 8 glorious virtual gpus for my vms.

 

[MasaMC]

We recently tried the MIG instances.
There's just no way to use them with VMs, but with LXC containers they seem to be working. Prefererrably with Nvidian container toolkit so it's simplier to mount the to containers config by using the MIG instances IDs.
The MIGs for VMs would require the paid Enterprise AI license for VGPU feature, but that's not yet officially supported with Proxmox and H200 card so not sure if this would work.