Dual A100 Passthrough to Ubuntu 22.04 VM - Proxmox8

Sam Clark

New Member
Mar 12, 2024
1
0
1
I have a system with 4 A100 40GB cards. I setup the proxmox Ubuntu 22.04 VM with 2 GPU's and I am having trouble detecting the second one. I have checked the IOMMU groups (at least I believe I did.) And they appear to be in different groups. I successfully passed one gpu in and it shows in nvidia-smi. Both GPU's show in lspci on the ubuntu VM but the second GPU is not detected in nvidia-smi. I have attached everything that may be useful below. Nothing crashes and I don't have errors that I can find, it just doesn't show up as a GPU when checking through nvidia-smi.

Any help would be appreciated. I am fairly new to proxmox and had good luck getting the first passthrough and am just struggling on the second. If you need more information let me know.
 

Attachments

  • GPU's.png
    GPU's.png
    36.9 KB · Views: 10
  • IOMMU.png
    IOMMU.png
    18.3 KB · Views: 10
  • nvidia-smi.png
    nvidia-smi.png
    23 KB · Views: 10
  • VMConfig.png
    VMConfig.png
    28.2 KB · Views: 10
  • VmHardware.png
    VmHardware.png
    34.6 KB · Views: 10
Hi,

Did you turn on SR-IOV in BIOS?

I assigned 2 gpus to resources mapping in datacenter.

1765262081746.png

And I added 2 gpus to VM.
1765262424227.png

Finally, I could see 2 gpus by nvidia-smi in ubuntu 24.04 server VM.
1765262394475.png