PVE8 - Change to IOMMU/Passthrough?

dlasher

Renowned Member
Mar 23, 2011
234
26
93
Recently upgraded from PMX7 (Proxmox Version 7 or PVE7) to PMX8 (Proxmox Version 8 or PVE8), and I noticed the passthrough doesn't seem to work the same way in the UI it used to.

<= PVE7, the way you knew IOMMU/etc wasn't working is that in the UI, the "MAPPED DEVICES" dropdown would be empty. Once you got it working right, your devices would be listed there, and you could choose them.

PVE6: (images from a test/nested VM - you can ignore the no MMU error)

Screen Shot 2023-12-08 at 12.41.02 PM.png


PVE7: (images from a test/nested VM - you can ignore the no MMU error)
Screen Shot 2023-12-08 at 1.07.10 PM.png

Screen Shot 2023-12-08 at 1.06.52 PM.png


Now in PVE8, nothing ever appears in the mapped devices list, the fields I'm used to configuring "Primary GPU, MDev Type, Device" are all grouped under "MAPPED DEVICE", and you're forced to use "RAW DEVICE" to passthrough.

Screen Shot 2023-12-06 at 4.57.02 PM.png


Thinks I've tried:
* I've experimented with 5.x kernel vs 6.x kernel - no change.
* tried every available grub switch I can find (using IOMMU=PT) - no change
* moved slots/etc of devices -- no change


To be clear, IOMMU is alive and working, but the MAPPED DEVICE functionality is no longer usable. Why?


Code:
EXAMPLE:
root@pmx5:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[    0.595598] AGP: Please enable the IOMMU option in the BIOS setup
[    1.317015] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[    2.849697] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.849791] AMD-Vi: Interrupt remapping enabled

IOMMU Group 22 04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107GL [Quadro P1000] [10de:1cb1] (rev a1)
IOMMU Group 22 04:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)

root@pmx5:~# dmesg | grep "10de:1cb1"
[    1.986129] pci 0000:04:00.0: [10de:1cb1] type 00 class 0x030000
[   69.129313] vfio_pci: add [10de:1cb1[ffffffff:ffffffff]] class 0x000000/00000000
 

Attachments

  • Screen Shot 2023-12-06 at 4.51.22 PM.png
    Screen Shot 2023-12-06 at 4.51.22 PM.png
    31.1 KB · Views: 18
Last edited:
Recently upgraded from PMX7 to PMX8, and I noticed the passthrough doesn't seem to work the same way in the UI it used to (which translates into difficulty of use as well)

<= PMX7, the way you knew IOMMU/etc wasn't working is that in the UI, the "MAPPED DEVICES" dropdown would be empty. Once you got it working right, your devices would be listed there, and you could choose them.

View attachment 59419

Now in PMX8, nothing ever appears in the mapped devices list, and you're forced to use "RAW Device" to pass anything through.

View attachment 59420


Thinks I've tried:
* I've experimented with 5.x kernel vs 6.x kernel - no change.
* tried every available grub switch I can find (using IOMMU=PT) - no change
* moved slots/etc of devices -- no change


To be clear, IOMMU is alive and working, but the MAPPED DEVICE functionality is no longer usable. Why?


Code:
EXAMPLE:
root@pmx5:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[    0.595598] AGP: Please enable the IOMMU option in the BIOS setup
[    1.317015] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[    2.849697] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    2.849791] AMD-Vi: Interrupt remapping enabled

IOMMU Group 22 04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107GL [Quadro P1000] [10de:1cb1] (rev a1)
IOMMU Group 22 04:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)

root@pmx5:~# dmesg | grep "10de:1cb1"
[    1.986129] pci 0000:04:00.0: [10de:1cb1] type 00 class 0x030000
[   69.129313] vfio_pci: add [10de:1cb1[ffffffff:ffffffff]] class 0x000000/00000000

"Resource mappings for PCI and USB devices": [1] and therefore "Mapped Device"s were only introduced in PVE 8: [2].
"Raw Device"s actually is what one knows from older PVE versions.

This, at least, concerns PVE (= Proxmox VE = Proxmox Virtual Environment). Maybe PMX, you are talking about, is something other that I am not aware of; but then I would suggest to contact their support channels...

[1] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#resource_mapping
[2] https://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_8.0
 
<shaking head>
While I appreciate the need for technical accuracy, it doesn't advance the problem solving efforts, and borders on snarky. However, because it obviously makes people feel better, I will edit the post to use only the correct names for the products.

<cheering>
Played with the "Resource Mapping" under the "DATACENTER" tab, and that's pretty cool! Thank you for the link on Resource Mapping, nifty to see new features added, useful data. Appreciate the info.

Screen Shot 2023-12-08 at 1.39.37 PM.png
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!