IOMMU groups differ in Proxmox compared to other linux distro

[nontii]

Hi!
I ran Unraid earlier on my server (MSI Z490Z Pro with a Intel i3 10100) and my devices was perfectly isolated with one IOMMU group each.
Then I installed Proxmox instead and to my surprise the iommu groups were totally different. Not isolated at all, many devices in same
IOMMU-group.

Rebooted with unraid from an USB-stick and the IOMMU-groups were perfect again..

How come this behaviour? I thought the IOMMU groups were decided by the CPU and motherboard?

Note: NO ACS Override enabled on Unraid or Proxmox!

Unraid has a newer kernel than Proxmox, can this be a factor? But again, I thought the groups were decided in the cpu and/or the motherboard?

 

[Stefan_R]

How did you set up your kernel commandline? I.e. what does 'cat /proc/cmdline' give you?

In general you're right, IOMMU isolation should be provided by the CPU/Chipset. Are you sure Unraid isn't using some trickery to make devices appear seperated even if they're not?

 

[nontii]

How did you set up your kernel commandline? I.e. what does 'cat /proc/cmdline' give you?

In general you're right, IOMMU isolation should be provided by the CPU/Chipset. Are you sure Unraid isn't using some trickery to make devices appear seperated even if they're not?

Thanks for your answer!
No I dont't think Unraid does trickery things, I turned off all Acs override before running it on the intel machine. I ran the unraid server on a Ryzen before and the groups were a mess and I had to enable the acs override patch on that hardware, but checked that all that was disabled when running on the intel machine. I think Unraid is running kernel 5.9 (a beta version of unraid).

I will post the cmd line as soon as I can start the machine (it is in peaces at the moment).

Also, I have pretty new hardware with different new nics (intel i225-v (2.5gbps) and realtek 8125 (2.5gbps) and i don't think none of them works atm.
When will proxmox run the latest lts kernel, linux 5.10? Will solve a lot of problems for me