AMD-Vi: Unable to read/write to IOMMU perf counter

[leesteken]

Hello,
I have a small Homeserver with the A520M-ITX/ac and Ryzen 4350G. I'm pretty new to proxmox and linux and had trouble to passthrough an Intel i340T4 but that was mostly because i used q35 machine with freebsd. Now its working fine although i get the same perf count error:
...
I have also just 5 IOMMO Groups with the newest BIOS:
...
When I tried to passthrough the m.2 E-Key Intel wifi module the system crashes because the M.2 SSD with the host system is in the same group. I used an old router for wifi for now but do you think with an old BIOS I could get better IOMMO Groups as well?
I try to passthrough the iGPU at the moment but even if the system does not crash the vm wont start at all.

As far as I know, nobody has been successful in passthrough of iGPU of Ryzen (even when it looks like it should work).
Yes, some BIOS version have better groups than others, sometimes that better version is older. The A520 chipset has only limit hardware support for passthrough.

 

[pottproll]

Yes, thats the result of my research so far. I know the A520 has limited capabilities but I thought the groups are determined by the soldering on the pcb or something like that. So i'll try a few different BIOS versions, maybe i can get the wifi passtrough working.

 

[leesteken]

The actual connections are of course determined by hardware, but the BIOS knows which devices and PCIe slots can be properly isolated using PCIe ACS. If devices are not isolated, they can talk to each other via DMA, which you don't want between VMs and from VM to host without permission. Sometimes they are properly isolated, but the manufacturer does not put this information in the BIOS, which means less IOMMU groups with more devices per group. If you don't care about the ACS (or know better than the BIOS) you can use the kernel parameter pcie_acs_override.

 

[Ramalama]

The IOMMU groups pretty good (don't worry about PCI bridges) except for group 5. I think this BIOS version is good enough to use. Most importantly: device 26:00.0 is now alone in a separate group!
EDIT: Is this BIOS version 1.00 (which I asked before)? What is pve-edge (because I have not heard it before)? Just to be sure: this is without ACS-override?

pve-edge:
https://github.com/fabianishere/pve-edge-kernel

@Lip90 you groups are perfect now, that was just a bios downgrade? Wth xD

@pottproll you have nothing to loose with a bios downgrade. Only the bios settings gets probably resetted...
But 4/5 groups are horrible for passthrough.

Edit: Ups, i stucked at page 1 of this thread, ignore this post xD
Just seen the second site here after posting this xD

 

[Lip90]

@Ramalama yes, ist was only a BIOS downgrade to 1.00 . But i think the mainboard is trash. I can‘t use the onboard 210 nics. If i do a bond my Speed Drops down to 11 MB/s. If i try to passthrough the 210 nics to pfsense the Proxmox crash. If i do the same with my i350-t2 pci Card everthing Looks Good...

 

[Ramalama]

@Lip90
I readed somewhere exactly about that issue with the 11mb/s.
As far i remember it has something todo with the ipmi.
But fml, i forgot where i readed this, and there is a workaround to either set the ipmi lan interface to dedicated or to shared, i forgot if it was the one or the other...
If i find the link again with this bug, I'll post it.

 

[Lip90]

@Ramalama thx for your help.

 

[Ramalama]

@Lip90
I think you can update your bios again xD

https://lwn.net/Articles/849642/
- iommu/amd: Fix performance counter initialization

I will provide after sleep the kernel xD

 

[Lip90]

@Ramalama
i no xD. do you think the new BIOS works with the 5.11 kernel?

one question ... what do I actually have to install? only the pve-edge-kernel-5.11.6-1_5.11.6-1+zen21_amd64.deb for my 4650G?
Do I have to install the other things linux-tools-5.11_5.11.6-1_amd64.deb, pve-edge-headers-5.11.6-1_5.11.6-1_amd64.deb and this pve-kernel-libc-dev_5.11.6-1_amd64.deb as well?

 

[Ramalama]

Nope, only the kernel.
But you need 5.11.7, not 5.11.6...
The change is in .7

Here i described the install, if you have troubles:
https://forum.proxmox.com/threads/windows-vm-really-bad-memory-performance.83509/post-376831

And in that same thread at the end, the uninstall process either.

 

[Lip90]

@Ramalama
do you compile the kernel yourself? on the page is the 5.11.7 https://github.com/fabianishere/pve-edge-kernel/releases unfortunately I can't find it.

have you tested it yet?

 

[Ramalama]

https://github.com/fabianishere/pve-edge-kernel/issues/59

I will provide 5.11.7 when i have compiled and tested it.
Hopefully today, maximal tomorrow.

In my first try on normal gcc-11 (not aocc) with lto enabled, i runned out of ram xD

Cheers

 

[Ramalama]

@Lip90
https://github.com/fabianishere/pve-edge-kernel/issues/59
https://cloud.golima.de/s/E7KiWLFgzSp4MNP

Cheers