Issues with AMD processor and IOMMU/PCI passthrough

[ramgine]

Evening,
I've been troubleshooting this issue for a while now. I may just have to replace the system.
Board: Gigabyte B350 DS3h.
Processor: 2600x
discrete GPU: Nvidia P2000
I've enabled SVM and made sure IOMMU was set to enabled instead of Auto. I have disabled C-States. I have set amd_iommu=on in my kernel options. I have updated the BIOS to the latest version that came out a couple months ago. PVE is updated to latest kernel. I check dmesg | grep IOMMU and I don't get "enabled" or "disabled" but it is listed.

I built a new PVE server due to having a shortage of drive support on my previous server. They were all Intel. This is my first AMD PVE server and my only that doesn't work.

I've had a ton of issues with this machine all relating to PCI passthrough. I've finally set that VM to disable auto start, and removed all hardware passthrough to it. This is the only way i can get the PVE host to boot. With it disabled I can get to the PVE site, i can ssh into the server everything works. If I pass *ANY* PCI device to the VM and turn on the VM, i lose my both NICs (onboard and PCI) and all USB ports. Video works.

I also get an error on the command prompt and this *only* shows up when I turn that VM on, so i don't have a dying disk (i tested it separately):
EXT4-fs (dm-1): Remounting filesystem read-only
89.026631] EXT4-fs error (device dm-1): ext4_Journal_check_start:B4: comm systemd-Journal: Detected aborted Journal

I'm guessing this board has issues with IOMMU. I'm hoping there's something i'm missing with kernel options to fix this. Any help?

 

[leesteken]

amd_iommu=on is nonsense since it is on by default and the manual does not mention it: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_pci_passthrough .
Check you IOMMU group isolation: https://pve.proxmox.com/wiki/PCI_Passthrough#Verify_IOMMU_isolation . You cannot share any devices from the same group between VM and/or the Proxmox host.
The B350 only supports passthrough of the one PCIe slot (connected directly to the CPU) and maybe one M,.2 slot. Lots of threads about this on this forum, from the time that AM4 was the latest platform.

 

[ramgine]

amd_iommu=on is nonsense since it is on by default and the manual does not mention it: https://pve.proxmox.com/pve-docs/pve-admin-guide.html#qm_pci_passthrough .
Check you IOMMU group isolation: https://pve.proxmox.com/wiki/PCI_Passthrough#Verify_IOMMU_isolation . You cannot share any devices from the same group between VM and/or the Proxmox host.
The B350 only supports passthrough of the one PCIe slot (connected directly to the CPU) and maybe one M,.2 slot. Lots of threads about this on this forum, from the time that AM4 was the latest platform.

I read a few times amd_iommu=on did nothing but figured it didn't hurt to try after all the other troubleshooting. The single passthrough explains a lot. I kept trying to pass two devices. I didn't come across a thread on the b350's limitations after lots of googling. I guess I do have to replace this machine. Thanks a ton for you help!

 

[leesteken]

I didn't come across a thread on the b350's limitations after lots of googling. I guess I do have to replace this machine.

Only X570(S) has almost everything in separate groups. X470/X370 have only two PCIe slots and one M.2 for passthrough, and you lose one PCIe slot when using a AM4 CPU with integrated graphics. B550/B450/B350 have only one PCIe slot for passthrough.
It's different for EPYC, Threadripper and Intel (Xeon). You could consider using pcie_acs_overrride but that might not be secure.

 

[ramgine]

Only X570(S) has almost everything in separate groups. X470/X370 have only two PCIe slots and one M.2 for passthrough, and you lose one PCIe slot when using a AM4 CPU with integrated graphics. B550/B450/B350 have only one PCIe slot for passthrough.
It's different for EPYC, Threadripper and Intel (Xeon). You could consider using pcie_acs_overrride but that might not be secure.

This is just for a plex server, im not super worried about security but I do need two devices. One for transcoding and one for the storage controller.
A board upgrade isn't really worth it on an old cpu platform so I ordered an i3-14100. Seems to have much better virtualization support and I can remove the GPU for transcoding. The B760 chipset I'll test out.