IOMMU whis 3 graphics cards

[legasmez]

hello,
I use proxmox V7.2-3, I plan to create two Gamer VMs on it, one for me and one for my son. I have two nvidia 1060 graphics cards and a basic card that I want to dedicate to the host.

When I put my basic card, whis a 1060 and I assign it to a VM, everything is ok, it works,

I followed this procedure:

https://github.com/lucduke/proxmox/blob/main/3-vm-gaming.md

But when I put the third, at boot, the network card deactivates it. that is to say that at the beginning of the boot it is active, and almost at the end of the boot it deactivates it.

I tested the PCI ports, they all work, riser or no riser, so it's not the risers that are at fault. I tried modifying the GRUB, replacing this line

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt kvm.ignore_msrs=1 video=vesafbff video=efifbff intremap=no_x2apic_optout"

by this line

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pcie_acs_override=downstream,multifunction"

so that each device is its own group, because I suspect a conflict between the network card and the one of the graphic cards.

But that doesn't work either.

I don't know what to do anymore, so if you have the solution that would be great.

Thanks in advance.

PS: sorry for my english i am french.

 

[leesteken]

The network configuration (in /etc/network/interfaces) depends on the name of the network card. In modern Linux systems like Proxmox, the name of the network card depends on the PCI ID. The PCI ID of PCI(e) devices (also the onboard devices, like the network) can shift by one when you add or remove other PCI(e) devices.
Login to the Proxmox host console using a physical display and keyboard. Check the new name of the network device with ip a and change all occurrences of the old name to the new name using nano /etc/network/interfaces.

 

[Dunuin]

Did you check with for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort and lspci that both GPUs really got their own IOMMU groups?

 

[legasmez]

The network configuration (in /etc/network/interfaces) depends on the name of the network card. In modern Linux systems like Proxmox, the name of the network card depends on the PCI ID. The PCI ID of PCI(e) devices (also the onboard devices, like the network) can shift by one when you add or remove other PCI(e) devices.
Login to the Proxmox host console using a physical display and keyboard. Check the new name of the network device with ip a and change all occurrences of the old name to the new name using nano /etc/network/interfaces.

I run to test it,

I'll get back to you

THanks

 

[legasmez]

Did you check with for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort and lspci that both GPUs really got their own IOMU group?

No I don't check it,

I will try both solutions, thank you

 

[legasmez]

The network configuration (in /etc/network/interfaces) depends on the name of the network card. In modern Linux systems like Proxmox, the name of the network card depends on the PCI ID. The PCI ID of PCI(e) devices (also the onboard devices, like the network) can shift by one when you add or remove other PCI(e) devices.
Login to the Proxmox host console using a physical display and keyboard. Check the new name of the network device with ip a and change all occurrences of the old name to the new name using nano /etc/network/interfaces.

It's Working, good, thanks a lot.