Note that you don't need
amd_iommu=on because it is on by default, but it does no harm.One of your RTX 3090 (Gigabyte) is alone in a IOMMU group (number 13) and can be passed to a VM.
The other RTX 3090 (Palit) and the RX 580 are in the same IOMMU group (number 12) as the motherboard chipset together with a USB, SATA and network controller. As I said before, all device in a IOMMU group (except bridges) need to go to one VM and cannot be shared. So as soon as you pass one of those GPUs to a VM, the Proxmox host loses the network and the drives connected to the SATA controller. This is the reason why your Proxmox appears to freeze and is unreachable from the network.
The only way to change the IOMMU group is to change the PCIe slot because it is determined by the motherboard hardware and BIOS. On the AMD X470 and B450 chipsets all the x1 PCIe slots and additional devices are part of the chipset groups. Only the AMD X570 does not have this problem.
You could try the
pcie_acs_override=downstream kernel parameter, which breaks the groups (as well as the device security isolation) and is not guaranteed to work, but it sometimes works fine for people.