PCI Passthrough with Overclocked Ram

[darkseer]

I have the following:

Motherboard: TRX40 AORUS PRO WIFI
CPU: AMD Ryzen Threadripper 3970X
Memory: VENGEANCE RGB PRO 128GB (4 x 32GB) DDR4 DRAM 3000MHz C16 Memory Kit

The Setup:
When I use all the stock settings everything I need works. However the memory is only running at 2133 Mhz and is rated for 3200 Mhz.
In the bios when I choose the xmp profile for 3200 Mhz and up the ram voltage to the specified 1.35 v proxmox boots.
They system I am running has quite a number of VMs, which you would expect from the specifications. All of the VMs that require nothing special work fine... and are now noticeably faster for what I'm doing.

The issue:
Any VM using PCI passthrough vifo now doesn't boot after the bootloader. This is true for both windows and linux. I use the PCI passthrough to access GPUs natively. What am I missing here? I exclude the PCI addresses at boot time for the host, and as I mentioned previously, the memory speed change is the only difference, all other VMs function normally and the VMs I need to have native GPU access function normally at the lower memory clock.

My question:
Has anyone else run into this?
Is this an upstream KVM issue?
Is this a known issue?

My preliminary searches have come up empty on the kvm resources so if someone can point me in the right direction it would help. Thank you.

 

[Stefan_R]

Hm, that's a weird issue indeed. Anything in the logs when this happens? dmesg, journalctl?

In general, virtualization on Threadripper 3000 is still a bit experimental and rough around the edges, but I can confirm that it does work in general, even with overclocked RAM (on an ASUS TRX40 Prime at least).

Maybe setting the RAM to use the XMP profile changes some other thing in the BIOS? BIOS setups are often weird and do stuff on their own when you don't expect them to. Check IOMMU, SVM, PCI AER/ACS etc...

 

[darkseer]

I will go over all of that when I can take it off line. Probably in the next few days. I'm also looking a bus voltage.

 

[Joris L.]

Did your try running the memtest at boot ? This should be a reliable indicator for memory speed as the core issue. Seeing the errors only pop up at some speeds and not others woke me up.

The issue i had is (was) somewhat similar, this for a 1700x CPU on an X370 motherboard with DDR4-3200 memory. I experienced freak things happening like OOM. I remember just one setting worked flawless here, not this time. Maybe it was the BIOS upgrade.

Wikichip confirms what was suggested before by a user on this forum (...Billl something), the 1700x is only rated to work with 2666Mhz. Setting the memory speed to 2666Mhz may have mostly fixed it for me, running without a hitch for 12 hours now.

 

[darkseer]

When I can have downtime I will give that a whirl.