AMD pstate Behavior on Epyc Rome

[dizzydre21]

Hello all,

I am curious about the behavior I am seeing while using amd_pstate=active in my grub configuration. I have EPP set to "balance_performance" and the scaling governor set to "powersave". This seems to so far to be the best method for power reduction without totally killing performance. My system will idle with most of the cores sitting at 400Mhz, which is what I want and it saves about 50 watts at idle and a lot more than that under load. The clocks I am seeing don't really make sense to me though. I am using an Epyc 7F72, which has a base clock of 3.2Ghz and should be able to boost single cores up to 3.7Ghz. I have a few VMs running that would benefit from hitting both of those clocks when required. With the above config, it doesn't look like any single core is boosting above 2.7Ghz. I tested this by running a few games inside of an EndeavourOS VM and Cinebench in a Windows VM. CPU frequencies were monitored with watch -n1 "grep Hz /proc/cpuinfo" in the Proxmox shell. Multi-core performance doesn't suffer too much from this (5-7%), but single core performance does. I didn't run Cinebench in single core mode, but I had games lose about 5-10FPS. When struggling to hit 4K60Hz in a game on a server CPU, that FPS makes a difference.

Now, if I change EPP to "performance", cores will regularly hit 3.5Ghz, but the power usage at idle and at load greatly increases. I seem similar results when changing the scaling governor to "performance" and leaving EPP at "balance_performance". This seems like expected behavior.

Is it normal for my original configuration to not even hit base clocks while using "balance_performance" and "powersave"? Is there a combination that will help me get a bit better performance under load, while keeping the idle wattage down?

One last thing to mention is that with either EPP or the scaling governor set to performance, my CPU voltage sits at 1.35v, which absolulely spams the IPMI event log with high voltage events. I've experienced this on two different ROME8D-2T motherboards.


My Hardware:

Proxmox 8.1.3
Motherboard - Asrock Rack Rome8d-2t
CPU - Epyc 7F72
RAM - 256GB 3200MHZ ECC
GPU - RTX-4070ti Super
OS Drive - Samsung 980 Pro 500GB
VM OS Drives - ZFS Mirror 2x960GB Samsung P9A3
VM Storage - ZFS Striped Mirror 4x1TB WD SN850
TrueNAS Drives - 6x6TB WD Ironwolf
PCIe NIC - 82599ES 10Gbe - passed through to TrueNAS

 

[apprisco]

Hey! I'm trying to do something similar on TrueNAS Scale. I'm running a 7532, and see that with either passive or guided it locks all cores to 2400mhz on a 2d16-2t.
This most likely says that I have a fucked up CPPC setting, but i enabled it in BIOS in NBIO options, SMU or something.
Any ideas?
What bios settings did you use?
Thanks!

 

[Ramalama]

https://helper-scripts.com/scripts?id=Proxmox+VE+Processor+Microcode
Try that at least.

Maybe a newer microcode helps. Im not sure tho.
On my genoa Servers i run performance+performance and pstate epp behaves exactly like expected. Funny thing is that even if i don't care about power consumption, the Genoa Servers run most of the time with a total Consumption of 140W.
Without EPP they take at least 220W.
But like i said, i don't target power consumption, just bare performance, so i setted ctdp to 400W etc....
You can set Targets in the bios to performance or Power, maybe you should try to set the target to Power and set the max ctdp/PPT/PPL/TDP (There are usually 2 settings in the bios)
I would set them all to 200W or 220W, even if your CPU is rated for 200W only 220W should be safe either.
And set the target to Power.

Pstate EPP should still downclock to 400mhz, especially with balance_performance+performance!
As far i understood only balance_performance+performance and performance+performance combos will turbo, every other combination won't turbo, at least in what i know so far.

Cheers

 

[apprisco]

https://helper-scripts.com/scripts?id=Proxmox+VE+Processor+Microcode
Try that at least.

Maybe a newer microcode helps. Im not sure tho.
On my genoa Servers i run performance+performance and pstate epp behaves exactly like expected. Funny thing is that even if i don't care about power consumption, the Genoa Servers run most of the time with a total Consumption of 140W.
Without EPP they take at least 220W.
But like i said, i don't target power consumption, just bare performance, so i setted ctdp to 400W etc....
You can set Targets in the bios to performance or Power, maybe you should try to set the target to Power and set the max ctdp/PPT/PPL/TDP (There are usually 2 settings in the bios)
I would set them all to 200W or 220W, even if your CPU is rated for 200W only 220W should be safe either.
And set the target to Power.

Pstate EPP should still downclock to 400mhz, especially with balance_performance+performance!
As far i understood only balance_performance+performance and performance+performance combos will turbo, every other combination won't turbo, at least in what i know so far.

Cheers

Thanks for the pointers. Unfortunately i'm on TrueNAS Scale so no microcode updates for me.
It seems that I just don't get any CPPC values.

it alternates between 1.8 and 2.4ghz locked, genuinely no clue what's going on. But yeah, that unable to call hardware line is a big no no.

Genuinely, zero clue what setting in BIOS is incorrect. Gave up and went back to aspi_cpufreq or whatever it's called and tried pstate downclocking for P2, and it decided to fail boot over and over again so I'm giving up for now.

 

[Ramalama]

did you tryed to check with "dmesg | grep -i cpc" or cppc or something?
I don't have any cpc or cppc lines in dmesg, but i guess its because cppc is working here, but you may have if its not.

I dont have anything in dmesg, no pstate, nothing.
Check at least with dmesg --level err or warn if there is something.

 

[apprisco]

did you tryed to check with "dmesg | grep -i cpc" or cppc or something?
I don't have any cpc or cppc lines in dmesg, but i guess its because cppc is working here, but you may have if its not.

I dont have anything in dmesg, no pstate, nothing.
Check at least with dmesg --level err or warn if there is something.

I also don't have any CPPC in dmesg. absolutely nothing.
I've now gotten boost clocks working but no clue what I did. Going back to acpi now though, i'm done.

 

[Ramalama]

what i've read on google is, that if you get:
amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled

then the mainboard bios, (if cppc is enable) doesn't support cppc, or have corrupted acpi tables for cppc or something like that.

Otherwise i didn't found anything relevant.

PS: If there is absolutely nothing, then cppc should work, dunno

 

[Tortillas]

https://helper-scripts.com/scripts?id=Proxmox+VE+Processor+Microcode
Try that at least.

Maybe a newer microcode helps. Im not sure tho.
On my genoa Servers i run performance+performance and pstate epp behaves exactly like expected. Funny thing is that even if i don't care about power consumption, the Genoa Servers run most of the time with a total Consumption of 140W.
Without EPP they take at least 220W.
But like i said, i don't target power consumption, just bare performance, so i setted ctdp to 400W etc....
You can set Targets in the bios to performance or Power, maybe you should try to set the target to Power and set the max ctdp/PPT/PPL/TDP (There are usually 2 settings in the bios)
I would set them all to 200W or 220W, even if your CPU is rated for 200W only 220W should be safe either.
And set the target to Power.

Pstate EPP should still downclock to 400mhz, especially with balance_performance+performance!
As far i understood only balance_performance+performance and performance+performance combos will turbo, every other combination won't turbo, at least in what i know so far.

Cheers

Trying this but do not understand the microcode options that it gives me. The two options are "amd64-microcode_3.20240820.1_amd64.deb" and "amd64-microcode_3.20240820.1~deb12u1_amd64.deb". Which one is correct?

 

[Ramalama]

Trying this but do not understand the microcode options that it gives me. The two options are "amd64-microcode_3.20240820.1_amd64.deb" and "amd64-microcode_3.20240820.1~deb12u1_amd64.deb". Which one is correct?

It has nothing todo with the pstate driver. However i think both does the same job, but for the sake of it i would take the deb12u1 one, because its from the debian guys and gut already an update? or sth.
To be honest, im pretty sure both are the same just different packaged.

Cheers

 

[Tortillas]

It has nothing todo with the pstate driver. However i think both does the same job, but for the sake of it i would take the deb12u1 one, because its from the debian guys and gut already an update? or sth.
To be honest, im pretty sure both are the same just different packaged.

Cheers

Cool, thanks.