Improving power efficiency and configuring ASPM

S

sleeper52

New Member
Mar 24, 2023
22
4
3
I'm currently on Proxmox 8.0.4 running kernel 6.2.16-12-pve and I've been working on improving the power efficiency of my server.
I need help to optimally configure the ASPM particularly for the Intel Arc A380 and my Dell H310 HBA card (should ASPM be enabled for this device?)
When looking at the current ASPM status, some devices are still disabled. How do I enable ASPM for these devices and which ones should I keep disabled?

SPECS
Code: 
CPU:  AMD Ryzen 5900x
MB: Asus ROG Strix X570-E Gaming WiFi II (4602XII Modded BIOS)
RAM:  Corsair Vengeance 2x32GB 3600MT C18 DDR4
GPU: ASRock Intel Arc A380 Challenger
PSU: Seasonic G550 Modular 550w
HBA: Dell H310 (LSI SAS2008)
Remote Access: Geekworm PiKVM-A8
Boot: 128GB Samsung 840 Pro
SSDs: 2x 1TB WD WDS100T2B0A
HDDs: 2x4TB in mirror (HGST Ultrastar); 3x10TB in RAIDZ1 (HGTST Ultrastar); 3x18TB in RIADZ1 (Seagate Exos x18)

AMD Pstate driver is enabled by configuring /etc/defaultgrub as follows
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_pstate=passive initcall_blacklist=acpi_cpufreq_init amd_pstate.shared_mem=1 cpufreq.default_governor=powersave"
Code: 
kristian@pve:~$ cpupower frequency-info
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 20.0 us
  hardware limits: 550 MHz - 4.98 GHz
  available cpufreq governors: conservative ondemand userspace powersave performance schedutil
  current policy: frequency should be within 550 MHz and 4.98 GHz.
                  The governor "powersave" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 550 MHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes
    AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.98 GHz.
    AMD PSTATE Nominal Performance: 130. Nominal Frequency: 3.90 GHz.
    AMD PSTATE Lowest Non-linear Performance: 61. Lowest Non-linear Frequency: 1.83 GHz.
    AMD PSTATE Lowest Performance: 20. Lowest Frequency: 550 MHz.

BIOS Settings

SETTING
CONFIGURATION
NOTES

PSS Support
Enabled

CPU PCIE ASPM Mode Control
L0s And L1 Entry
Other options: Auto, L0s Entry, L1 Entry

ErP Ready
Enable(S4+S5)

ASPM Support
Auto
Other options: Force L0s

PM L1 SS
Auto
Not sure what this does entirely;
other options: L1.1, L1.2, L1.1_L1.2

Ai Overclock Tuner
DOCP (1.35V DRAM Voltage)

AMD Overclocking\ECO Mode
Eco-Mode (65W)

Curve Optimizer
All Core Curve Optimizer Sign
All Core Curve Optimizer Magnitude
All Cores
Negative
30

Precision Boost Overdrive
PBO Limits
PPT Limit
TDC Limit
EDC Limit
Precision Boost Overdrive Scalar
Advanced
Manual
87
60
90
Auto

CPU Boost Clock Override
Enabled (Negative)

Max CPU Boost Clock Override(-)
250

Power Supply Idle Control
Low Current Idle

EfficiencyModeEn
Enabled

DF Cstates
Enabled

Global C-state Control
Enabled

IOMMU
Enabled

CPPC
Enabled

CPPC Preferred Cores
Disabled


Current ASPM status
Code: 
kristian@pve:~$ sudo dmesg | grep ASPM
[    0.988560] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI EDR HPX-Type3]
[    1.002169] pci 0000:09:00.0: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
kristian@pve:~$ sudo lspci -vv | awk '/ASPM/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM )'
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L1, Exit Latency L1 <32us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
00:03.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #2, Speed 5GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x8, ASPM L1, Exit Latency L1 <32us
                LnkCtl: ASPM L1 Enabled; Disabled- CommClk+
02:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 16GT/s, Width x1, ASPM L1, Exit Latency L1 <32us
                LnkCtl: ASPM Disabled; Disabled- CommClk+
02:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; Disabled- CommClk+
02:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; Disabled- CommClk+
02:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; Disabled- CommClk+
03:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
                LnkCap: Port #5, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
04:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
04:00.1 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller (prog-if 30 [XHCI])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
04:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller (prog-if 30 [XHCI])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
05:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
06:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
07:00.0 PCI bridge: Intel Corporation Device 4fa1 (rev 01) (prog-if 00 [Normal decode])
                LnkCap: Port #0, Speed 16GT/s, Width x8, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM Disabled; Disabled- CommClk+
08:01.0 PCI bridge: Intel Corporation Device 4fa4 (prog-if 00 [Normal decode])
                LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L1 <1us
                LnkCtl: ASPM Disabled; Disabled- CommClk-
08:04.0 PCI bridge: Intel Corporation Device 4fa4 (prog-if 00 [Normal decode])
                LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L1 <1us
                LnkCtl: ASPM Disabled; Disabled- CommClk+
09:00.0 VGA compatible controller: Intel Corporation DG2 [Arc A380] (rev 05) (prog-if 00 [VGA controller])
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
pcilib: sysfs_read_vpd: read failed: No such device
0a:00.0 Audio device: Intel Corporation DG2 Audio Controller
                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
0b:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
                LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Exit Latency L0s <64ns
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
0c:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0d:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0d:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
0d:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller (prog-if 30 [XHCI])
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk+
 
Last edited:
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back