More then anything, wondering what project is feeding the Linux topology to the KVM topology buckets. (cluster_cpus, cluster_id, core_cpus_list, core_siblings, die_cpus, die_id, package_cpus_lis, ppin, thread_siblings_list, cluster_cpus_list, core_cpus, core_id, core_siblings_list, die_cpus_list, package_cpus, physical_package_id, thread_siblings)
But this is what we are seeing. Example below is from a 16c/32t EPYC 7002 single socket test system.
lstopo (hwloc)
1socket 16core 4CCD 8CCX 7002 EPYC
SMC H11SSL-i BIOS 2.8
Default APIC tables, UMA memory, no L3 as NUMA, no NPS
From KVMs point of view
At the very least we would expect the DIE ID and Core ID to be presented at the L3 Cache NUMA domains and they are not. This makes it impossible to pass the physical topology up through the virtual topology to VMs by using -smp values.
Also there is a incoming change to better support Big.Little alderlake support to include clusters - https://lore.kernel.org/kvm/20240321144048.3699388-7-zhao1.liu@linux.intel.com/T/ which will help in a major way for Intel's platform.
Anyway, I dont see this as a Proxmox issue but rather a KVM issue. Just wondering if anyone here knows what project handles the transition from Linux to the KVM topology in this way.
But this is what we are seeing. Example below is from a 16c/32t EPYC 7002 single socket test system.
lstopo (hwloc)
1socket 16core 4CCD 8CCX 7002 EPYC
SMC H11SSL-i BIOS 2.8
Default APIC tables, UMA memory, no L3 as NUMA, no NPS
Package L#0 NUMANode L#0 (P#0 126GB) L3 L#0 (16MB) L2 L#0 (512KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 PU L#0 (P#0) PU L#1 (P#16) L2 L#1 (512KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 PU L#2 (P#1) PU L#3 (P#17) L3 L#1 (16MB) L2 L#2 (512KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 PU L#4 (P#2) PU L#5 (P#18) L2 L#3 (512KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 PU L#6 (P#3) PU L#7 (P#19) L3 L#2 (16MB) L2 L#4 (512KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4 PU L#8 (P#4) PU L#9 (P#20) L2 L#5 (512KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5 PU L#10 (P#5) PU L#11 (P#21) L3 L#3 (16MB) L2 L#6 (512KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6 PU L#12 (P#6) PU L#13 (P#22) L2 L#7 (512KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7 PU L#14 (P#7) PU L#15 (P#23) L3 L#4 (16MB) L2 L#8 (512KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8 PU L#16 (P#8) PU L#17 (P#24) L2 L#9 (512KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9 PU L#18 (P#9) PU L#19 (P#25) L3 L#5 (16MB) L2 L#10 (512KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10 PU L#20 (P#10) PU L#21 (P#26) L2 L#11 (512KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11 PU L#22 (P#11) PU L#23 (P#27) L3 L#6 (16MB) L2 L#12 (512KB) + L1d L#12 (32KB) + L1i L#12 (32KB) + Core L#12 PU L#24 (P#12) PU L#25 (P#28) L2 L#13 (512KB) + L1d L#13 (32KB) + L1i L#13 (32KB) + Core L#13 PU L#26 (P#13) PU L#27 (P#29) L3 L#7 (16MB) L2 L#14 (512KB) + L1d L#14 (32KB) + L1i L#14 (32KB) + Core L#14 PU L#28 (P#14) PU L#29 (P#30) L2 L#15 (512KB) + L1d L#15 (32KB) + L1i L#15 (32KB) + Core L#15 PU L#30 (P#15) PU L#31 (P#31)lscpuCaches (sum of all): L1d: 512 KiB (16 instances) L1i: 512 KiB (16 instances) L2: 8 MiB (16 instances) L3: 128 MiB (8 instances)NUMA: NUMA node(s): 1 NUMA node0 CPU(s): 0-31From KVMs point of view
Package ID - 0Die ID - 0Cluster ID - 65535Core ID - 0,1,2,3,...30,31Thread sibling ID - 0,16 1,17 2,16 3,15 .... 13,29 14,30 15,31package CPU - 0-31die CPU - 0-31Cluster CPus - 0,16 1,17 2,16 3,15 .... 13,29 14,30 15,31CPUs - 0,16 1,17 2,16 3,15 .... 13,29 14,30 15,31Core Sibling - 0-31At the very least we would expect the DIE ID and Core ID to be presented at the L3 Cache NUMA domains and they are not. This makes it impossible to pass the physical topology up through the virtual topology to VMs by using -smp values.
Also there is a incoming change to better support Big.Little alderlake support to include clusters - https://lore.kernel.org/kvm/20240321144048.3699388-7-zhao1.liu@linux.intel.com/T/ which will help in a major way for Intel's platform.
Anyway, I dont see this as a Proxmox issue but rather a KVM issue. Just wondering if anyone here knows what project handles the transition from Linux to the KVM topology in this way.
Last edited: