[SOLVED] NUMA hostnodes and affinity

[lenny47]

I am seeing a guest core able to execute outside its numa node.

Eg. loading up guest cores 0-7 (which are mapped to node0) results in utilising host cores 32,35,45,47,48,49,52,55 which are across 3 different nodes.

How can I get the host NUMA arch to be correctly reflected in the guest?

Guest lscpu:
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23

Host lscpu
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23
NUMA node3 CPU(s): 24-31
NUMA node4 CPU(s): 32-39
NUMA node5 CPU(s): 40-47
NUMA node6 CPU(s): 48-55
NUMA node7 CPU(s): 56-63

VM config contains:
affinity: 32-55
cores: 8
numa: 1
numa0: cpus=0-7,hostnodes=4,memory=131072,policy=bind
numa1: cpus=8-15,hostnodes=5,memory=131072,policy=bind
numa2: cpus=16-23,hostnodes=6,memory=131072,policy=bind
sockets: 3

 

[lenny47]

Solved from https://forum.proxmox.com/threads/hey-proxmox-community-lets-talk-about-resources-isolation.124256/ and using the pin function of script mention there.

Summary: numa<N> lines don't result in vcpu pinning to correct host cpus on its own, you need to do this manually on the pve host (not exposed in proxmox gui or config).

I am seeing a guest core able to execute outside its numa node.

Eg. loading up guest cores 0-7 (which are mapped to node0) results in utilising host cores 32,35,45,47,48,49,52,55 which are across 3 different nodes.

How can I get the host NUMA arch to be correctly reflected in the guest?

Guest lscpu:
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23

Host lscpu
NUMA node0 CPU(s): 0-7
NUMA node1 CPU(s): 8-15
NUMA node2 CPU(s): 16-23
NUMA node3 CPU(s): 24-31
NUMA node4 CPU(s): 32-39
NUMA node5 CPU(s): 40-47
NUMA node6 CPU(s): 48-55
NUMA node7 CPU(s): 56-63

VM config contains:
affinity: 32-55
cores: 8
numa: 1
numa0: cpus=0-7,hostnodes=4,memory=131072,policy=bind
numa1: cpus=8-15,hostnodes=5,memory=131072,policy=bind
numa2: cpus=16-23,hostnodes=6,memory=131072,policy=bind
sockets: 3