Best Practice for NUMA?

[e100]

I have a few dual socket servers and want to know how best to configure VMs.

Should VMs always have NUMA enabled and have CPU sockets set to the number of physical sockets?

Some VMs only need a single socket and single core, these should have NUMA enabled too?

Some VMs only need two cores, is it better to use two sockets and one core vs once socket and two cores?

 

[bogesman]

Good question, i also would like to know.

 

[LnxBil]

As far as I know, virtualized cores 1x2 or 2x1 processor does not matter, as long as they do not have more cores than the physical cpu and numa enabled.

The NUMA checkbox in Proxmox will result in better performance due to local memory placement in multiprocessor systems

Here is an article that tried to explain what happens:
http://www.admin-magazine.com/Archive/2014/20/Best-practices-for-KVM-on-NUMA-servers

Also you will get information in the forum itself:
https://forum.proxmox.com/threads/numa-config-option.21313/

 

[spirit]

I have a few dual socket servers and want to know how best to configure VMs.

Should VMs always have NUMA enabled and have CPU sockets set to the number of physical sockets?

Some VMs only need a single socket and single core, these should have NUMA enabled too?

Some VMs only need two cores, is it better to use two sockets and one core vs once socket and two cores?

I think it's depend of how much memory how you have in your vms.

For example, you have a 128GB server, with 2 sockets x 4cores. (so 2 numa nodes with 64GB).

If the vm have < 64GB ram and <=4cores, you can use 1socket for your vm. like this all cores will try to go to same numa node, and will have better access memory access.

Of course, if you have a lot of vms, maybe you can simply always use 2 sockets in your guests, and split the cores across the numa nodes

for vm with 1socket/1core, enabling numa can avoid the single qemu process to switch between physical numa nodes, so yes, it's helping too.


You can use the command "numastat" on host, to see numa_hit ratio.

 

[LnxBil]

On some AMD systems I also saw multiple NUMA nodes on one CPU, so please check as @spirit already mentioned with numastat.

 

[spirit]

On some AMD systems I also saw multiple NUMA nodes on one CPU, so please check as @spirit already mentioned with numastat.

yes, with amd, it's a little more complex, because they also use numa nodes inside socket itself.