numa and cpu pinning

[robertbovens]

Hello,

i am trying to pin cpus to a numa node but find it difficult to understand why it is not working, i have the impression it is a bug, maybe someone can enlighten me with the issue i have.

These are my findings at the moment:

numa1:cpus=8-11,hostnodes=1,memory=4096,policy=bind
hugepages: 1024
kvm: -numa node,nodeid=1,cpus=8-11,memdev=ram-node1: CPU index (8) should be smaller than maxcpus (4)

numa1:cpus=8-11,hostnodes=1,memory=4096,policy=bind
hugepages:1024
start failed: hugepage allocation failed at /usr/share/perl5/PVE/QemuServer/Memory.pm line 541.

numa1 :cpus=8-11,hostnodes=1,memory=4096,policy=bind
#hugepages :1024
starts with no issues

numa1: cpus=8-11,hostnodes=1,memory=4096,policy=bind
#hugepages: 1024
kvm: -numa node,nodeid=1,cpus=8-11,memdev=ram-node1: CPU index (8) should be smaller than maxcpus (4)

numa1:cpus=8-11,hostnodes=1,memory=4096,policy=bind
#hugepages: 1024
kvm: -numa node,nodeid=1,cpus=8-11,memdev=ram-node1: CPU index (8) should be smaller than maxcpus (4)

numa1 :cpus=8-11,hostnodes=1,memory=4096,policy=bind
hugepages :1024
no error

numa1: cpus=8-11,hostnodes=1,memory=4096,policy=bind
#hugepages: 1024
kvm: -numa node,nodeid=1,cpus=8-11,memdev=ram-node1: CPU index (8) should be smaller than maxcpus (4)

numa1: cpus=8-11,hostnodes=1,memory=4096,policy=bind
hugepages: 1024
will give errror with the hugepages

numa1: cpus=8-11, hostnodes=1, memory=4096, policy=bind
#hugepages: 1024
Give me the following error:
vm 101 - unable to parse value of 'numa1' - format error
policy: property is not defined in schema and the schema does not allow additional properties
hostnodes: property is not defined in schema and the schema does not allow additional properties
memory: property is not defined in schema and the schema does not allow additional properties
vm 101 - unable to parse value of 'numa1' - format error
hostnodes: property is not defined in schema and the schema does not allow additional properties
memory: property is not defined in schema and the schema does not allow additional properties
policy: property is not defined in schema and the schema does not allow additional properties


Can someone advise me how the config should look like ?

Robert

 

[dcsapak]

the cpus you give in the config are the ones from the vm, not the host
you can only bind the cores to a numa node of the host, not specific cores with that feature

 

[dcsapak]

CPU index (8) should be smaller than maxcpus (4)

you should probably write
cpus=1-4 (or 0-3, i can not remember if it starts with 0 or 1)

 

[robertbovens]

When i assign 4 cores to the vm, it will be automatically 0 1 2 3.

For me it doesnt make any sense why i would want or need to specify that again in the numa config.

 

[robertbovens]

Will cpu pinning be supported in the future? Would be nice if it would be able to pin those through the gui...

Now i have a duel socket ibm 3650 m4bd server with only 16GB.

I notice the vms go slower over time and when i use numastat i can see node misses. I also get a log irq took to long or something like that, not behind the computer at this moment

 

[dcsapak]

When i assign 4 cores to the vm, it will be automatically 0 1 2 3.

For me it doesnt make any sense why i would want or need to specify that again in the numa config.

please read my post and the docs[0] again

with the numa config you specify a numaX config
there will be a virtual numa node in the guest and you have to specify:
* the guest cores belonging to the virtual numa node
* the host numa node to bind to
* the amount of memory belonging to that numa node

0: https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines

a good config would be to have as much cores in the vm as a single numa node on the host
and assign it to a numa node

or simply replicate the numa config from the host
(e.g. if your host hast 2 numa nodes, create a vm with 2 sockets and 2 numa nodes)

if the guest is then numa aware, there should not be any node misses