So i am using opnsense as a vm. With multiple rounds of testing i seem to be getting the best download speeds when using cpu affinity and only 4 cores. I pinned this vm to cores 1,3 ,5 and 7 since these are on numa node 1, the same node that the nic i am passing through is directly connected to. For some reason when pinning more then 4 cores it seems they do not boost as high and i loose speed. This is another question i have. My main question for this thread is whether i should enable the numa option on the vm. Do i still need to use this option even if the vm is only using cpu cores from a single numa node? I have read stuff online that the numa option relates to memory. What is the correct options to run to get the absolute best performance?
Numa Settings for dual socket Dell R630 with 2x E5-2667 V4
- Thread starter AWP Technologies
- Start date
Enabling NUMA and setting the number of virtual sockets to the same number as the NUMA domains (usually the physical sockets) makes Proxmox allocate the memory for the VM on each domain and pass this information to the VM (which will then also see NUMA domains). That way, the VM knows about the higher latency (w.r.t. the virtual core on the virtual socket) for each part of the memory that is connected to a different CPU and it can make allocation decisions based on that.
Whether this is helps your case, I cannot say. Your PCI(e) device is connected to one of the sockets and might have additional latency if it is accessed from another CPU. I don't think Proxmox takes this into consideration. There is a recent thread with a similar question but I can't find it right now.
EDIT: Here is the thread: https://forum.proxmox.com/threads/how-smart-is-proxmox-ve-choosing-correct-cpu-to-run-a-vm.160867/
Whether this is helps your case, I cannot say. Your PCI(e) device is connected to one of the sockets and might have additional latency if it is accessed from another CPU. I don't think Proxmox takes this into consideration. There is a recent thread with a similar question but I can't find it right now.
EDIT: Here is the thread: https://forum.proxmox.com/threads/how-smart-is-proxmox-ve-choosing-correct-cpu-to-run-a-vm.160867/
Last edited:
Thank you for your response as of right now i have sockets set to 1 and cores set to 4 with affinity set to 1,3,5,7 with numa option enabled. This seems to give me the best performance with my opnsense vm. Running intrusion detection netflow and ntopng i still get the full 1gig from spectrum. Before with 2 sockets and 5 cores per so 10 vcpus in all and numa enabled with no affinity i was getting capped at 350 down. Now with only 4 vcpus and affinity i get 980 down
When assigning affinity cores is it better to assign the first set of cores or the las set. Say my cpu has cores 0 - 31 is it better to set affinity to 0-3 or 28-31