CPU performance on proxmox VM

[LeapSwapnil]

Hello,

I've proxmox node on which CPU performance on the VM is much lower than the node.

https://browser.geekbench.com/v6/cpu/2998119 - node
https://browser.geekbench.com/v6/cpu/2998435 - VM

PFA screenshot.

 

[tjingeling]

The orange text means that the settings you have changed have not been activated yet. You have to stop and then start the VM again to make it use the settings. So currently it's not using "host" as the processor type.

 

[LeapSwapnil]

Hello,

Yes, I am aware of that @tjingeling, I am waiting for a client to get to stop and start.

is there anything else we need to do?

 

[tjingeling]

Just a thought: Is the CPU really the main problem?

The VM underperforms a lot in benchmarks like multi-core Photo Library, while it's not that bad at other calculations.

Maybe it's a disk problem.

 

[aaron]

You give the VM 64 cores, how many cores does the physical host have? If you overprovision the CPU or assign the VM too many cores, it might need to wait a long time to be scheduled as all 64 cores need to be free.

 

[tjingeling]

You give the VM 64 cores, how many cores does the physical host have?

The benchmark detects an AMD EPYC 7543 / 1 Processor, 32 Cores, 64 Threads

 

[aaron]

The benchmark detects an AMD EPYC 7543 / 1 Processor, 32 Cores, 64 Threads

ah thanks. Yep, in that case, don't assign all physical threads to the VM. Leave some for the host and if you have other VMs on the host, assign even less.
Many VMs with a lower CPU core number doable, as they won't need to utilize the CPU constantly (except running simulations and stuff). But having few VMs with many CPU cores, close to the physical number, can cause issues. The scheduler can give the VM only CPU time if the needed cores/threads are free. For example, in that 64 thread machine, getting 16 threads free at the same time is much more likely than all 64.

 

[LeapSwapnil]

We wish to create a single VM on the server and allocate the entire CPU to it. In this case, how can I get the full CPU performance in the VM ?

This is the output when the VM is using CPU - host and has 1 socket 64 cores/threads assigned to it.
https://browser.geekbench.com/v6/cpu/2998435 - VM

As you can see in the main node/server's benchmark, the VM performs 40% slower.
https://browser.geekbench.com/v6/cpu/2998119 - server

Also, the server shows full L3 cache whereas the VM does not get the full L3 cache. How can I pass the full cache to the VM ?

 

[admarty]

We wish to create a single VM on the server and allocate the entire CPU to it. In this case, how can I get the full CPU performance in the VM ?

This is the output when the VM is using CPU - host and has 1 socket 64 cores/threads assigned to it.
https://browser.geekbench.com/v6/cpu/2998435 - VM

As you can see in the main node/server's benchmark, the VM performs 40% slower.
https://browser.geekbench.com/v6/cpu/2998119 - server

Also, the server shows full L3 cache whereas the VM does not get the full L3 cache. How can I pass the full cache to the VM ?

here we go qm set VMID --args "-cpu host,+kvm_pv_eoi,+kvm_pv_unhalt,hv-vendor-id=GenuineIntel,+host-cache-info,+invtsc,+apic"
and dont forget to enable NUMA

 

[aaron]

We wish to create a single VM on the server and allocate the entire CPU to it. In this case, how can I get the full CPU performance in the VM ?

You will always leave some resources for the host available! So maybe only give the VM 60 cores, that leaves 4 threads available for the host, to run the web interface and all the other services needed.

 

[JensF]

Well, virtualization is for server workloads consolidation. If you have one application which needs all resources of a server, there is no need for virtualization and just install it bare metal...

 

[Dunuin]

Setting the CPU type to "host" and disabling mitigations via grub might help a bit.

 

[ITT]

The benchmark detects an AMD EPYC 7543 / 1 Processor, 32 Cores, 64 Threads

So, the way to go is 30 vCores with cpulimit=30.

 

[linkpeng0]

Hi,

I recommend using Geekbench 5 for testing large cores of CPU(more than 16), Geekbench 6 does not optimize for large cores.

Different operating systems(or kernels) will cause different result in details of scores , no matter it is guest or host. In Geekbench 5 , Windows always get higher score in "AES-XTS" test than Linux, but the other scores may lower than Linux. You can run Geekbench 5 in different systems and see the details of scores. Note that Geekbench total score is a "mixed score".

To compare performance between physical and virtual machine , you should run same test in same os(and kernel) on host and guest (host AlmaLinux 9.2(kernel 5.14) vs AlmaLinux 9.2(kernel 5.14) VM in PVE 8).

This is my Geekbench 5 test on EPYC 9554p:
https://forum.proxmox.com/threads/epyc-9554p-with-pve-8-0-benchmark.134070/

 

[VictorSTS]

So, the way to go is 30 vCores with cpulimit=30.

Why limiting also with cpulimit?
QEMU I/O or network threads shouldn't take that much more host CPU time. I don't know if the host kernel will schedule them within the 30 vCores of the VM or in some other cores of the host.