Hello,
I know hyperthreading comes up from time to time, and I have read several of the other messages about it over time, but the recommendations I've seen a few times don't seem to hold up with our testing, so I hope you don't mind me asking about it again. I'm trying to learn some more insights, and to see if I've missed anything.
In testing proxmox a while back (pve 8) on a 4x14 xeon server running postgresql benchmarks, using 4x13 for postgres vm out performed 4x16 vm.
the host had 56 physical cores, and 56 hyperthread "cores" proxmox showing 112 as expected. The issue is on a busy postgresql database, if we assign more than 52 cores to the VM then performance degrades. If we assign 56 cores, then the proxmox host becomes laggy and the agent displays don't reliably update and we may or may not be able to get in to the "console".
This means the typical suggestion to provision all your "cores" or to provision beyond physical cores, and let proxmox manage it is not really an effective option for us.
this means we have to leave at least 2 and typically 4 cores to the host, and limit the busy VMs to physical cores - 4 to keep everything happy. That seemingly leaves a lot of power on the table. even with the estimated 30% performance of a hyperthread "core", that's theoretically 16 "cores" untapped. Yet if we go above physical cores the VM and even the host bogs down.
This is for a fairly busy database. and we proved with the standard pgsql benchmarks ( i no longer have the test data or procedure notes, but we used external clients, 10 gb network, and it was all high speed ssd), and as soon as you hit 56 or higher vcpus in the vm, the database performance dropped.
What I've done for the few blades I need to run this way is switch to Host cpu, disable numa, disable memory ballooning, and allocate all but ~48 gb of memory to the VM (so the blade has 4 cores and 48 gb of memory, and the VM gets everything else), and it works pretty well, a bit faster than having those on. we lose some flexibility but it gets us by.
However, I'd like to see if we can pick up some of what's been left on the table. Anyone have suggestions or ideas on how to pick up some of that wasted power?
Thanks
I know hyperthreading comes up from time to time, and I have read several of the other messages about it over time, but the recommendations I've seen a few times don't seem to hold up with our testing, so I hope you don't mind me asking about it again. I'm trying to learn some more insights, and to see if I've missed anything.
In testing proxmox a while back (pve 8) on a 4x14 xeon server running postgresql benchmarks, using 4x13 for postgres vm out performed 4x16 vm.
the host had 56 physical cores, and 56 hyperthread "cores" proxmox showing 112 as expected. The issue is on a busy postgresql database, if we assign more than 52 cores to the VM then performance degrades. If we assign 56 cores, then the proxmox host becomes laggy and the agent displays don't reliably update and we may or may not be able to get in to the "console".
This means the typical suggestion to provision all your "cores" or to provision beyond physical cores, and let proxmox manage it is not really an effective option for us.
this means we have to leave at least 2 and typically 4 cores to the host, and limit the busy VMs to physical cores - 4 to keep everything happy. That seemingly leaves a lot of power on the table. even with the estimated 30% performance of a hyperthread "core", that's theoretically 16 "cores" untapped. Yet if we go above physical cores the VM and even the host bogs down.
This is for a fairly busy database. and we proved with the standard pgsql benchmarks ( i no longer have the test data or procedure notes, but we used external clients, 10 gb network, and it was all high speed ssd), and as soon as you hit 56 or higher vcpus in the vm, the database performance dropped.
What I've done for the few blades I need to run this way is switch to Host cpu, disable numa, disable memory ballooning, and allocate all but ~48 gb of memory to the VM (so the blade has 4 cores and 48 gb of memory, and the VM gets everything else), and it works pretty well, a bit faster than having those on. we lose some flexibility but it gets us by.
However, I'd like to see if we can pick up some of what's been left on the table. Anyone have suggestions or ideas on how to pick up some of that wasted power?
Thanks
