After reading countless discussions, articles, whitepapers and not finding suitable answer, I decided to take matter in hand and do some testing to come to conclusion, if i should be using Hyper-Threading or not. While trying to find this answer, i also found the answer of using Fixed or Variable Memory for VMs in Proxmox.
[[Disclaimer: These are all based on my testing and configuration so my opinion may be biased towards my virtual environment]]
Short Answer
-YES!! Use Hyper-Threding whenever possible, even on a newer multi CPU node.
-Use Fixed memory allocation in VMs where performance is needed. Do Not use variable memory specially on Windows based VM.
Long Answer
Hardware used for testing
-------------------------
Node 1 = Dual Xeon E5-5620v2, 64GB Ram, Hyper-Threading Disabled, Total core 12(12+0 HT)
Node 2 = Dual Xeon E5-5620v2, 64GB Ram, Hyper-Threading Enabled, Total core 24(12+12 HT)
Initial VMs used for testing
--------------------------
2 x Windows Server 2008 R2 64 Bit - 6 vCPU, Max 16GB Ram Min. 4GB Ram
In my environment both Windows Servers are RDS server. Each with about 20 users. I used Performance monitor in Windows to gather performance data. No other tool was used. Initially i put both servers on Node 2 but we have been some performance issue for some time. I did not want to believe my hardware was not capable enough to run just 2 RDS servers. Performance monitor showed RAM was not even used up half way but CPU consumption were above 80% during work hours, sometimes hitting 100% and staying there. The RDS servers only have standard office programs such as MS Office, Quickbooks, etc. No graphic heavy work not even youtube. To make sure that it was not the hardware problem, i moved one of the RDS server to Node 1 which had HT enabled at that time. No noticeable difference really. To find out it wasnt the HT causing issue, i disabled HT on the first node, but the performance on that VM actually got worse. Rebooted the VM with HT enabled and it got better.
Trying many difference settings such as:
-increasing/decreasing vCPU, memory
-fixing/increasing/decreasing windows pagefile
-putting VMs on local disk rather than RBD
Lastly, i rebooted the VM without any other user logged in or program running. But the performance monitor showed about 45% CPU were getting consumed on both VMs. The VMs on Node 1 with HT disabled was actually showing slightly higher consumption. In all cases memory consumption were way down.
Finally, i disabled variable memory and fixed it 16GB. After rebooting the VM, performance monitor showed..................... 1% consumption!!! Did the same setting to HT disabled node VM, same result. CPU was quiet! After logging all users with RDP and running programs to simulate maximum load in all their session the consumption went up to 49% on HT enabled node and 61% on HT disabled node. I was even able to drop the vCPU count to 2 and make it work just fine. But since it is a RDS server and user might cause spike sometime, i changed it to 4 vCPU.
I configured fixed memory on ALL the VMs both windows and linux in the cluster and were able to reclaim 32 vCPUs!!!! The VMs that were running with high CPU usage given the nature of their role now churning away just fine with lower vCPU count.
I should mention that in our clusters we keep very close eye on vCPU allocation so we do not over allocate or abuse HT enabled nodes. So the vCPUs that were assigned previously were needed to fulfill consumption. With all Windows VMs memory fixed i did HT or No HT test again. Same VM seemed happier on HT enabled node than disabled one. Again, all these from performance monitor data.
Even with lower vCPU count and fixed memory our RDP users are much happier, as per the reports we have getting all day. I monitored the clusters for last 2 days, vCPUs are much quieter now with fixed memory. Just to make sure fixed memory is the one made the difference, i changed one of the VM to old setting with variable memory and the same old high vCPU consumption came back. Changed it back to fixed, consumption went down significantly.
So the question to all, is it normal behavior? Anybody else had or noticed similar trend? I found many articles with talk of HT or No HT. Some said disable HT for multi CPU nodes. Some said just leave it on. I am sure many of you tried to find the same answer. My finding says leave it on. I am still monitoring the cluster. So far things looks much better on HT enabled than disabled.
Also why fixed memory will consume less vCPU than variable? Although things are working pretty excellent with these changes, i would like to know "why" and possibly other experiences.
[[Disclaimer: These are all based on my testing and configuration so my opinion may be biased towards my virtual environment]]
Short Answer
-YES!! Use Hyper-Threding whenever possible, even on a newer multi CPU node.
-Use Fixed memory allocation in VMs where performance is needed. Do Not use variable memory specially on Windows based VM.
Long Answer
Hardware used for testing
-------------------------
Node 1 = Dual Xeon E5-5620v2, 64GB Ram, Hyper-Threading Disabled, Total core 12(12+0 HT)
Node 2 = Dual Xeon E5-5620v2, 64GB Ram, Hyper-Threading Enabled, Total core 24(12+12 HT)
Initial VMs used for testing
--------------------------
2 x Windows Server 2008 R2 64 Bit - 6 vCPU, Max 16GB Ram Min. 4GB Ram
In my environment both Windows Servers are RDS server. Each with about 20 users. I used Performance monitor in Windows to gather performance data. No other tool was used. Initially i put both servers on Node 2 but we have been some performance issue for some time. I did not want to believe my hardware was not capable enough to run just 2 RDS servers. Performance monitor showed RAM was not even used up half way but CPU consumption were above 80% during work hours, sometimes hitting 100% and staying there. The RDS servers only have standard office programs such as MS Office, Quickbooks, etc. No graphic heavy work not even youtube. To make sure that it was not the hardware problem, i moved one of the RDS server to Node 1 which had HT enabled at that time. No noticeable difference really. To find out it wasnt the HT causing issue, i disabled HT on the first node, but the performance on that VM actually got worse. Rebooted the VM with HT enabled and it got better.
Trying many difference settings such as:
-increasing/decreasing vCPU, memory
-fixing/increasing/decreasing windows pagefile
-putting VMs on local disk rather than RBD
Lastly, i rebooted the VM without any other user logged in or program running. But the performance monitor showed about 45% CPU were getting consumed on both VMs. The VMs on Node 1 with HT disabled was actually showing slightly higher consumption. In all cases memory consumption were way down.
Finally, i disabled variable memory and fixed it 16GB. After rebooting the VM, performance monitor showed..................... 1% consumption!!! Did the same setting to HT disabled node VM, same result. CPU was quiet! After logging all users with RDP and running programs to simulate maximum load in all their session the consumption went up to 49% on HT enabled node and 61% on HT disabled node. I was even able to drop the vCPU count to 2 and make it work just fine. But since it is a RDS server and user might cause spike sometime, i changed it to 4 vCPU.
I configured fixed memory on ALL the VMs both windows and linux in the cluster and were able to reclaim 32 vCPUs!!!! The VMs that were running with high CPU usage given the nature of their role now churning away just fine with lower vCPU count.
I should mention that in our clusters we keep very close eye on vCPU allocation so we do not over allocate or abuse HT enabled nodes. So the vCPUs that were assigned previously were needed to fulfill consumption. With all Windows VMs memory fixed i did HT or No HT test again. Same VM seemed happier on HT enabled node than disabled one. Again, all these from performance monitor data.
Even with lower vCPU count and fixed memory our RDP users are much happier, as per the reports we have getting all day. I monitored the clusters for last 2 days, vCPUs are much quieter now with fixed memory. Just to make sure fixed memory is the one made the difference, i changed one of the VM to old setting with variable memory and the same old high vCPU consumption came back. Changed it back to fixed, consumption went down significantly.
So the question to all, is it normal behavior? Anybody else had or noticed similar trend? I found many articles with talk of HT or No HT. Some said disable HT for multi CPU nodes. Some said just leave it on. I am sure many of you tried to find the same answer. My finding says leave it on. I am still monitoring the cluster. So far things looks much better on HT enabled than disabled.
Also why fixed memory will consume less vCPU than variable? Although things are working pretty excellent with these changes, i would like to know "why" and possibly other experiences.
Last edited:
