Bad memory performance on Windows 10

Dashwood

New Member
Feb 13, 2020
4
2
3
36
Hi all

I set up a new server with a Proxmox VE installation:
- AMD Epyc 7301
- 64 GB DDR4 ECC memory
- 4x SSD in RAID5 with hardware RAID-Controller

I also created a Win10 KVM for code compiling:
- CPU=host and numa=1 (with all cores for testing)
- storage=virtio SCSI
- disk cache = I tested noCache, writeBack and unsafe
All VirtIO drivers, ballooning and guest agent are installed.

The problem now is that the compiling of the code takes more than twice the time as it takes on my few years old Intel i7 6 Gen. workstation with the same software installation and DDR3 memory. All cpu cores are unter full load while compiling.

So I took a system-benchmark on both machines and the VM was overall far better but the memory, which had less than half of the performance as on the workstation.

Next I compared the memory with sysbench on the Proxmox-server and the Workstation:

Sysbench command: sysbench --test=memory --num-threads=4 run

Result on the Proxmox-server (not in a VM):
Total operations: 19500912 (1949722.62 per second)
19043.86 MiB transferred (1904.03 MiB/sec)

Result on the Win10 VM in Windows subsystem for Linux:
Total operations: 8084654 (808256.14 per second)
7895.17 MiB transferred (789.31 MiB/sec)

Result on the Win10 VM booted in a Live Ubuntu:
Total operations: 17865358 (1786183.87 per second)
7895.17 MiB transferred (1744.32 MiB/sec)

Result on the Workstation on Win10 subsystem for Linux:
Total operations: 14111898 (1410967.57 per second)
13781.15 MiB transferred (1377.90 MiB/sec)

To me it looks like a driver problem, since the VM booted with Ubuntu gets nearly the full performance.
Does anyone have an idea what could cause this behaviour?
 
  • Like
Reactions: rushandrush
Hi,

why do you use a win10 with Linux subsystems?
Instead of compiling it in a Linux?

There are so many things involved in such an Environment and it is hard to debug such setup.
 
Hi,

Linux subsystem is only used for the sysbench benchmark to get comparable results from the Windows VM.
MSBuild is used to compile the actual code.
 
Doesn't wsl add a layer of virtualization? I do a lot of Linux development and tried out wsl (v1...IT hasn't updated us yet) to see if it was better than remote servers or regular virtualization. Basically running Linux in hyper-v is significantly faster for compiling code than doing the same task in wsl1. Even virtualbox is faster :cool:
 
Last edited:
Sorry if it´s not clear enough.
I only use WSL to compare sysbench (which is twice as fast on the old physical machine) and also tried a Windows benchmark software with the same result. Everything is faster on the VM except the RAM which gets only half of the benchmark score than the old physical machine.

The actual code is gets compiled with Msbuild and not WSL.
 
I think this is the wrong way to compare it.
It is generally complicated to measure this because Windows and Linux memory systems work entirely different.
Also, the memory allocation function has a massive impact on the results, and they are different on both platforms.
 
Hi, I agree with @wolfgang that the comparison could be in the wrong way.
I'm experiencing the same issue, and I believe there is a memory performance issue in Windows 10 VM.
I did similar things with sysbench as you, but since the result is not comparable, I also go with another approach for comparison:

I installed a Windows 10 Pro in a bare metal way, and run the AIDA64 Cache & Memory Benchmark tool, and then I installed a Proxmox in the same machine with a Windows 10 Pro as a VM on it and run the same benchmark tool again.

On the bare metal:

1597853092390.png

On the VM:

1597852179543.png

my vm config file:

Code:
args: -machine type=q35,kernel_irqchip=on
balloon: 0
bios: ovmf
boot: c
bootdisk: scsi0
cores: 12
cpu: host
efidisk0: local-lvm:vm-103-disk-1,size=4M
hostpci0: 83:00,pcie=1,x-vga=1
machine: q35
memory: 12288
name: SHA003
net0: virtio=xxx,bridge=vmbr0
numa: 1
numa0: cpus=0-5,hostnodes=0,memory=6144,policy=bind
numa1: cpus=6-11,hostnodes=1,memory=6144,policy=bind
ostype: win10
scsi0: local-lvm:vm-103-disk-0,backup=0,cache=writeback,discard=on,iothread=1,replicate=0,size=250G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=xxx
sockets: 1

Hope someone could point out where I went wrong. Very much appreciate it.
 
  • Like
Reactions: rushandrush
I agree that numa didn't explain it. Have you tried wolfgang's suggestion of disabling prefetching?

Hi @SamirD yes, I tried to disable hardware prefetching from BIOS and the performance slightly improves but still I haven't found a way to solve the major problem.

Here is the result of the VM after h/w prefetching is disabled.

1600336106685.png
 
So, i made an Account, just to say, that i have EXACTLY the same Problem. Memory Performance is really bad. I tried to debug this since 3 or 4 Weeks now and I´m all out of Ideas. In a Linux VM i get around 24GB/s with 1M BS which is around the maximum my Board/System can handle. I used the Phoronix Test Suite as a Measurement. In Windows 10 and Windows 7 i use AIDA64 and get around 1.5GB/s read and 900 MB/s write. I also ran Pheronix Test Suite on the Windows VM and got around 100 Times Slower Speeds, compared to a Linux VM.

Things i tried:
Disable/Enable NUMA
Disabled HEPT in the VM and BIOS.
Enabled Hugepages on the Host and inside the Windows VM
Checked Memory on the Host
Changed Performance Options in the BIOS
Pinned CPU Cores
Updated to the Latest Stable Virt-IO Drivers
Asked in Level1Techs Forum
surely there is something missing from this List.

Is there ANYTHING i can test further ?
BTW, CPU Interrupts are in the High 30ks with spikes to 80ks, if the Windows VM is running.

My System:
CPU: Dual Intel x5675 Xeon
MB: Asus Z8NAD6
RAM: 96GB 1066Mhz DDR3 CL7 from Samsung. (On supported List)
Graphics Card: R9 380 (Passed Through to Windows VM)
PSU: 550W Cheapo. Not crappy, but not well either. Good one is on its Way.
 
Last edited:
I think, it's a bug of AIDA64 test algorithm in guest VM, because visually system run fast.
 
I think, it's a bug of AIDA64 test algorithm in guest VM, because visually system run fast.
It's not. Phoronix Test Suite shows the same bad Performance in a various of tests, at least for me.
Proxmox has just really bad Windows VM Performance and I, for myself, am going to switch to ESXi, which is a shame, because then i need another VM for ZFS Stuff.
 
Hmmm.. i have test in my system (WS2019), and get very low speeds (almost 20x lower than on bare metal):
111.jpg

So, i think that it's AIDA64 bug, in other case, with that speeds. WS2019 boots takes very long time. In my case it's around 3-5 seconds....
 
I have test in WinRAR:
Proxmox (6.3): ~7600 KB/s
Hyper-V (2019) ~ 8100 KB/s
Bare metal: ~8600 KB/s
Looks like 13% performance hit, not 20x as AIDA64 shows. So, i think, it's AIDA64 measurements bug in VM.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!