Hello. I decided to check the performance of the disk subsystem on a fresh installation of Proxmox 5.4. At the heart of the Dell PowerEdge R720 server with a PERC H710 mini controller. The disk subsystem is assembled as follows Virtual Disks:
Raid-1: 2 * Toshiba DT01ACA2 - 2 Tb (Proxmox + ISO + Backup + Test VM)
Raid-10: 4 * Seagate Exos 7E8 - 4 Tb (Main storage lvmthin)
Raid-1: 1 * Samsung 750 EVO - 512 Gb (SSD storage lvmthin)
CachCade: 1 * Kingston SVP200S SSD - 120 Gb
System parameters
Test Virtual Machine Configuration (Operating system is Windows Server 2019 Standard)
agent: 1
bootdisk: virtio0
cores: 4
cpu: host
machine: q35
memory: 4096
name: test
net0: virtio=1E:B4:3B:AF:2F:74,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
sata0: none,media=cdrom
scsihw: virtio-scsi-pci
smbios1: uuid=53d6aa22-4cba-477e-a67b-e6195ddf3b5e
sockets: 1
virtio0: local:101/vm-101-disk-0.qcow2,cache=writeback,size=32G
virtio1: raid:vm-101-disk-0,cache=writeback,size=32G
virtio2: ssd:vm-101-disk-0,cache=writeback,size=32G
virtio3: local:101/vm-101-disk-1.qcow2,cache=writeback,size=32G
vmgenid: cb95ac6d-cfbe-4e90-80a3-685648b9c297
Testing was performed using the utility CrystalDiskMark 6.0.2 x64
I got some unreal results. Can someone comment on them?
For example test on RAID-10:
1 Gb without Write Back Cache
-----------------------------------------------------------------------
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : https://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 490.872 MB/s
Sequential Write (Q= 32,T= 1) : 471.191 MB/s
Random Read 4KiB (Q= 8,T= 8) : 105.181 MB/s [ 25679.0 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 7.354 MB/s [ 1795.4 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 107.443 MB/s [ 26231.2 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 7.235 MB/s [ 1766.4 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 9.617 MB/s [ 2347.9 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 6.541 MB/s [ 1596.9 IOPS]
Test : 1024 MiB [E: 0.2% (0.1/32.0 GiB)] (x5) [Interval=5 sec]
Date : 2019/05/13 13:25:59
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 17763] (x64)
1 Gb with Write Back Cache
-----------------------------------------------------------------------
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : https://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 10153.791 MB/s
Sequential Write (Q= 32,T= 1) : 6770.621 MB/s
Random Read 4KiB (Q= 8,T= 8) : 707.773 MB/s [ 172796.1 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 617.034 MB/s [ 150643.1 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 555.786 MB/s [ 135689.9 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 393.689 MB/s [ 96115.5 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 18.836 MB/s [ 4598.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 17.976 MB/s [ 4388.7 IOPS]
Test : 1024 MiB [E: 0.2% (0.1/32.0 GiB)] (x5) [Interval=5 sec]
Date : 2019/05/13 13:38:07
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 17763] (x64)
The full test results document can be downloaded from Google Drive.
My colleague sent me the results from a similar server with the same Raid controller on which ESXi is installed and 4 SSDs are assembled in 10 Raid. The result is much lower. At the same time, it coincides with the result in the virtual machine even if the Windows Server is deployed on the host. How can this be explained?
Raid-1: 2 * Toshiba DT01ACA2 - 2 Tb (Proxmox + ISO + Backup + Test VM)
Raid-10: 4 * Seagate Exos 7E8 - 4 Tb (Main storage lvmthin)
Raid-1: 1 * Samsung 750 EVO - 512 Gb (SSD storage lvmthin)
CachCade: 1 * Kingston SVP200S SSD - 120 Gb
System parameters
Test Virtual Machine Configuration (Operating system is Windows Server 2019 Standard)
agent: 1
bootdisk: virtio0
cores: 4
cpu: host
machine: q35
memory: 4096
name: test
net0: virtio=1E:B4:3B:AF:2F:74,bridge=vmbr0,firewall=1
numa: 0
ostype: win10
sata0: none,media=cdrom
scsihw: virtio-scsi-pci
smbios1: uuid=53d6aa22-4cba-477e-a67b-e6195ddf3b5e
sockets: 1
virtio0: local:101/vm-101-disk-0.qcow2,cache=writeback,size=32G
virtio1: raid:vm-101-disk-0,cache=writeback,size=32G
virtio2: ssd:vm-101-disk-0,cache=writeback,size=32G
virtio3: local:101/vm-101-disk-1.qcow2,cache=writeback,size=32G
vmgenid: cb95ac6d-cfbe-4e90-80a3-685648b9c297
Testing was performed using the utility CrystalDiskMark 6.0.2 x64
I got some unreal results. Can someone comment on them?
For example test on RAID-10:
1 Gb without Write Back Cache
-----------------------------------------------------------------------
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : https://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 490.872 MB/s
Sequential Write (Q= 32,T= 1) : 471.191 MB/s
Random Read 4KiB (Q= 8,T= 8) : 105.181 MB/s [ 25679.0 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 7.354 MB/s [ 1795.4 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 107.443 MB/s [ 26231.2 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 7.235 MB/s [ 1766.4 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 9.617 MB/s [ 2347.9 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 6.541 MB/s [ 1596.9 IOPS]
Test : 1024 MiB [E: 0.2% (0.1/32.0 GiB)] (x5) [Interval=5 sec]
Date : 2019/05/13 13:25:59
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 17763] (x64)
1 Gb with Write Back Cache
-----------------------------------------------------------------------
CrystalDiskMark 6.0.2 x64 (C) 2007-2018 hiyohiyo
Crystal Dew World : https://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
* KB = 1000 bytes, KiB = 1024 bytes
Sequential Read (Q= 32,T= 1) : 10153.791 MB/s
Sequential Write (Q= 32,T= 1) : 6770.621 MB/s
Random Read 4KiB (Q= 8,T= 8) : 707.773 MB/s [ 172796.1 IOPS]
Random Write 4KiB (Q= 8,T= 8) : 617.034 MB/s [ 150643.1 IOPS]
Random Read 4KiB (Q= 32,T= 1) : 555.786 MB/s [ 135689.9 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 393.689 MB/s [ 96115.5 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 18.836 MB/s [ 4598.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 17.976 MB/s [ 4388.7 IOPS]
Test : 1024 MiB [E: 0.2% (0.1/32.0 GiB)] (x5) [Interval=5 sec]
Date : 2019/05/13 13:38:07
OS : Windows Server 2016 Server Standard (full installation) [10.0 Build 17763] (x64)
The full test results document can be downloaded from Google Drive.
My colleague sent me the results from a similar server with the same Raid controller on which ESXi is installed and 4 SSDs are assembled in 10 Raid. The result is much lower. At the same time, it coincides with the result in the virtual machine even if the Windows Server is deployed on the host. How can this be explained?
Last edited:
