Windows Server Setup VIRTIO or SCSI ?

L

LKPMX

New Member
Dec 6, 2016
18
0
1
40
What is the best setup for Windows Server 2012 R2/2016 using NVMe SSD ?

Is VIRTIO or SCSI the way to go ?

I installed both to test and I cannot tell a difference,

So I just want to ask is there any difference ?

It also looks like the random (16.0) Read speeds are really slow for both ?

Am I doing something wrong ?

Thanks.
 
Same problem!! Slow write speeds and slow random read speeds in windows using samsung nvme ssd.
 
If you compare VIRTIO with SCSI on VIRTIO Controller and no, there is no big difference in speed if at all. For Linux, you'll get different I/O paths for the disks.

Proxmox VE is currently optimized to give you the "proven" setup for each VM automatically if you set the OS setting correct.

One thing that can make a difference is the I/O scheduler in the OS. If you would install the OS on the "real" disk, it would see the SSD architecture, but for a virtualized disk, this is normally not propagated, so the OS thinks, it is a "normal" (spinning) disk and therefore will use different I/O scheduler. I'm not sure if this is still true, but you can easily (at least in Linux) check if you've optimized everything (e.g. https://wiki.debian.org/SSDOptimization#Low-Latency_IO-Scheduler)
 
LnxBil said:
If you compare VIRTIO with SCSI on VIRTIO Controller and no, there is no big difference in speed if at all. For Linux, you'll get different I/O paths for the disks.

Proxmox VE is currently optimized to give you the "proven" setup for each VM automatically if you set the OS setting correct.

One thing that can make a difference is the I/O scheduler in the OS. If you would install the OS on the "real" disk, it would see the SSD architecture, but for a virtualized disk, this is normally not propagated, so the OS thinks, it is a "normal" (spinning) disk and therefore will use different I/O scheduler. I'm not sure if this is still true, but you can easily (at least in Linux) check if you've optimized everything (e.g. https://wiki.debian.org/SSDOptimization#Low-Latency_IO-Scheduler)
Click to expand...

So do you think that the random (16) read speeds I tested in Linux and Windows are false speeds or actual speeds ?
 
LnxBil said:
What is random (16)? Please use fio and report back.
Click to expand...

I ran winsat disk in dos to benchmark. So I'm guessing the 16 is 16 bytes ?
I attached the screenshot from my physical box benchmark and the VM (no cache).

I installed fio on windows but I guess I need some file to load into it?
I never used fio before have to find out who to use it.
 

Attachments

  • winsat.JPG
    winsat.JPG
    59.5 KB · Views: 35
  • VMwinsat.JPG
    VMwinsat.JPG
    70.3 KB · Views: 32
Last edited:
This is the ultimate SSD-Benchmark 4K Blocksize, queue Depth 32. Most SSD vendors use this to present read and write performance (if you change rw=randwrite):

Code: 
cat > /tmp/4KQD32.fio <<EOF
[global]
# old fio versions need this
#filename=/path/to/file
time_based
runtime=30

[4kqd32_read]
description=4K QD32
blocksize=4K
rw=randread
direct=1
ioengine=libaio
iodepth=32
EOF

You can give the filename on the command line if fio is very recent or your a block device (whole disk). Be aware on the write-test and whole disk, it overwrites data.

Here is an example:

fio-disk.png
 
LnxBil said:
This is the ultimate SSD-Benchmark 4K Blocksize, queue Depth 32. Most SSD vendors use this to present read and write performance (if you change rw=randwrite):

Code: 
cat > /tmp/4KQD32.fio <<EOF
[global]
# old fio versions need this
#filename=/path/to/file
time_based
runtime=30

[4kqd32_read]
description=4K QD32
blocksize=4K
rw=randread
direct=1
ioengine=libaio
iodepth=32
EOF

You can give the filename on the command line if fio is very recent or your a block device (whole disk). Be aware on the write-test and whole disk, it overwrites data.

Here is an example:

View attachment 4486
Click to expand...

Hi, thanks for your help.

I created a file named fio.fio in notepad and copied and pasted your code into the file and saved it.
then in DOS I ran fio c:\fio.fio and it gives me the following error. fio: unable to open 'c:\fio.fio' job file.
I also tried fio filename=c:\fio.fio still the same error.
What am I doing wrong ?
 
The filename is for the test file (the one that is created to perform the I/O benchmark). Just set it inside the config file (the part is commented out) and set filename=c:\fio.test (maybe more quoting) and run it with fio 4KQD32.fio

 
LnxBil said:
The filename is for the test file (the one that is created to perform the I/O benchmark). Just set it inside the config file (the part is commented out) and set filename=c:\fio.test (maybe more quoting) and run it with fio 4KQD32.fio
Click to expand...

This is the error I get:
Code: 
> fio runtest.fio
fio: engine libaio not loadable
fio: failed to load engine libao
Bad option <EOF>
fio job 4kqd32_read dropped
fio: file:ioengines.c:91, func=dlopen, error=(null)

My file looks like this:
Code: 
[global]
#old fio versions need this
#filename=c:\test.fio
time_based
runtime=30

[4kqd32_read]
description=4k QD32
blocksize=4k
rw=randread
direct=1
ioengine=libaio
iodepth=32
EOF
 
]
maxprox said:
hi,
at first try
go to the line:
Code: 
ioengine=libaio
and comment out
Code: 
## ioengine=libaio
And vice versa -like I understand it- with the following:
Code: 
#filename=c:\test.fio
to
Code: 
filename=c:\test.fio
Click to expand...

Hi, thanks for the reply.
This is what I get now:

Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:833, func=total_file_size, error=Invalid argument


Run status group 0 (all jobs):

C:\>
 
spirit said:
for windows,

-ioengine=windowsaio

and also

--filename=C\:\filename
Click to expand...

Hello,
Now I am getting this.
Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:833, func=total_file_size, error=Invalid argument


Run status group 0 (all jobs):
 
LKPMX said:
Hello,
Now I am getting this.
Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:833, func=total_file_size, error=Invalid argument


Run status group 0 (all jobs):
Click to expand...

you need to specify a "size=", if you bench through a file.
(ou can also bench a raw disk, E:/ for example, but it'll delete the filesystem)
 
LKPMX said:
Hello,
Now I am getting this.
Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: you need to specify size=
fio: pid=0, err=22/file:filesetup.c:833, func=total_file_size, error=Invalid argument


Run status group 0 (all jobs):
Click to expand...

GOT IT, Thanks :)
 
LnxBil said:
What is random (16)? Please use fio and report back.
Click to expand...

RandRead Size=1g
Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [r(1)] [93.3% done] [304.6MB/0KB/0KB /s] [77.1K/0/0 iops] [eta 00m:02s]
4kqd32_read: (groupid=0, jobs=1): err= 0: pid=2576: Mon Dec 12 14:34:02 2016
  Description  : [4K QD32]
  read : io=8838.6MB, bw=301677KB/s, iops=75419, runt= 30001msec
  slat (usec): min=5, max=739, avg= 9.92, stdev= 2.66
  clat (usec): min=22, max=9421, avg=286.12, stdev=303.65
  lat (usec): min=91, max=9430, avg=296.04, stdev=303.78
  clat percentiles (usec):
  |  1.00th=[  121],  5.00th=[  141], 10.00th=[  159], 20.00th=[  187],
  | 30.00th=[  215], 40.00th=[  241], 50.00th=[  266], 60.00th=[  294],
  | 70.00th=[  318], 80.00th=[  342], 90.00th=[  370], 95.00th=[  390],
  | 99.00th=[  462], 99.50th=[  828], 99.90th=[ 4896], 99.95th=[ 5024],
  | 99.99th=[ 5280]
  lat (usec) : 50=0.01%, 100=0.03%, 250=43.44%, 500=55.66%, 750=0.34%
  lat (usec) : 1000=0.06%
  lat (msec) : 2=0.03%, 4=0.04%, 10=0.41%
  cpu  : usr=6.67%, sys=63.33%, ctx=0, majf=0, minf=0
  IO depths  : 1=0.1%, 2=0.1%, 4=0.1%, 8=17.1%, 16=77.5%, 32=5.3%, >=64=0.0%
  submit  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=95.4%, 8=0.1%, 16=0.2%, 32=4.4%, 64=0.0%, >=64=0.0%
  issued  : total=r=2262655/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
  latency  : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  READ: io=8838.6MB, aggrb=301677KB/s, minb=301677KB/s, maxb=301677KB/s, mint=30001msec, maxt=30001msec

C:\>


RandWrite Size=1g
Code: 
C:\>fio runtest.fio
fio: this platform does not support process shared mutexes, forcing use of threads. Use the 'thread' option to get rid of this warning.
4kqd32_read: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=windowsaio, iodepth=32
fio-2.15
Starting 1 thread
4kqd32_read: Laying out IO file(s) (1 file(s) / 1024MB)
Jobs: 1 (f=1): [w(1)] [100.0% done] [0KB/284.8MB/0KB /s] [0/72.9K/0 iops] [eta 00m:00s]
4kqd32_read: (groupid=0, jobs=1): err= 0: pid=3060: Mon Dec 12 14:38:56 2016
  Description  : [4K QD32]
  write: io=8447.8MB, bw=288340KB/s, iops=72085, runt= 30001msec
  slat (usec): min=8, max=471744, avg=11.15, stdev=328.49
  clat (usec): min=1, max=657908, avg=245.84, stdev=541.70
  lat (usec): min=48, max=697589, avg=256.99, stdev=719.95
  clat percentiles (usec):
  |  1.00th=[  63],  5.00th=[  79], 10.00th=[  98], 20.00th=[  133],
  | 30.00th=[  169], 40.00th=[  205], 50.00th=[  241], 60.00th=[  274],
  | 70.00th=[  310], 80.00th=[  350], 90.00th=[  386], 95.00th=[  406],
  | 99.00th=[  466], 99.50th=[  540], 99.90th=[  716], 99.95th=[ 1192],
  | 99.99th=[ 7968]
  lat (usec) : 2=0.01%, 50=0.06%, 100=10.33%, 250=42.35%, 500=46.59%
  lat (usec) : 750=0.60%, 1000=0.02%
  lat (msec) : 2=0.01%, 4=0.02%, 10=0.02%, 20=0.01%, 50=0.01%
  lat (msec) : 100=0.01%, 250=0.01%, 750=0.01%
  cpu  : usr=3.33%, sys=16.67%, ctx=0, majf=0, minf=0
  IO depths  : 1=0.1%, 2=0.1%, 4=7.0%, 8=29.7%, 16=59.6%, 32=3.7%, >=64=0.0%
  submit  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
  complete  : 0=0.0%, 4=96.4%, 8=0.1%, 16=0.1%, 32=3.6%, 64=0.0%, >=64=0.0%
  issued  : total=r=0/w=2162625/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
  latency  : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
  WRITE: io=8447.8MB, aggrb=288340KB/s, minb=288340KB/s, maxb=288340KB/s, mint=30001msec, maxt=30001msec

And the test file looks like this
Code: 
[global]
# old fio versions need this
filename=c\:\test.fio
time_based
runtime=30
size=1g
[4kqd32_read]
description=4K QD32
blocksize=4K
rw=randread # <--  I CHANGED THIS TO randwrite for the 2nd test.
direct=1
ioengine=windowsaio
iodepth=32
 
Last edited:
77k is OK (compared to other disks and SSDs) but I have on Proxmox VE on the raw device 331k for randread (4K, QD32) on my Samsung Pro 950 (write is currently not possible due to "data"). I can try to provide benchmark results from Windows, but that'll take time.
 
LnxBil said:
77k is OK (compared to other disks and SSDs) but I have on Proxmox VE on the raw device 331k for randread (4K, QD32) on my Samsung Pro 950 (write is currently not possible due to "data"). I can try to provide benchmark results from Windows, but that'll take time.
Click to expand...

Can you please. I really want to use Proxmox but I need it to run Windows for the MS SQL DB.

I will eagerly wait for your reply :)

THANKS A LOT FOR EVERYTHING. YOUR AMAZING! :)
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back