Hard Drive performance difference Windows/Linux Partition Size ?

[crocobc]

Hi,

I'm having on same machine with the same configuration two VM.

One with Windows 2008 R2 with only one partition.
Size: 1600Go

One with Ubuntu.
Size: 32Go


hdparm returns me an average of:

- Timing buffered disk reads: 352 MB in 3.02 seconds = 116.70 MB/sec
on the physical machine.

- Timing buffered disk reads: 346 MB in 2.99 seconds = 114.30 MB/sec
on the linux machine.

20M/s on the windows machine.....

below the configuration of the two VM


Windows
name: Win2008
ide2: cdrom,media=cdrom
bootdisk: virtio0
ostype: w2k8
memory: 10000
onboot: 1
sockets: 1
cores: 4
boot: c
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
virtio0: local:101/vm-101-disk-1.raw
vlan0: e1000=


Linux
name: Ubuntu
ide2: cdrom,media=cdrom
bootdisk: virtio0
virtio0: local:104/vm-104-disk-1.raw
ostype: l26
memory: 2000
onboot: 1
sockets: 1
cores: 4
boot: dc
freeze: 0
cpuunits: 1000
acpi: 1
kvm: 1
vlan0:


Linux version 2.6.32-4-pve (unknown) (root@oahu) (gcc version 4.3.2 (Debian 4.3.2-1.1)


Does someone know why this huge difference ?
Partition size ?

I've looked through the forum for 2 days but nothing really helps me...

Thanks a lot.
Sorry for my english...

 

[udo]

Hi,
meassure hdparm the real speed? The different between host and vm is very small.
Check host with "pveperf /var/lib/vz" for read and fsyncs and so on.

And also "dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conv=fdatasync" for writing.
Reading again with "echo 3 > /proc/sys/vm/drop_caches; dd if=/var/lib/vz/bigfile of=/dev/null bs=1024k" (for host and VM)

How do you messure the win-performance?
Do you use the latest virtio-driver for windows?

Please report also the output of "pveversion -v"

Udo

 

[crocobc]

proxmox:/vz# pveversion -v
pve-manager: 1.7-11 (pve-manager/1.7/5470)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.32: 1.7-30
pve-kernel-2.6.32-4-pve: 2.6.32-30
pve-kernel-2.6.35-1-pve: 2.6.35-10
qemu-server: 1.1-28
pve-firmware: 1.0-10
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-10
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.13.0-3
ksm-control-daemon: 1.0-4



proxmox:~# pveperf /var/lib/vz
CPU BOGOMIPS: 29791.88
REGEX/SECOND: 682239
HD SIZE: 1673.33 GB (/dev/mapper/pve-vz)
BUFFERED READS: 104.62 MB/sec
AVERAGE SEEK TIME: 12.77 ms
DNS EXT: 13.64 ms
DNS INT: 3.09 ms

dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conf=fdatasync
3210739712 bytes (3,2 GB) copied, 11,287 s, 284 MB/s



linux machine
dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conf=fdatasync
1375731712 octets (1,4 GB) copiés, 32,6711 s, 42,1 MB/s


Yes i use the latest win drivers...
I use HDtune for win-performance.


Thanks...

 

[udo]

proxmox:/vz# pveversion -v
pve-manager: 1.7-11 (pve-manager/1.7/5470)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.32: 1.7-30
pve-kernel-2.6.32-4-pve: 2.6.32-30
pve-kernel-2.6.35-1-pve: 2.6.35-10
qemu-server: 1.1-28
pve-firmware: 1.0-10
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-10
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.13.0-3
ksm-control-daemon: 1.0-4



proxmox:~# pveperf /var/lib/vz
CPU BOGOMIPS: 29791.88
REGEX/SECOND: 682239
HD SIZE: 1673.33 GB (/dev/mapper/pve-vz)
BUFFERED READS: 104.62 MB/sec
AVERAGE SEEK TIME: 12.77 ms
DNS EXT: 13.64 ms
DNS INT: 3.09 ms

dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conf=fdatasync
3210739712 bytes (3,2 GB) copied, 11,287 s, 284 MB/s

Hi,
do you want kidding me?

This can be the output of the command! Because a typo in conv, and 1M*8192 is 8GiB and not 3,2GB...
And 284MB/s write speed is much to high, because your pveperf shows, that you read only much less and the seektime looks like single sata-disk. Also you don't provide the fsyncs...

I someone should help or gives hint please bring the whole and right output!
The same is the kernel - in your first post you you write an 2.6.32. Now you have an 2.6.35???

The output must similiar to this

dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conv=fdatasync
8192+0 Datensätze ein
8192+0 Datensätze aus
8589934592 Bytes (8,6 GB) kopiert, 47,9111 s, 179 MB/s

This output fits to

pveperf /var/lib/vz
...
BUFFERED READS:    317.79 MB/sec
AVERAGE SEEK TIME: 7.24 ms
FSYNCS/SECOND:     4640.75

linux machine
dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conf=fdatasync
1375731712 octets (1,4 GB) copiés, 32,6711 s, 42,1 MB/s

42 MB/s looks normal - but the other values are wrong!

Yes i use the latest win drivers...
I use HDtune for win-performance.


Thanks...

Perhaps someone else want to spend his/her time to answer such postings. I not.

Udo

 

[crocobc]

Sorry but i'm not very good in linux command and in your first post YOU made a mistake in the command you ask me to do...:

"dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conF=fdatasync"

So i just modify the command and stop it after a moment...sorry for my ignorance...this is not my job...

And for the kernel i have just upgrade it this morning because a friend tell me to try...no change


don't be so rude please...


Here is the complete output with the right command:

Host
proxmox:~# dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conv=fdatasync
8192+0 enregistrements lus
8192+0 enregistrements écrits
8589934592 bytes (8,6 GB) copied, 114,89 s, 74,8 MB/s


VM
sudo dd if=/dev/zero of=/var/lib/bigfile bs=1024k count=8192 conv=fdatasync
8192+0 enregistrements lus
8192+0 enregistrements écrits
8589934592 octets (8,6 GB) copiés, 193,544 s, 44,4 MB/s

 

[tom]

update your host to the current version 1.8 with KVM 0.14 (post your pveversion -v) and test your hdtune results again.
also post the new results of pveperf and pveversion -v.

do we talk here about read or write performance?

 

[udo]

Sorry but i'm not very good in linux command and in your first post YOU made a mistake in the command you ask me to do...:

"dd if=/dev/zero of=/var/lib/vz/bigfile bs=1024k count=8192 conF=fdatasync"

So i just modify the command and stop it after a moment...sorry for my ignorance...this is not my job...

Hi,
sorry - yes the typo was my fault, but with the different datavalues i assume you play a joke with me...
Perhaps you can also test on windows with dd - see here: http://www.chrysocome.net/dd (i haven't tried this) to measure the same things under win.

The relation of speed between VM and host looks normal for me.

Udo

 

[crocobc]

Hi,

Thanks for replying back to me.
It's write speed.


pveversion -v
pve-manager: 1.8-15 (pve-manager/1.8/5754)
running kernel: 2.6.35-1-pve
pve-kernel-2.6.32-4-pve: 2.6.32-30
pve-kernel-2.6.35-1-pve: 2.6.35-10
qemu-server: 1.1-30
pve-firmware: 1.0-10
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-10
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1


pveperf
CPU BOGOMIPS: 29790.74
REGEX/SECOND: 810594
HD SIZE: 19.22 GB (/dev/sda1)
BUFFERED READS: 103.11 MB/sec
AVERAGE SEEK TIME: 8.80 ms
FSYNCS/SECOND: 25.16
DNS EXT: 13.22 ms
DNS INT: 3.59 ms


HDTune: 8,4Mb/s !!!! Hope undo the changes will give back to me my poor 20Mo/s

I have a 1Gbs SDSL on this server and i'm downloading at 8Mo/s max ichhh.

update your host to the current version 1.8 with KVM 0.14

I'm not sure i did this step right...KVM 0.14 how can i be sure ? (sorry)

Perhaps you can also test on windows with dd - see here: http://www.chrysocome.net/dd (i haven't tried this) to measure the same things under win

I cannot have an output with the write speed under windows. (but the command line is working)

 

[udo]

Hi,

Thanks for replying back to me.
It's write speed.


pveversion -v
pve-manager: 1.8-15 (pve-manager/1.8/5754)
running kernel: 2.6.35-1-pve
pve-kernel-2.6.32-4-pve: 2.6.32-30
pve-kernel-2.6.35-1-pve: 2.6.35-10
qemu-server: 1.1-30
pve-firmware: 1.0-10
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-10
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1


pveperf
CPU BOGOMIPS: 29790.74
REGEX/SECOND: 810594
HD SIZE: 19.22 GB (/dev/sda1)
BUFFERED READS: 103.11 MB/sec
AVERAGE SEEK TIME: 8.80 ms
FSYNCS/SECOND: 25.16
DNS EXT: 13.22 ms
DNS INT: 3.59 ms


HDTune: 8,4Mb/s !!!! Hope undo the changes will give back to me my poor 20Mo/s

I have a 1Gbs SDSL on this server and i'm downloading at 8Mo/s max ichhh.



I'm not sure i did this step right...KVM 0.14 how can i be sure ? (sorry)

Hi,
the intresting line in "pveversion -v" is missing!
pve-qemu-kvm: 0.14.0-3
this mean, kvm 0.14 is installed.

I cannot have an output with the write speed under windows. (but the command line is working)

then you must stop the time. Caclulate 8192/(sec which dd need) and you have MB/s.
You can test further things:
1. change from virtio to ide (you can do it by gui - remove harddisk, and then add a new - there you can select the unused image again, or modify the VM.conf (/etc/qemu-server) change virtio0 to ide0 (if ide0 not allways used).
2. change the cache-modi in the VM.conf - if now the results not so good (with 1.8) you should append a ",cache=writethrough" on the diskentry. Since pve1.8 chache=none is the default.

You HDD looks very slow! fsyncs of 25 is nothing. Do you have an faster HDD? Are there other precesses which use (and slow down) the HDD? You should do your meassuring on a calm system.

Udo

 

[crocobc]

the intresting line in "pveversion -v" is missing!
pve-qemu-kvm: 0.14.0-3
this mean, kvm 0.14 is installed.

Ok but why didn't i have this line ? Does this means it's not installed ? But VM working ?

Ok i will try the rest right now.
The server is a dedicated server i cannot change the HDD.
It's two 2To raid 0.

And no there are no other process running...

 

[dietmar]

Ok but why didn't i have this line ? Does this means it's not installed ?

yes. Please install the correct package.

But VM working ?

Maybe your installed the qemu-kvm package from debian during upgrade?

 

[udo]

Ok but why didn't i have this line ? Does this means it's not installed ? But VM working ?

perhaps "apt-get install proxmox-ve-2.6.35"?

Ok i will try the rest right now.
The server is a dedicated server i cannot change the HDD.
It's two 2To raid 0.

And no there are no other process running...

raid-0 also striping for better performance?! I can't believe that. Is this a software- or hardwareraid?
Please show a "lspci -v"

Dedicated sever: does the provider install (or support) proxmox? Is this a standard installation, or ontop of debian 5?

Udo

 

[crocobc]

Pffff sorry again it's RAID1 not RAID0 (hardware)

Yes provider install proxmox but no support.

perhaps "apt-get install proxmox-ve-2.6.35"?

Thanks it did the trick


pveversion -v
pve-manager: 1.8-15 (pve-manager/1.8/5754)
running kernel: 2.6.35-1-pve
proxmox-ve-2.6.35: 1.8-10
pve-kernel-2.6.32-4-pve: 2.6.32-30
pve-kernel-2.6.35-1-pve: 2.6.35-10
qemu-server: 1.1-30
pve-firmware: 1.0-10
libpve-storage-perl: 1.0-16
vncterm: 0.9-2
vzctl: 3.0.24-1pve4
vzdump: 1.2-10
vzprocps: 2.0.11-1dso2
vzquota: 3.0.11-1
pve-qemu-kvm: 0.14.0-3
ksm-control-daemon: 1.0-4


pveperf
CPU BOGOMIPS: 29790.98
REGEX/SECOND: 671057
HD SIZE: 19.22 GB (/dev/sda1)
BUFFERED READS: 115.65 MB/sec
AVERAGE SEEK TIME: 9.21 ms
FSYNCS/SECOND: 36.63
DNS EXT: 10.29 ms
DNS INT: 3.39 ms

you should append a ",cache=writethrough" on the diskentry

This helps a lot i'm back with my 20mb/s on windows...thanks




lspci -v

00:00.0 Host bridge: Intel Corporation Clarksfield/Lynnfield DMI (rev 11)
Subsystem: Dell Device 02a5
Flags: fast devsel
Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit- Queue=0/1 Enable-
Capabilities: [90] Express Root Port (Slot-), MSI 00
Capabilities: [e0] Power Management version 3
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [150] Access Controls <?>
Capabilities: [160] Vendor Specific Information <?>

00:03.0 PCI bridge: Intel Corporation Clarksfield/Lynnfield PCI Express Root Port 1 (rev 11) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
I/O behind bridge: 0000f000-0000ffff
Memory behind bridge: df100000-df2fffff
Capabilities: [40] Subsystem: Dell Device 02a5
Capabilities: [60] Message Signalled Interrupts: Mask+ 64bit- Queue=0/1 Enable+
Capabilities: [90] Express Root Port (Slot+), MSI 00
Capabilities: [e0] Power Management version 3
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [150] Access Controls <?>
Capabilities: [160] Vendor Specific Information <?>
Kernel driver in use: pcieport
Kernel modules: shpchp

00:08.0 System peripheral: Intel Corporation Clarksfield/Lynnfield System Management Registers (rev 11)
Flags: fast devsel
Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [100] Vendor Specific Information <?>

00:08.1 System peripheral: Intel Corporation Clarksfield/Lynnfield Semaphore and Scratchpad Registers (rev 11)
Flags: fast devsel
Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [100] Vendor Specific Information <?>

00:08.2 System peripheral: Intel Corporation Clarksfield/Lynnfield System Control and Status Registers (rev 11)
Flags: fast devsel
Capabilities: [40] Express Root Complex Integrated Endpoint, MSI 00
Capabilities: [100] Vendor Specific Information <?>

00:08.3 System peripheral: Intel Corporation Clarksfield/Lynnfield Miscellaneous Registers (rev 11)
Flags: fast devsel

00:10.0 System peripheral: Intel Corporation Device d150 (rev 11)
Flags: fast devsel

00:10.1 System peripheral: Intel Corporation Device d151 (rev 11)
Flags: fast devsel

00:1a.0 USB Controller: Intel Corporation Ibex Peak USB2 Enhanced Host Controller (rev 05) (prog-if 20 [EHCI])
Subsystem: Dell Device 02a5
Flags: bus master, medium devsel, latency 0, IRQ 22
Memory at df0fa000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCIe advanced features <?>
Kernel driver in use: ehci_hcd

00:1c.0 PCI bridge: Intel Corporation Ibex Peak PCI Express Root Port 1 (rev 05) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
Memory behind bridge: da000000-ddffffff
Capabilities: [40] Express Root Port (Slot-), MSI 00
Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
Capabilities: [90] Subsystem: Dell Device 02a5
Capabilities: [a0] Power Management version 2
Kernel driver in use: pcieport
Kernel modules: shpchp

00:1d.0 USB Controller: Intel Corporation Ibex Peak USB2 Enhanced Host Controller (rev 05) (prog-if 20 [EHCI])
Subsystem: Dell Device 02a5
Flags: bus master, medium devsel, latency 0, IRQ 22
Memory at df0fc000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Capabilities: [58] Debug port: BAR=1 offset=00a0
Capabilities: [98] PCIe advanced features <?>
Kernel driver in use: ehci_hcd

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a5) (prog-if 01 [Subtractive decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=03, subordinate=03, sec-latency=32
Memory behind bridge: de000000-deffffff
Prefetchable memory behind bridge: 00000000d9800000-00000000d9ffffff
Capabilities: [50] Subsystem: Dell Device 02a5

00:1f.0 ISA bridge: Intel Corporation Ibex Peak LPC Interface Controller (rev 05)
Subsystem: Dell Device 02a5
Flags: bus master, medium devsel, latency 0
Capabilities: [e0] Vendor Specific Information <?>
Kernel modules: iTCO_wdt

00:1f.2 SATA controller: Intel Corporation Ibex Peak 6 port SATA AHCI Controller (rev 05) (prog-if 01 [AHCI 1.0])
Subsystem: Dell Device 02a5
Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 48
I/O ports at ecd0
I/O ports at ecc8
I/O ports at ecd8
I/O ports at eccc
I/O ports at ece0
Memory at df0fe000 (32-bit, non-prefetchable) [size=2K]
Capabilities: [80] Message Signalled Interrupts: Mask- 64bit- Queue=0/0 Enable+
Capabilities: [70] Power Management version 3
Capabilities: [a8] SATA HBA <?>
Capabilities: [b0] PCIe advanced features <?>
Kernel driver in use: ahci
Kernel modules: ahci

01:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic Device 0072 (rev 02)
Subsystem: Dell Device 1f1d
Flags: bus master, fast devsel, latency 0, IRQ 16
I/O ports at fc00
Memory at df2b0000 (64-bit, non-prefetchable) [size=64K]
Memory at df2c0000 (64-bit, non-prefetchable) [size=256K]
Expansion ROM at df100000 [disabled] [size=1M]
Capabilities: [50] Power Management version 3
Capabilities: [68] Express Endpoint, MSI 00
Capabilities: [d0] Vital Product Data <?>
Capabilities: [a8] Message Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable-
Capabilities: [c0] MSI-X: Enable+ Mask- TabSize=15
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [138] Power Budgeting <?>
Kernel driver in use: mpt2sas
Kernel modules: mpt2sas

02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
Subsystem: Dell Device 02a5
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at da000000 (64-bit, non-prefetchable) [size=32M]
Capabilities: [48] Power Management version 3
Capabilities: [50] Vital Product Data <?>
Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+ Queue=0/4 Enable-
Capabilities: [a0] MSI-X: Enable+ Mask- TabSize=9
Capabilities: [ac] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 21-11-86-fe-ff-b9-26-00
Capabilities: [110] Advanced Error Reporting <?>
Capabilities: [150] Power Budgeting <?>
Capabilities: [160] Virtual Channel <?>
Kernel driver in use: bnx2
Kernel modules: bnx2

02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5716 Gigabit Ethernet (rev 20)
Subsystem: Dell Device 02a5
Flags: bus master, fast devsel, latency 0, IRQ 17
Memory at dc000000 (64-bit, non-prefetchable) [size=32M]
Capabilities: [48] Power Management version 3
Capabilities: [50] Vital Product Data <?>
Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+ Queue=0/4 Enable-
Capabilities: [a0] MSI-X: Enable- Mask- TabSize=9
Capabilities: [ac] Express Endpoint, MSI 00
Capabilities: [100] Device Serial Number 22-11-86-fe-ff-b9-26-00
Capabilities: [110] Advanced Error Reporting <?>
Capabilities: [150] Power Budgeting <?>
Capabilities: [160] Virtual Channel <?>
Kernel driver in use: bnx2
Kernel modules: bnx2

 

[udo]

Pffff sorry again it's RAID1 not RAID0 (hardware)

Yes provider install proxmox but no support.

Thanks it did the trick
...

pveperf
CPU BOGOMIPS: 29790.98
REGEX/SECOND: 671057
HD SIZE: 19.22 GB (/dev/sda1)
BUFFERED READS: 115.65 MB/sec
AVERAGE SEEK TIME: 9.21 ms
FSYNCS/SECOND: 36.63
DNS EXT: 10.29 ms
DNS INT: 3.39 ms


This helps a lot i'm back with my 20mb/s on windows...thanks
...

Hi,
fine, that you reach the 20MB/s again.
With lspci I don't see any raid-controller. I assume you have an softraid! But your root (pveperf use sda1) is not on a md-device... curios. With a normal installation sda1 is /boot and not root.

So i think the best way is, to ask your provider why you have such poor fsyncs with pveperf.

Udo

 

[crocobc]

ok i will and return the information

Thanks