Slow IO performance with LVM and no RAID
- Thread starter did-vmonroig
- Start date
I've made a new system, with ext3 this time, still a single SATA disk, and what a change in the numbers:
This time it's using Proxmox version 3.1, but partitions are built the same way.
Code:
root@servidor03:~# pveperf /var/lib/vz
CPU BOGOMIPS: 54394.16
REGEX/SECOND: 1635928
HD SIZE: 1771.76 GB (/dev/mapper/vg1-lv1)
BUFFERED READS: 195.13 MB/sec
AVERAGE SEEK TIME: 16.32 ms
FSYNCS/SECOND: 1177.53
DNS EXT: 35.34 ms
DNS INT: 6.96 msThis time it's using Proxmox version 3.1, but partitions are built the same way.
I think pveperf is useless when measuring fsyncs on ext4 partitions. The numbers simply does not add up. Any other tool I have been using to measure disk performance all show ext4 is superior to ext3.
I have had this happen to me, don't remember the disk brand/model tho.
Everything worked ok on my desk.
At the datacenter IOPS was horrible, seems the vibration in the rack was more than that particular disk could handle.
Put in different brand disk, worked fine.
The reason to the big performance difference seen between ext3 in proxmox and ext4 is caused by the fact that default mount option regarding ext3 in the kernel used by proxmox is barrier=0 while the default mount option used by the kernel when dealing with ext4 is barrier=1. Test I have made shows than fsync/sec measured with pveperf shows an improvement by a factor 40 when barrier=0 is used as mount option so using default mount options comparing ext3 and ext4 in proxmox is totally misleading. Adding barrier=0 to the mount options with ext4 will show 3-5% better fsync/sec with ext4 compared to ext3.
Since auto_da_alloc is enabled by default using barrier=0 with ext4 gives exactly the same kind of security as using ext3 with barrier=0 then using barrier=1 on ext4 is not the choice for proxmox if performance with ext4 should not suffer unneeded.
Since auto_da_alloc is enabled by default using barrier=0 with ext4 gives exactly the same kind of security as using ext3 with barrier=0 then using barrier=1 on ext4 is not the choice for proxmox if performance with ext4 should not suffer unneeded.
Last edited:
I did not state pveperf is useless in general, what I was stating is that using pveperf on the current proxmox kernel to decide whether to use ext3 and ext4 is useless since the default mount options greatly favor performance for ext3.
Code:
"[COLOR=#000000]When comparing versus ext3,[/COLOR] note that ext4 enables write barriers by default, while ext3 does
not enable write barriers by default. So it is useful to use
explicitly specify whether barriers are enabled or not when via the
'-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems [COLOR=#000000] for a fair comparison"
[/COLOR][URL]https://www.kernel.org/doc/Documentation/filesystems/ext4.txt[/URL][COLOR=#000000]
[/COLOR]Using ext4 with barrier=0 is as safe as using ext3 with barrier=0 due to the new default mount option:
Code:
auto_da_alloc(*) Many broken applications don't use fsync() when
noauto_da_alloc replacing existing files via patterns such as
fd = open("foo.new")/write(fd,..)/close(fd)/
rename("foo.new", "foo"), or worse yet,
fd = open("foo", O_TRUNC)/write(fd,..)/close(fd).
If auto_da_alloc is enabled, ext4 will detect
the replace-via-rename and replace-via-truncate
patterns and force that any delayed allocation
blocks are allocated such that at the next
journal commit, in the default data=ordered
mode, the data blocks of the new file are forced
to disk before the rename() operation is
committed. This provides roughly the same level
of guarantees as ext3, and avoids the
"zero-length" problem that can happen when a
system crashes before the delayed allocation
blocks are forced to disk.It is the other way round. It is a bug by Redhat that ext3's default mount option is barrier=0 for ext3. This is strictly against recommendations from Ted Ts'o (http://lwn.net/Articles/283161/) since using barrier=0 for ext3 is as insecure as using it for ext4. So if you are concerned with security you should change ext3 to use barrier=1 as well and settle with the horrible performance. If you do not want to change default mount option for ext3 you should do the same for ext4. Sticking to the current situation with barrier=0 for ext3 and barrier=1 for ext4 and then claim ext3 is superior and thereby the recommend way is obtuse.