[SOLVED] ext3 pveperf bad fsync/sec?

[Stroker89]

Hi,

i have a server with 2 x 2TB SW-Raid 1.
Im using Proxmox since version 3.0.
With version 3.x the fsync/sec with pveperf was around 800/sec

Since i installed version 4 the fsync/sec dropped dramatically
The load of the server is very low...

pveperf:

CPU BOGOMIPS:      54395.92
REGEX/SECOND:      2935422
HD SIZE:           19.56 GB (/dev/dm-0)
BUFFERED READS:    156.46 MB/sec
AVERAGE SEEK TIME: 7.10 ms
FSYNCS/SECOND:     31.93
DNS EXT:           92.45 ms
DNS INT:           14.25 ms

Version:

proxmox-ve: 4.1-34 (running kernel: 4.2.6-1-pve)
pve-manager: 4.1-13 (running version: 4.1-13/cfb599fb)
pve-kernel-4.2.6-1-pve: 4.2.6-36
lvm2: 2.02.116-pve2
corosync-pve: 2.3.5-2
libqb0: 1.0-1
pve-cluster: 4.0-32
qemu-server: 4.0-55
pve-firmware: 1.1-7
libpve-common-perl: 4.0-48
libpve-access-control: 4.0-11
libpve-storage-perl: 4.0-40
pve-libspice-server1: 0.12.5-2
vncterm: 1.2-1
pve-qemu-kvm: 2.5-5
pve-container: 1.0-44
pve-firewall: 2.0-17
pve-ha-manager: 1.0-21
ksm-control-daemon: 1.2-1
glusterfs-client: 3.5.2-2+deb8u1
lxc-pve: 1.1.5-7
lxcfs: 0.13-pve3
cgmanager: 0.39-pve1
criu: 1.6.0-1
fence-agents-pve: not correctly installed
openvswitch-switch: 2.3.2-2

Mounts /etc/fstab:

proc /proc proc defaults 0 0
/dev/md/0 none swap sw 0 0
/dev/md/1 /boot ext3 defaults 0 0
# /dev/md/2 belongs to LVM volume group 'system'
# /dev/md/3 belongs to LVM volume group 'images'
# /dev/md/4 belongs to LVM volume group 'storage'
# /dev/md/5 belongs to LVM volume group 'backup'
/dev/system/root  /  ext3  defaults 0 0
/dev/system/tmp  /tmp  ext3  defaults 0 0
/dev/system/var  /var  ext3  defaults 0 0
/dev/images/vmimages     /vmimages      ext3    defaults        0       0
/dev/backup/vmbackups   /vmbackup       ext3    defaults        0       0

The Raid isnt rebuilding or anything like that...
Any idea why the fsync/sec are so low?

Thanks.

 

[MasterTH]

sw raid isn't really for running vms on it. Performance always was not that good.

Maybe in pve3 the filesystem was mounted with no sync option?

 

[dietmar]

Old system used barrier=0 as default.

 

[Stroker89]

Thanks dietmar.
Could i use barrier=0 in the new version two?
Is there anything to observe with barrier=0?

 

[Stroker89]

Ok my /etc/fstab looks now like this

proc /proc proc defaults 0 0
/dev/md/0 none swap sw 0 0
/dev/md/1 /boot ext3 defaults,barrier=0 0 1
# /dev/md/2 belongs to LVM volume group 'system'
# /dev/md/3 belongs to LVM volume group 'images'
# /dev/md/4 belongs to LVM volume group 'storage'
# /dev/md/5 belongs to LVM volume group 'backup'
/dev/system/root  /  ext3  defaults,barrier=0 0 2
/dev/system/tmp  /tmp  ext3  defaults,barrier=0 0 0
/dev/system/var  /var  ext3  defaults,barrier=0 0 2
/dev/images/vmimages     /vmimages      ext3    defaults,barrier=0      0       0
/dev/backup/vmbackups   /vmbackup       ext3    defaults,barrier=0      0       0

and the fsync/sec performance is now:

CPU BOGOMIPS:      54403.60
REGEX/SECOND:      3020933
HD SIZE:           19.56 GB (/dev/dm-0)
BUFFERED READS:    141.97 MB/sec
AVERAGE SEEK TIME: 7.30 ms
FSYNCS/SECOND:     1213.35
DNS EXT:           80.58 ms
DNS INT:           11.97 ms

Thanks for the tip.