Lvm group performances

telemaco

New Member
Jul 16, 2010
7
0
1
Hi, i ve set up a proxmox 1.5 test server. I've used the proxmox iso for my installation, and i've saved the VM (kvm) on /dev/pve/data formated with an ext3 filesysten and mounted at /var/lib/vz.
When i create a local backup with vzdump and then restore with qmrestore, the performances are very good.
I've install a second disk, create lvm group without filesystem and add as storage.
When i create a local backup with vzdump and then restore with qmrestore, the performances are very bad.
Why are so slow? The lvm raw solution should be more fast than the filesystem, or no?

Thanks, Luca.
 
are you sure that it is slow? post all details (logs) and the size of the LVM and we will see how fast it is.
 
Hi, i ve set up a proxmox 1.5 test server. I've used the proxmox iso for my installation, and i've saved the VM (kvm) on /dev/pve/data formated with an ext3 filesysten and mounted at /var/lib/vz.
When i create a local backup with vzdump and then restore with qmrestore, the performances are very good.
I've install a second disk, create lvm group without filesystem and add as storage.
When i create a local backup with vzdump and then restore with qmrestore, the performances are very bad.
Why are so slow? The lvm raw solution should be more fast than the filesystem, or no?

Thanks, Luca.
Hi,
i think it's depends from the type and connection of the lvm-disk (raid, SAN/iSCSI...).
I have no performance issue with lvm-disk. Made a short test:
Code:
proxmox1:~# lvcreate -n test -L 10G es6600_2_sas_lun17_vg
proxmox1:~# mkfs.ext3 /dev/es6600_2_sas_lun17_vg/test
proxmox1:~# mount /dev/es6600_2_sas_lun17_vg/test /mnt
proxmox1:~# pveperf /mnt
CPU BOGOMIPS:      27293.38
REGEX/SECOND:      1092299
HD SIZE:           9.84 GB (/dev/mapper/es6600_2_sas_lun17_vg-test)
BUFFERED READS:    323.13 MB/sec
AVERAGE SEEK TIME: 3.25 ms
FSYNCS/SECOND:     2332.47
DNS EXT:           61.34 ms
DNS INT:           0.58 ms
How looks your pveperf-values on lvm-storage and /var/lib/vz?

Udo
 
are you sure that it is slow? post all details (logs) and the size of the LVM and we will see how fast it is.


These are the bad performance:
The VM size is 20GB.


backup report:

Jul 16 12:19:02 INFO: Starting Backup of VM 101 (qemu)
Jul 16 12:19:02 INFO: running
Jul 16 12:19:02 INFO: status = running
Jul 16 12:19:03 INFO: backup mode: snapshot
Jul 16 12:19:03 INFO: bandwidth limit: 10240 KB/s
Jul 16 12:19:03 INFO: Logical volume "vzsnap-v1-0" created
Jul 16 12:19:03 INFO: creating archive '/home/backup/vzdump-qemu-101-2010_07_16-12_19_02.tar'
Jul 16 12:19:03 INFO: adding '/home/backup/vzdump-qemu-101-2010_07_16-12_19_02.tmp/qemu-server.conf' to archive ('qemu-server.conf')
Jul 16 12:19:03 INFO: adding '/dev/virtual/vzsnap-v1-0' to archive ('vm-disk-ide0.raw')
Jul 16 12:24:59 INFO: Total bytes written: 2734538240 (7.32 MiB/s)
Jul 16 12:24:59 INFO: archive file size: 2.55GB
Jul 16 12:24:59 INFO: delete old backup '/home/backup/vzdump-qemu-101-2010_07_16-10_24_02.tar'
Jul 16 12:25:00 INFO: Logical volume "vzsnap-v1-0" successfully removed
Jul 16 12:25:00 INFO: Finished Backup of VM 101 (00:05:58)



restore report:

v1:/home/backup# qmrestore vzdump-qemu-101-2010_07_16-12_19_02.tar 120
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_07_16-12_19_02.tar' using ID 120
INFO: extracting 'qemu-server.conf' from archive
INFO: extracting 'vm-disk-ide0.raw' from archive
INFO: Rounding up size to full physical extent 20.00 GB
INFO: Logical volume "vm-120-disk-1" created
INFO: new volume ID is 'macchine:vm-120-disk-1'
INFO: restore data to '/dev/virtual/vm-120-disk-1' (21474836480 bytes)
INFO: 41943040+0 records in
INFO: 41943040+0 records out
INFO: 21474836480 bytes (21 GB) copied, 1272.36 s, 16.9 MB/s
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_07_16-12_19_02.tar' successful



The test with ext3 filesystem and the same size of VM, are:
-2 minutes and 20 seconds for backup
-26 seconds for restore!!!
 
787 MB/s - wow. Are you sure that is true? That sounds like cache to me. What does pveperf /var/lib/vz give you?

v1:/home/backup# pveperf /var/lib/vz
CPU BOGOMIPS: 42559.80
REGEX/SECOND: 890740
HD SIZE: 59.06 GB (/dev/mapper/virtual-dati)
BUFFERED READS: 76.31 MB/sec
AVERAGE SEEK TIME: 9.64 ms
FSYNCS/SECOND: 529.62
DNS EXT: 244.72 ms
DNS INT: 109.35 ms
 
Those specs aren't particularly impressive. Unfortunately they are exactly the same as mine :) No way would they give you 787 MB/s...
 
The test with ext3 filesystem and the same size of VM, are:
-2 minutes and 20 seconds for backup
-26 seconds for restore!!!

well, if i read well:
backup report:
...
Jul 16 12:25:00 INFO: Finished Backup of VM 101 (00:05:58)

and

restore report:
...
INFO: 21474836480 bytes (21 GB) copied, 1272.36 s, 16.9 MB/s


means: 6 minutes backup, 21 minutes restore

then your pve performance is not exceptional as other said, mine for reference:

Code:
proxmox1:~# pveperf /var/lib/vz
CPU BOGOMIPS:      72537.40
REGEX/SECOND:      805731
HD SIZE:           36.84 GB (/dev/mapper/pve-data)
BUFFERED READS:    137.89 MB/sec
AVERAGE SEEK TIME: 4.27 ms
FSYNCS/SECOND:     2212.65
DNS EXT:           19.82 ms
DNS INT:           0.63 ms
and as Dietmar said, yo have to think that during restore, the whole hard disk has to be restored, and this takes time, maybe also due to the network speed between pve and lvm storage.

to increase restore times ,one could use faster dedicated network connections between pve and lvm storage (10G nics, or bonding 1G nics probably helps)

but, i'm not an expert.

Marco
 
Last edited:
Those specs aren't particularly impressive. Unfortunately they are exactly the same as mine :) No way would they give you 787 MB/s...


I've installed a new VM (20GB) on a ext3 partition and i've created the backup:

Jul 19 14:07:01 INFO: Starting Backup of VM 101 (qemu)
Jul 19 14:07:02 INFO: running
Jul 19 14:07:02 INFO: status = running
Jul 19 14:07:02 INFO: backup mode: snapshot
Jul 19 14:07:02 INFO: bandwidth limit: 10240 KB/s
Jul 19 14:07:03 INFO: Logical volume "vzsnap-v1-0" created
Jul 19 14:07:03 INFO: creating archive '/home/backup/vzdump-qemu-101-2010_07_19-14_07_01.tar'
Jul 19 14:07:03 INFO: adding '/home/backup/vzdump-qemu-101-2010_07_19-14_07_01.tmp/qemu-server.conf' to archive ('qemu-server.conf')
Jul 19 14:07:03 INFO: adding '/mnt/vzsnap0/images/101/vm-101-disk-1.raw' to archive ('vm-disk-virtio0.raw')
Jul 19 14:09:17 INFO: Total bytes written: 706043392 (5.02 MiB/s)
Jul 19 14:09:17 INFO: archive file size: 673MB
Jul 19 14:09:17 INFO: delete old backup '/home/backup/vzdump-qemu-101-2010_07_16-12_19_02.tar'
Jul 19 14:09:23 INFO: Logical volume "vzsnap-v1-0" successfully removed
Jul 19 14:09:23 INFO: Finished Backup of VM 101 (00:02:22)


Then i've deleted the VM, rebooted the server (host), and i've restored from backup:

v1:/home/backup# qmrestore vzdump-qemu-101-2010_07_19-14_07_01.tar 110
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_07_19-14_07_01.tar' using ID 110
INFO: extracting 'qemu-server.conf' from archive
INFO: extracting 'vm-disk-virtio0.raw' from archive
INFO: Formatting '/mnt/dati/images/110/vm-110-disk-1.raw', fmt=raw size=32768
INFO: new volume ID is 'macchine:110/vm-110-disk-1.raw'
INFO: restore data to '/mnt/dati/images/110/vm-110-disk-1.raw' (21474836480 bytes)
INFO: 21474836480 bytes copied, 42 s, 487.62 MiB/s <------
INFO: restore QemuServer backup 'vzdump-qemu-101-2010_07_19-14_07_01.tar' successful

Is possible?
 
qmrestore computes the restore "speed" using
- the full disk value (21474836480 bytes copied) instead of
- the restored file size (Total bytes written: 706043392)

so it reports

21474836480 / (42 * 1024 * 1024) gives 487.62 MiB/s

while really you did

706043392 / (42 * 1024 * 1024) that gives ~ 16MiB/s

i think,
Marco
 
qmrestore computes the restore "speed" using
- the full disk value (21474836480 bytes copied) instead of
- the restored file size (Total bytes written: 706043392)

so it reports

21474836480 / (42 * 1024 * 1024) gives 487.62 MiB/s

while really you did

706043392 / (42 * 1024 * 1024) that gives ~ 16MiB/s

i think,
Marco


ok, i think you have right! :D Thank you.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!