pve backup speeds on my server: should be better?

[m.ardito]

Hi, need to better understand backups.

i have a single backup setup for all my vms (7kvm, 1 openvz) and got logs like this every morning:

|------------------------VMS-------------------------------------------||---------backup setup-------------------||------backup log------------------------|
[FONT=courier new]VMID    type       os              disk n.    disk GB disk drv   storage   mode      compress storage status  mode        suspend    MiB/s
102     kvm        ubuntu server   2          8+8     IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    yes         5,10 
103     kvm        ubuntu server   2          8+8     IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    yes         5,80 
104     kvm        ubuntu desktop  1          8       IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    no          9,18 
106     kvm        win2003         2          5+15    VIRTIO     lvm/iscsi snapshot  yes      nfs     running snapshot    yes        10,58 
107     kvm        ubuntu desktop  2          12+12   IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    yes        10,64 
108     openvz     ubuntu          1          2       local      local     snapshot  yes      nfs     running snapshot    no         11,00 
109     kvm        win2000         1          32      IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    no         11,42 
111     kvm        ubuntu server   1          32      IDE        lvm/iscsi snapshot  yes      nfs     running snapshot    no          7,09[/FONT]

the server is an ibm x3650m2 with 2 hw mirrored local sata disks
disk storage is lvm/scsi over a 1gb eth connection to a qnap ts809u
backup storage is nfs over a 1gb eth connection to a qnap ts809u
the 1gb eth connection is on the same nic
qnap ts809u has 8 1,5TB sata2 disk in sw raid5

proxmox2:~# pveperf
CPU BOGOMIPS: 72537.76
REGEX/SECOND: 788290
HD SIZE: 16.49 GB (/dev/mapper/pve-root)
BUFFERED READS: 151.74 MB/sec
AVERAGE SEEK TIME: 4.21 ms
FSYNCS/SECOND: 1721.37
DNS EXT: 32.65 ms
DNS INT: 0.75 ms (domain omitted)

I know that suspend (0-1 seconds) is invoked when more than 1 disk per vm has to be copied

I can't understand
* why i get those speeds
- are they good, for my setup? i feel they're slow
- shoud them be better?
- how could i improve them? easy/cheap ways?

* why i get different speeds (between 5 and 12) is it about different compression rates? or else?

Thanks to anyone who cares...

Marco

 

[tom]

looks expected. the backup tool needs to scan the whole disk one and then it starts doing the backup. so finally the speed also depends on the total size of your disk on the used data on it. e.g. if you have a 8 GB disk in your which is almost full the calculated backup speed is higher.

the compression uses only one cpu core, so you will see a 100 % gzip task on the use core.

you can try without compression and also a backup in stop mode, compare results.

 

[SuSt]

How do you invoke the backup process?

AFAIK, by default vzdump is restricted by maximum speed in order to limit CPU IO load during backups.

 

[m.ardito]

thanks for your hint. i set up the backup from the pve gui, so i guess i'm using defaults...
i feel there's a way to make pve use other values at cpu expenses.. and i could try...
any hint about how to perform tests and find a beste value? how can i monitor cpu io load efficiently on pve servers?
i have 16 threads on 2 cpu here.... i guess there's some margin to improve the backup speed, so?

Marco

 

[SuSt]

Up to me, I prefer to invoke backup manually or by cron with the next parameters:

vzdump --snapshot --dumpdir /mnt/nfs/backup \
 --tmpdir /opt/backup \
 --stdexcludes --bwlimit 102400 \
 100

Please see the key "--bwlimit" in vzdump's manual.

The desirable B/W limit depends on current server usage and on its available hardware resources. For example, if at night your box stands idle, then you may safely define your bwlimit as maximum possible throughput of your network adapter or backup storage device.