backup usb speed

R

raqdedicados

Renowned Member
Nov 26, 2008
85
0
71
Hi there

we re using vzdump in stop mode from the web interface to backup the VM's. the main server is a 2950 with 16gb ram 8 cores and 4 disk sas 15 running at raid 10. attached to this we have a usb disk drive mounted on /backup directory.

As we can see vzump is writting the dump files in about 1mb at a time on the destination directory, this is very slow. so one vm of 4gb demands a lot of time... output of the backup log at the bottom.

(With slow we mean the writing of the /backup/miercoles/ dir the rsync process was pretty fast)

Is there any chance to speed up the proceess. ? why this is so slow ?


Thanks

Diego

pd: we change our backup mode, we were using a samba share but, the samba hangs and suddenly this make all the HW to crash, a kernel panic...strange..so now we will use usb ...

Jul 22 02:39:01 INFO: Starting Backup of VM 101 (openvz)
Jul 22 02:39:01 INFO: status = CTID 101 exist mounted running
Jul 22 02:39:01 INFO: starting first sync /var/lib/vz/private/101 to /var/tmp/vzdumptmp28821
Jul 22 03:01:57 INFO: Number of files: 286638
Jul 22 03:01:57 INFO: Number of files transferred: 266833
Jul 22 03:01:57 INFO: Total file size: 10242706155 bytes
Jul 22 03:01:57 INFO: Total transferred file size: 10057541193 bytes
Jul 22 03:01:57 INFO: Literal data: 10057541266 bytes
Jul 22 03:01:57 INFO: Matched data: 0 bytes
Jul 22 03:01:57 INFO: File list size: 16053367
Jul 22 03:01:57 INFO: File list generation time: 0.001 seconds
Jul 22 03:01:57 INFO: File list transfer time: 0.000 seconds
Jul 22 03:01:57 INFO: Total bytes sent: 10086401551
Jul 22 03:01:57 INFO: Total bytes received: 5354453
Jul 22 03:01:57 INFO: sent 10086401551 bytes received 5354453 bytes 7331460.95 bytes/sec
Jul 22 03:01:57 INFO: total size is 10242706155 speedup is 1.01
Jul 22 03:01:57 INFO: first sync finished (1376 seconds)
Jul 22 03:01:57 INFO: stopping vps
Jul 22 03:01:57 INFO: Stopping container ...
Jul 22 03:02:28 INFO: Container was stopped
Jul 22 03:02:28 INFO: Container is unmounted
Jul 22 03:02:28 INFO: final sync /var/lib/vz/private/101 to /var/tmp/vzdumptmp28821
Jul 22 03:02:39 INFO: Number of files: 286611
Jul 22 03:02:39 INFO: Number of files transferred: 118
Jul 22 03:02:39 INFO: Total file size: 10242759062 bytes
Jul 22 03:02:39 INFO: Total transferred file size: 106103110 bytes
Jul 22 03:02:39 INFO: Literal data: 2938243 bytes
Jul 22 03:02:39 INFO: Matched data: 103164867 bytes
Jul 22 03:02:39 INFO: File list size: 16053311
Jul 22 03:02:39 INFO: File list generation time: 0.001 seconds
Jul 22 03:02:39 INFO: File list transfer time: 0.000 seconds
Jul 22 03:02:39 INFO: Total bytes sent: 19198343
Jul 22 03:02:39 INFO: Total bytes received: 302430
Jul 22 03:02:39 INFO: sent 19198343 bytes received 302430 bytes 1695719.39 bytes/sec
Jul 22 03:02:39 INFO: total size is 10242759062 speedup is 525.25
Jul 22 03:02:39 INFO: final sync finished (11 seconds)
Jul 22 03:02:39 INFO: restarting vps
Jul 22 03:02:39 INFO: Starting container ...
Jul 22 03:02:39 INFO: Container is mounted
Jul 22 03:02:39 INFO: Adding IP address(es): 200.49.152.67
Jul 22 03:02:41 INFO: Setting CPU units: 100000
Jul 22 03:02:41 INFO: Setting CPUs: 1
Jul 22 03:02:41 INFO: Configure meminfo: 1048576
Jul 22 03:02:41 INFO: Set hostname: raq67.raqdedicados.com
Jul 22 03:02:41 INFO: File resolv.conf was modified
Jul 22 03:02:41 INFO: Setting quota ugidlimit: 276447231
Jul 22 03:02:41 INFO: Container start in progress...
Jul 22 03:02:41 INFO: vps is online again after 44 seconds
Jul 22 03:02:41 INFO: creating archive '/backup/miercoles/vzdump-101.dat' (/var/tmp/vzdumptmp28821/101)
Jul 22 03:23:04 INFO: Total bytes written: 10449305600 (9.8GiB, 8.2MiB/s)
Jul 22 03:23:04 INFO: file size 4.87GB
Jul 22 03:23:36 INFO: Finished Backup of VM 101 (00:44:35)
 
  1. what raid controller do you use?
  2. write cache enabled (with BBU)?
  3. Results of 'pveperf' (only run when the server is not under heavy load)
  4. write speed of a normal file to the external USB drive?
 
Well, if you see, it takes double the time it should since it first writes to a temp directory (rsync, then stop the container, then rsync again) (time from 02:39:01 to 03:02:39, total 23 minutes) and at the end compresses (that takes CPU time) and stores in USB (03:02:39 to 03:23:36, total 21 minutes).
See my post about "stop" behaviour, seems a "superstop" could be useful to you too, if, like me, are more concerned about backup time than downtime:
http://www.proxmox.com/forum/showthread.php?t=1668
also:
Total bytes written: 10449305600 (9.8GiB, 8.2MiB/s)
sounds good to me, it transferred 10GB to /var/tmp folder (beware, is in /, so pay attention not to fill it with bigger VM) with 2 rsync, then it compresses (reading 10GB) to a 4GB. I usually have, with less powerful hardware but still with external USB:
...
101: Jul 19 02:57:37 INFO: creating archive '/backup/vm/vzdump-101.dat' (/backup/tempdump/vzdumptmp21607/101)
101: Jul 19 03:51:18 INFO: Total bytes written: 34333675520 (32GiB, 11MiB/s)
101: Jul 19 03:51:18 INFO: file size 20.91GB
101: Jul 19 03:55:39 INFO: Finished Backup of VM 101 (01:55:37)
...
An option could be exclude directory of VPS containing only data (i.e. /serv/smbshare) and backup it with an independent cronjob. Unfortunately exclude dir is not among the options web interface provides (you could enter in a global config, but will work for all vps).
Maybe I'm missing something, I'm not very expert of Proxmox, that anyway appreciate a lot.
 
Tom

We use

perc 6/i with 4 sas drivers 15k running at raid 10

write cache is enabled on the controller

CPU BOGOMIPS: 39904.25
REGEX/SECOND: 642112
HD SIZE: 94.49 GB (/dev/pve/root)
BUFFERED READS: 275.52 MB/sec
AVERAGE SEEK TIME: 15.01 ms
FSYNCS/SECOND: 2123.75

USB: iso file of 700mb would take about 5/6 seconds
thanks!
 
Oh, I should mention that vzdump has a bandwidth limit. The default is "--bwlimit 10240" - max 10MB/s.

Try to set bwlimit in global vzdump config file /etc/vzdump.conf (man vzdump) to a higher value.

- Dietmar
 
to raqdedicados:
" iso file of 700mb would take about 5/6 seconds". Let's suppose it takes 7 seconds -> 100MB/sec -> 800Mbit/s WOW, double the max speed of USB-2 specification (that is 480Mbit/s)!
Sure is not real transfer, maybe if you unmount the device you have to wait until all the data is REALLY wrote to the HD.
 
Hello,

I have the same problem, I think.
Did changing -bwlimit option make it better ?

--
Cordially
Christophe Charron
 
dietmar said:
Again, vzdump limits istelfs to 10MB/s. Use -bwlimit option to adjust that.
Click to expand...


Where is the file exactly used for? Yesterday I generated the file and entered bwlimit: 20400 because
the backup actually takes>12 hours and I would basicly like to limit backup to the night.

Today it took as long as yesterday. Is /etc/vzdump.conf used as a template? This means
I have to recreate the cronjobs? Or do I have to add the parameter manually into
the cronjobs?

Thanks in advance

Jörn Plewka
 
You must log in or register to reply here.
Top Bottom