Backup timeout ... any way of configuring timeout ?

[sigma]

Hello!

I have an ftp account with space for my backups. I just mounted the FTP with curlftp into /mnt/ftp-backup

Problem is that when I try I live backup from proxmox web interfase, it gets cancelled with timeout.

root@proxmox:~# pveperf /mnt/ftp-backup


CPU BOGOMIPS: 54397.92
REGEX/SECOND: 2702433
HD SIZE: 7629.39 GB (curlftpfs#ftp://ftp-chicago.swiftway.net/)
FSYNCS/SECOND: 0.10
DNS EXT: 2064.02 ms
DNS INT: 2398.98 ms (notshown)

So now I am wondering if I may still pursue this line of backup schema, or rather do a local backup and THEN ftp out those files.

Any comments, considerations, ideas, etc. are very much welcome!

Enrique

 

[spirit]

FSYNCS/SECOND: 0.10

Forgot about it.
you need a backup storage not too slow.

If you have write in the vm and backup at the same time, the write need to go to the backup storage, then after to the vm disk.
If the storage is too slow and a big timeout, you'll have timeout in the vm.
That's why the backup is cancelled. (to avoid vm crash)

 

[sigma]

Hi Spirit,

Thanks for your answer!

Anyway I think there must be something more to it other than the apparent slow speed:

When I STOP the test virtual machine, I can do backup with no problem at all:

INFO: starting new backup job: vzdump 200 --remove 0 --mode stop --compress lzo --storage ftp --node proxmox
INFO: Starting Backup of VM 200 (qemu)
INFO: status = stopped
INFO: update VM 200: -lock backup
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: creating archive '/mnt/ftp-backup/dump/vzdump-qemu-200-2014_08_21-16_48_36.vma.lzo'
INFO: starting kvm to execute backup task
INFO: started backup task '2cf8a8b4-cbfa-4063-9317-c933144d990b'
INFO: status: 1% (368705536/34359738368), sparse 1% (368705536), duration 3, 122/0 MB/s
INFO: status: 11% (3894673408/34359738368), sparse 11% (3894673408), duration 6, 1175/0 MB/s
INFO: status: 42% (14702608384/34359738368), sparse 42% (14702608384), duration 9, 3602/0 MB/s
INFO: status: 74% (25481576448/34359738368), sparse 74% (25481576448), duration 12, 3592/0 MB/s
INFO: status: 100% (34359738368/34359738368), sparse 100% (34359738368), duration 15, 2959/0 MB/s
INFO: transferred 34359 MB in 15 seconds (2290 MB/s)
INFO: stopping kvm after backup task
INFO: archive file size: 2MB
INFO: Finished Backup of VM 200 (00:00:27)
INFO: Backup job finished successfully
TASK OK

Problem arises only when the virtual machine is running:


INFO: starting new backup job: vzdump 200 --remove 0 --mode snapshot --compress lzo --storage ftp --node proxmox
INFO: Starting Backup of VM 200 (qemu)
INFO: status = running
INFO: update VM 200: -lock backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: creating archive '/mnt/ftp-backup/dump/vzdump-qemu-200-2014_08_21-16_51_56.vma.lzo'
ERROR: got timeout
INFO: aborting backup job

Server is a XEON 1240 with an DC 3700 INTEL SSD (no production, only virtual machine, for testing purposes) Latest PROXMOX. FTP is on the same company, on high speed data access.

 

[spirit]

Yes,
like I said,

your backup storage need to be enough fast to handle new write inside the guest during the backup.
If the vm is shutdown, no write in guest, so it can be super slow. (ftp mount use fuse, so it's slow, better to use nfs if you can)

So, the question is : what kind of vm it is ? do you have a (lot) of writes inside ?

 

[sigma]

Spirit,

Thanks for your explanations. Frankly I am a bit curious about this behavior. It really feels like an inhability to properly multitask. I cannot understand why the live backup utility needs TO PAUSE, given that the local machine is running on an "ultra fast" SSD ... and the backup is FTP'd into another machine (with a certainly NOT slow connection). Different disks, different places, it seems to be a matter of just doing things right (read from live KVM, placing the read packet into a queue buffer, and output it into the backup medium.

I think that this extreme difference between speeds on a local SSD and a remote FTP, just helps to show a possible "bug" that evidently it will go against performance in any live backup scenario where the destination storage is slower than the local storage.... which usually is the case !!

Also probably I am not understanding some essential way of working inherent to a backup procedure, and all that I said over here is a fallacy

For now, I will resource into doing a LOCAL backup and then ftp'ing the backup files manually.