Backup Problem. Way to gracefully stop backup cron job.

vcp_ai

Renowned Member
Jul 28, 2010
177
5
83
Valencia -Spain-
Hi, I have a Server with 6 WinXP machines and 2 debian (trunkey core), all them KVM

I have three mounts for our NFS Server (Openfiler)
-1 x isos
-1 x backups2 ( keep two copies)
-1 x backups4 ( keep four copies)

I have an scheduled backup every two days for this 8 machines, running against backups4 mount.
Usually backup time for each machine (10 Gb raw disk) is 3-4 minutes.

Server Web is on a 100Mb network
NFS is on a 1 Gb independent network

But some times do not why, I find that backups take form 30 to 40 minutes each machine.

During this process I can find in SYSLOG:
Code:
Jun 19 11:03:00 servidor176 pvestatd[729170]: status update time (6.139 seconds)
Jun 19 11:03:06 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.isos' failed: got timeout
Jun 19 11:03:08 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups2' failed: got timeout
Jun 19 11:03:10 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups4' failed: got timeout
Jun 19 11:03:10 servidor176 pvestatd[729170]: status update time (6.145 seconds)
Jun 19 11:03:16 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.isos' failed: got timeout
Jun 19 11:03:18 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups2' failed: got timeout
Jun 19 11:03:20 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups4' failed: got timeout
Jun 19 11:03:20 servidor176 pvestatd[729170]: status update time (6.149 seconds)
Jun 19 11:03:30 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.isos' failed: got timeout
Jun 19 11:03:32 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups2' failed: got timeout
Jun 19 11:03:34 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.backups4' failed: got timeout
Jun 19 11:03:34 servidor176 pvestatd[729170]: status update time (9.573 seconds)
Jun 19 11:03:36 servidor176 pvestatd[729170]: WARNING: command 'df -P -B 1 /mnt/pve/Of.201.20.isos' failed: got timeout



It seems that vzdump program is constantly doing 'df -P -B 1 /mnt ...... ' to check for space on ALL mounts on the server (backup is executing against /mnt/pve/Of.201.20.backups4) .
, df gets a timeout, and it 'gets in some kind of loop' that makes backup last for a very long time .

On ssh df takes about 8 to 10 seconds during this process.


May be increasing df timeout OR reducing periodicity of it, OR doing df only against the destination of the backup, would help ....

Code:
root@servidor176:~# pveversion -v
pve-manager: 2.1-1 (pve-manager/2.1/f9b0f63a)
running kernel: 2.6.32-12-pve
proxmox-ve-2.6.32: 2.1-68
pve-kernel-2.6.32-12-pve: 2.6.32-68
lvm2: 2.02.95-1pve2
clvm: 2.02.95-1pve2
corosync-pve: 1.4.3-1
openais-pve: 1.1.4-2
libqb: 0.10.1-2
redhat-cluster-pve: 3.1.8-3
resource-agents-pve: 3.9.2-3
fence-agents-pve: 3.1.7-2
pve-cluster: 1.0-26
qemu-server: 2.0-39
pve-firmware: 1.0-16
libpve-common-perl: 1.0-27
libpve-access-control: 1.0-21
libpve-storage-perl: 2.0-18
vncterm: 1.0-2
vzctl: 3.0.30-2pve5
vzprocps: 2.0.11-2
vzquota: 3.0.12-3
pve-qemu-kvm: 1.0-9
ksm-control-daemon: 1.1-1
root@servidor176:~#

During this time Server GUI is not partially responsive (status, grafs, do not work)

So will like to have a way to gracefully stop the complete backup schedule (from ssh).

Regards
 
On ssh df takes about 8 to 10 seconds during this process.


May be increasing df timeout OR reducing periodicity of it, OR doing df only against the destination of the backup, would help ....

This just shows that there is too much load on your NFS server. I suggest you limit backup speed.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!