[solved] Very slow backup for one container

liane

Renowned Member
Nov 25, 2008
40
1
73
Hi,

On a host with 5 containers, one of them (VM 104) has very poor backup speed, more than 10 times slower than others.

Here is the last backup log of VM 104 compared to 107, very similar in configuration.
Both have very little activity, same debian system (stretch). All containers are on a LVM partition, all backups are done on local disk.
Backups are done daily, and speed (or lack of it) is consistent every day.

What can I check to find the culprit?

104: 2019-08-30 01:54:23 INFO: Starting Backup of VM 104 (lxc) 104: 2019-08-30 01:54:23 INFO: status = running 104: 2019-08-30 01:54:23 INFO: CT Name: ********** 104: 2019-08-30 01:54:23 INFO: backup mode: snapshot 104: 2019-08-30 01:54:23 INFO: ionice priority: 5 104: 2019-08-30 01:54:23 INFO: create storage snapshot 'vzdump' 104: 2019-08-30 01:54:24 INFO: creating archive '/backup/dump/vzdump-lxc-104-2019_08_30-01_54_23.tar.gz' 104: 2019-08-30 02:38:21 INFO: Total bytes written: 2839234560 (2.7GiB, 1.1MiB/s) 104: 2019-08-30 02:38:21 INFO: archive file size: 835MB 104: 2019-08-30 02:38:21 INFO: delete old backup '/backup/dump/vzdump-lxc-104-2019_08_29-01_54_03.tar.gz' 104: 2019-08-30 02:38:22 INFO: remove vzdump snapshot 104: 2019-08-30 02:38:23 INFO: Finished Backup of VM 104 (00:44:00) 107: 2019-08-30 02:56:53 INFO: Starting Backup of VM 107 (lxc) 107: 2019-08-30 02:56:53 INFO: status = running 107: 2019-08-30 02:56:53 INFO: CT Name: ********** 107: 2019-08-30 02:56:53 INFO: backup mode: snapshot 107: 2019-08-30 02:56:53 INFO: ionice priority: 5 107: 2019-08-30 02:56:53 INFO: create storage snapshot 'vzdump' 107: 2019-08-30 02:56:54 INFO: creating archive '/backup/dump/vzdump-lxc-107-2019_08_30-02_56_53.tar.gz' 107: 2019-08-30 03:00:39 INFO: Total bytes written: 3413780480 (3.2GiB, 15MiB/s) 107: 2019-08-30 03:00:39 INFO: archive file size: 1.42GB 107: 2019-08-30 03:00:39 INFO: delete old backup '/backup/dump/vzdump-lxc-107-2019_08_29-02_58_14.tar.gz' 107: 2019-08-30 03:00:40 INFO: remove vzdump snapshot 107: 2019-08-30 03:00:40 INFO: Finished Backup of VM 107 (00:03:47) # pveperf CPU BOGOMIPS: 54272.32 REGEX/SECOND: 3307147 HD SIZE: 19.55 GB (/dev/md1) BUFFERED READS: 38.51 MB/sec AVERAGE SEEK TIME: 27.51 ms FSYNCS/SECOND: 11.48 DNS EXT: 20.21 ms DNS INT: 1.17 ms (online.net) # pveversion -v proxmox-ve: 5.4-2 (running kernel: 4.15.18-19-pve) pve-manager: 5.4-13 (running version: 5.4-13/aee6f0ec) pve-kernel-4.15: 5.4-7 pve-kernel-4.15.18-19-pve: 4.15.18-45 corosync: 2.4.4-pve1 criu: 2.11.1-1~bpo90 glusterfs-client: 3.8.8-1 ksm-control-daemon: 1.2-2 libjs-extjs: 6.0.1-2 libpve-access-control: 5.1-12 libpve-apiclient-perl: 2.0-5 libpve-common-perl: 5.0-54 libpve-guest-common-perl: 2.0-20 libpve-http-server-perl: 2.0-14 libpve-storage-perl: 5.0-44 libqb0: 1.0.3-1~bpo9 lvm2: 2.02.168-pve6 lxc-pve: 3.1.0-3 lxcfs: 3.0.3-pve1 novnc-pve: 1.0.0-3 proxmox-widget-toolkit: 1.0-28 pve-cluster: 5.0-37 pve-container: 2.0-40 pve-docs: 5.4-2 pve-edk2-firmware: 1.20190312-1 pve-firewall: 3.0-22 pve-firmware: 2.0-7 pve-ha-manager: 2.0-9 pve-i18n: 1.1-4 pve-libspice-server1: 0.14.1-2 pve-qemu-kvm: 3.0.1-4 pve-xtermjs: 3.12.0-1 qemu-server: 5.0-54 smartmontools: 6.5+svn4324-1 spiceterm: 3.0-5 vncterm: 1.5-3 zfsutils-linux: 0.7.13-pve1~bpo2 root@pm1:/backup/dump# # pct config 104 arch: i386 cpulimit: 4 cpuunits: 256 description: hostname: ********** memory: 1536 net0: name=eth0,bridge=vmbr0,firewall=1,gw=**.210.0.1,hwaddr=**:**:**:**:1b:17,ip=**.**.177.132/24,type=veth onboot: 1 ostype: debian rootfs: storage:vm-104-disk-1,size=8G swap: 512 # pct config 107 arch: i386 cpulimit: 4 cpuunits: 256 description: hostname: ********** memory: 1536 net0: name=eth0,bridge=vmbr0,firewall=1,gw=**.210.0.1,hwaddr=**:**:**:**:1b:16,ip=**.**.177.131/24,type=veth onboot: 1 ostype: debian rootfs: storage:vm-107-disk-1,size=8G swap: 512
 
What kind of system are 104 and 107? ( maybe they just have a quite different number of files, which would explain the speed difference)

I hope that helps!
 
Both are debian stretch, but you're right for the number of files, it happens that 104 has a *huge* number of PHP session files not properly cleaned up, that explained the extreme slowdown
 
*huge* number of PHP session files not properly cleaned up, that explained the extreme slowdown
probably the reason - please reply if removing them really fixed the issue - Thanks!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!