LXC Containers Backing Up Incredibly Slow

infinityM

Member
Dec 7, 2019
165
1
18
27
Hey Guys,

We have a LXC container which is about 2TB large, When backing up this server though it takes about 7+ hours to backup.
While I have several VM's which only take about 20 minutes to backup but is 5-6TB large...

Why are LXC's so slow to backup?
 

infinityM

Member
Dec 7, 2019
165
1
18
27
to help better, please provide your storage config cat /etc/pve/storage.cfg and your container config
Hey Matrix,

root@c6:~# cat /etc/pve/storage.cfg
dir: local
path /var/lib/vz
content vztmpl,iso,backup

lvmthin: local-lvm
thinpool data
vgname pve
content images,rootdir

rbd: Default
content images,rootdir
krbd 0
pool Default

nfs: NAS
export /volume1/Backups
path /mnt/pve/NAS
server 10.161.0.247
content snippets,vztmpl,backup,iso,rootdir,images
maxfiles 3

pbs: backups
datastore backups
server pmb.local
content backup
fingerprint # I removed the fingerprint
maxfiles 99
username root@pam

LXC Config

root@c6:~# cat /etc/pve/lxc/127.conf
arch: amd64
cores: 6
hostname: VPS127
memory: 8192
net0: name=eth0,bridge=vmbr0,firewall=1,gw=156.38.175.34,hwaddr=22:72:12:00:8F:59,ip=156.38.175.58/27,type=veth
onboot: 1
ostype: ubuntu
rootfs: Default:vm-127-disk-0,size=2000G
swap: 8192
unprivileged: 1
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,042
822
163
because for containers, PBS needs to read and chunk all the data to find out what to upload, for (running) VMs there is a shortcut because we can tell Qemu to keep track of which chunks changed.
 

infinityM

Member
Dec 7, 2019
165
1
18
27
because for containers, PBS needs to read and chunk all the data to find out what to upload, for (running) VMs there is a shortcut because we can tell Qemu to keep track of which chunks changed.
Hey Fabian,

Is there any plans to get around this problem?
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,042
822
163
there is no cheap solution for this - the short term would definitely be to move such workloads to VMs where there is this intermediate layer that can keep track of writes from the guest and speedup the backup.
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,042
822
163
yes, basically. then you only need to re-read all of the data when
- the VM was down (live-migrating to another node will now transfer the bitmap as well!)
- the encryption key or mode was changed
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,042
822
163
of course, increasing your read/chunk performance might also be an option depending on your current hardware ;)
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
5,042
822
163
read: I/O benchmark (more/faster disks, to some degree more RAM)
chunk/hash: proxmox-backup-client benchmark (faster CPU/more cores)
 

carsten2

Member
Mar 25, 2017
182
7
23
52
I opened a feature request https://bugzilla.proxmox.com/show_bug.cgi?id=3138 to speed up container backup, but it gets closed multiple times without discussion and even with wrong claims. The feature request is perfectly valid, because PBS would be the ONLY backup solution in the world, which needs several hours to backup UNCHANGED file systems. I claim that a dramatic improvement IS possible and it is even not too complicated to implement.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!