Really slow backup for a LXC VM

Autrui

Active Member
Jan 19, 2019
10
0
41
30
Hi,

I've got an issue with a LXC VM, with a really slow backup speed/time. It's the nextcloud VM, vmid 150. I've got no idea why this one is slow, but not the others.

Here are my logs :

Code:
Details
VMID    Name    Status    Time    Size    Filename
104    omv    ok    16m 24s    500 GiB    vm/104/2025-11-18T00:30:01Z
105    nextcloud    ok    3h 48m 27s    465.55 GiB    ct/105/2025-11-18T00:46:25Z
106    database    ok    36s    5.108 GiB    ct/106/2025-11-18T04:34:52Z

Total running time: 4h 5m 27s
Total size: 970.659 GiB
Logs
vzdump 105 106 104 --fleecing 0 --quiet 1 --storage pbs --notification-mode legacy-sendmail --mailto XXX --mode snapshot --node proxmox --notes-template '{{guestname}}' --mailnotification always


104: 2025-11-18 01:30:01 INFO: Starting Backup of VM 104 (qemu)
104: 2025-11-18 01:30:01 INFO: status = running
104: 2025-11-18 01:30:01 INFO: VM Name: omv
104: 2025-11-18 01:30:01 INFO: include disk 'scsi0' 'local-zfs:vm-104-disk-0' 50G
104: 2025-11-18 01:30:01 INFO: exclude disk 'scsi1' 'zfs-data:vm-104-disk-0' (backup=no)
104: 2025-11-18 01:30:01 INFO: include disk 'scsi2' 'zfs-data:vm-104-disk-1' 250G
104: 2025-11-18 01:30:01 INFO: include disk 'scsi3' 'zfs-data:vm-104-disk-2' 200G
104: 2025-11-18 01:30:01 INFO: backup mode: snapshot
104: 2025-11-18 01:30:01 INFO: ionice priority: 7
104: 2025-11-18 01:30:01 INFO: snapshots found (not included into backup)
104: 2025-11-18 01:30:01 INFO: creating Proxmox Backup Server archive 'vm/104/2025-11-18T00:30:01Z'
104: 2025-11-18 01:30:01 INFO: issuing guest-agent 'fs-freeze' command
104: 2025-11-18 01:30:11 INFO: issuing guest-agent 'fs-thaw' command
104: 2025-11-18 01:30:17 INFO: started backup task '032791ab-14e1-4a90-aceb-e05159df58ef'
104: 2025-11-18 01:30:17 INFO: resuming VM again
104: 2025-11-18 01:30:17 INFO: scsi0: dirty-bitmap status: OK (13.3 GiB of 50.0 GiB dirty)
104: 2025-11-18 01:30:17 INFO: scsi2: dirty-bitmap status: OK (4.0 MiB of 250.0 GiB dirty)
104: 2025-11-18 01:30:17 INFO: scsi3: dirty-bitmap status: OK (2.5 GiB of 200.0 GiB dirty)
104: 2025-11-18 01:30:17 INFO: using fast incremental mode (dirty-bitmap), 15.9 GiB dirty of 500.0 GiB total
104: 2025-11-18 01:30:20 INFO:   3% (620.0 MiB of 15.9 GiB) in 3s, read: 206.7 MiB/s, write: 188.0 MiB/s
104: 2025-11-18 01:30:23 INFO:   5% (952.0 MiB of 15.9 GiB) in 6s, read: 110.7 MiB/s, write: 61.3 MiB/s
104: 2025-11-18 01:30:26 INFO:   6% (1.0 GiB of 15.9 GiB) in 9s, read: 25.3 MiB/s, write: 24.0 MiB/s
104: 2025-11-18 01:30:29 INFO:   8% (1.3 GiB of 15.9 GiB) in 12s, read: 104.0 MiB/s, write: 80.0 MiB/s
104: 2025-11-18 01:30:32 INFO:   9% (1.5 GiB of 15.9 GiB) in 15s, read: 52.0 MiB/s, write: 18.7 MiB/s
104: 2025-11-18 01:30:35 INFO:  10% (1.6 GiB of 15.9 GiB) in 18s, read: 56.0 MiB/s, write: 45.3 MiB/s
104: 2025-11-18 01:30:38 INFO:  11% (1.8 GiB of 15.9 GiB) in 21s, read: 57.3 MiB/s, write: 56.0 MiB/s
104: 2025-11-18 01:30:41 INFO:  12% (2.0 GiB of 15.9 GiB) in 24s, read: 57.3 MiB/s, write: 54.7 MiB/s
104: 2025-11-18 01:30:44 INFO:  13% (2.1 GiB of 15.9 GiB) in 27s, read: 41.3 MiB/s, write: 41.3 MiB/s
104: 2025-11-18 01:30:47 INFO:  14% (2.3 GiB of 15.9 GiB) in 30s, read: 73.3 MiB/s, write: 69.3 MiB/s
104: 2025-11-18 01:30:50 INFO:  15% (2.5 GiB of 15.9 GiB) in 33s, read: 66.7 MiB/s, write: 64.0 MiB/s
104: 2025-11-18 01:30:53 INFO:  16% (2.6 GiB of 15.9 GiB) in 36s, read: 50.7 MiB/s, write: 44.0 MiB/s
104: 2025-11-18 01:30:56 INFO:  19% (3.1 GiB of 15.9 GiB) in 39s, read: 169.3 MiB/s, write: 157.3 MiB/s
104: 2025-11-18 01:30:59 INFO:  24% (3.9 GiB of 15.9 GiB) in 42s, read: 269.3 MiB/s, write: 264.0 MiB/s
104: 2025-11-18 01:31:02 INFO:  28% (4.6 GiB of 15.9 GiB) in 45s, read: 226.7 MiB/s, write: 226.7 MiB/s
104: 2025-11-18 01:31:05 INFO:  33% (5.3 GiB of 15.9 GiB) in 48s, read: 232.0 MiB/s, write: 229.3 MiB/s
104: 2025-11-18 01:31:08 INFO:  35% (5.6 GiB of 15.9 GiB) in 51s, read: 117.3 MiB/s, write: 117.3 MiB/s
104: 2025-11-18 01:31:11 INFO:  38% (6.1 GiB of 15.9 GiB) in 54s, read: 160.0 MiB/s, write: 160.0 MiB/s
104: 2025-11-18 01:31:14 INFO:  40% (6.4 GiB of 15.9 GiB) in 57s, read: 113.3 MiB/s, write: 113.3 MiB/s
104: 2025-11-18 01:31:17 INFO:  44% (7.0 GiB of 15.9 GiB) in 1m, read: 193.3 MiB/s, write: 193.3 MiB/s
104: 2025-11-18 01:31:20 INFO:  47% (7.5 GiB of 15.9 GiB) in 1m 3s, read: 177.3 MiB/s, write: 177.3 MiB/s
104: 2025-11-18 01:31:23 INFO:  50% (8.1 GiB of 15.9 GiB) in 1m 6s, read: 193.3 MiB/s, write: 193.3 MiB/s
104: 2025-11-18 01:31:26 INFO:  53% (8.5 GiB of 15.9 GiB) in 1m 9s, read: 164.0 MiB/s, write: 164.0 MiB/s
104: 2025-11-18 01:31:29 INFO:  55% (8.8 GiB of 15.9 GiB) in 1m 12s, read: 102.7 MiB/s, write: 101.3 MiB/s
104: 2025-11-18 01:31:53 INFO:  56% (8.9 GiB of 15.9 GiB) in 1m 36s, read: 1.8 MiB/s, write: 1.8 MiB/s
104: 2025-11-18 01:32:22 INFO:  57% (9.0 GiB of 15.9 GiB) in 2m 5s, read: 5.4 MiB/s, write: 5.4 MiB/s
104: 2025-11-18 01:33:00 INFO:  58% (9.2 GiB of 15.9 GiB) in 2m 43s, read: 4.3 MiB/s, write: 4.3 MiB/s
104: 2025-11-18 01:33:21 INFO:  59% (9.4 GiB of 15.9 GiB) in 3m 4s, read: 7.6 MiB/s, write: 7.6 MiB/s
104: 2025-11-18 01:33:49 INFO:  60% (9.6 GiB of 15.9 GiB) in 3m 32s, read: 8.1 MiB/s, write: 8.1 MiB/s
104: 2025-11-18 01:34:23 INFO:  61% (9.7 GiB of 15.9 GiB) in 4m 6s, read: 3.2 MiB/s, write: 3.2 MiB/s
104: 2025-11-18 01:34:44 INFO:  62% (9.9 GiB of 15.9 GiB) in 4m 27s, read: 10.9 MiB/s, write: 10.9 MiB/s
104: 2025-11-18 01:35:31 INFO:  63% (10.0 GiB of 15.9 GiB) in 5m 14s, read: 1.8 MiB/s, write: 1.8 MiB/s
104: 2025-11-18 01:35:55 INFO:  64% (10.2 GiB of 15.9 GiB) in 5m 38s, read: 6.8 MiB/s, write: 6.8 MiB/s
104: 2025-11-18 01:36:06 INFO:  65% (10.3 GiB of 15.9 GiB) in 5m 49s, read: 14.5 MiB/s, write: 14.5 MiB/s
104: 2025-11-18 01:36:52 INFO:  66% (10.5 GiB of 15.9 GiB) in 6m 35s, read: 3.6 MiB/s, write: 3.5 MiB/s
104: 2025-11-18 01:36:55 INFO:  67% (10.7 GiB of 15.9 GiB) in 6m 38s, read: 80.0 MiB/s, write: 80.0 MiB/s
104: 2025-11-18 01:36:58 INFO:  68% (10.8 GiB of 15.9 GiB) in 6m 41s, read: 28.0 MiB/s, write: 28.0 MiB/s
104: 2025-11-18 01:37:58 INFO:  69% (11.1 GiB of 15.9 GiB) in 7m 41s, read: 5.3 MiB/s, write: 5.3 MiB/s
104: 2025-11-18 01:38:01 INFO:  70% (11.1 GiB of 15.9 GiB) in 7m 44s, read: 4.0 MiB/s, write: 4.0 MiB/s
104: 2025-11-18 01:39:00 INFO:  71% (11.3 GiB of 15.9 GiB) in 8m 43s, read: 3.4 MiB/s, write: 3.3 MiB/s
104: 2025-11-18 01:39:04 INFO:  72% (11.5 GiB of 15.9 GiB) in 8m 47s, read: 39.0 MiB/s, write: 39.0 MiB/s
104: 2025-11-18 01:39:32 INFO:  73% (11.6 GiB of 15.9 GiB) in 9m 15s, read: 4.7 MiB/s, write: 4.7 MiB/s
104: 2025-11-18 01:39:41 INFO:  74% (11.8 GiB of 15.9 GiB) in 9m 24s, read: 26.7 MiB/s, write: 26.7 MiB/s
104: 2025-11-18 01:39:44 INFO:  75% (12.0 GiB of 15.9 GiB) in 9m 27s, read: 70.7 MiB/s, write: 70.7 MiB/s
104: 2025-11-18 01:40:01 INFO:  76% (12.1 GiB of 15.9 GiB) in 9m 44s, read: 6.6 MiB/s, write: 6.6 MiB/s
104: 2025-11-18 01:40:04 INFO:  77% (12.2 GiB of 15.9 GiB) in 9m 47s, read: 26.7 MiB/s, write: 26.7 MiB/s
104: 2025-11-18 01:40:07 INFO:  78% (12.4 GiB of 15.9 GiB) in 9m 50s, read: 68.0 MiB/s, write: 66.7 MiB/s
104: 2025-11-18 01:40:18 INFO:  79% (12.6 GiB of 15.9 GiB) in 10m 1s, read: 16.7 MiB/s, write: 16.7 MiB/s
104: 2025-11-18 01:40:21 INFO:  80% (12.8 GiB of 15.9 GiB) in 10m 4s, read: 84.0 MiB/s, write: 84.0 MiB/s
104: 2025-11-18 01:40:26 INFO:  81% (12.8 GiB of 15.9 GiB) in 10m 9s, read: 1.6 MiB/s, write: 1.6 MiB/s
104: 2025-11-18 01:40:46 INFO:  82% (13.1 GiB of 15.9 GiB) in 10m 29s, read: 12.4 MiB/s, write: 12.4 MiB/s
104: 2025-11-18 01:40:49 INFO:  83% (13.2 GiB of 15.9 GiB) in 10m 32s, read: 25.3 MiB/s, write: 24.0 MiB/s
104: 2025-11-18 01:40:52 INFO:  84% (13.4 GiB of 15.9 GiB) in 10m 35s, read: 65.3 MiB/s, write: 65.3 MiB/s
104: 2025-11-18 01:40:55 INFO:  85% (13.5 GiB of 15.9 GiB) in 10m 38s, read: 45.3 MiB/s, write: 45.3 MiB/s
104: 2025-11-18 01:41:15 INFO:  86% (13.7 GiB of 15.9 GiB) in 10m 58s, read: 9.2 MiB/s, write: 9.2 MiB/s
104: 2025-11-18 01:41:19 INFO:  87% (13.8 GiB of 15.9 GiB) in 11m 2s, read: 37.0 MiB/s, write: 37.0 MiB/s
104: 2025-11-18 01:41:23 INFO:  88% (14.0 GiB of 15.9 GiB) in 11m 6s, read: 37.0 MiB/s, write: 37.0 MiB/s
104: 2025-11-18 01:41:27 INFO:  89% (14.1 GiB of 15.9 GiB) in 11m 10s, read: 43.0 MiB/s, write: 43.0 MiB/s
104: 2025-11-18 01:41:42 INFO:  90% (14.3 GiB of 15.9 GiB) in 11m 25s, read: 12.3 MiB/s, write: 12.3 MiB/s
104: 2025-11-18 01:41:50 INFO:  91% (14.4 GiB of 15.9 GiB) in 11m 33s, read: 18.0 MiB/s, write: 18.0 MiB/s
104: 2025-11-18 01:42:24 INFO:  92% (14.6 GiB of 15.9 GiB) in 12m 7s, read: 4.9 MiB/s, write: 4.9 MiB/s
104: 2025-11-18 01:42:28 INFO:  93% (14.8 GiB of 15.9 GiB) in 12m 11s, read: 60.0 MiB/s, write: 60.0 MiB/s
104: 2025-11-18 01:42:31 INFO:  94% (15.0 GiB of 15.9 GiB) in 12m 14s, read: 49.3 MiB/s, write: 49.3 MiB/s
104: 2025-11-18 01:42:34 INFO:  96% (15.3 GiB of 15.9 GiB) in 12m 17s, read: 92.0 MiB/s, write: 90.7 MiB/s
104: 2025-11-18 01:42:41 INFO:  97% (15.4 GiB of 15.9 GiB) in 12m 24s, read: 21.1 MiB/s, write: 21.1 MiB/s
104: 2025-11-18 01:42:44 INFO:  98% (15.6 GiB of 15.9 GiB) in 12m 27s, read: 61.3 MiB/s, write: 60.0 MiB/s
104: 2025-11-18 01:42:58 INFO:  99% (15.9 GiB of 15.9 GiB) in 12m 41s, read: 20.0 MiB/s, write: 20.0 MiB/s
104: 2025-11-18 01:43:43 INFO: 100% (15.9 GiB of 15.9 GiB) in 13m 26s, read: 91.0 KiB/s, write: 91.0 KiB/s
104: 2025-11-18 01:46:25 INFO: backup was done incrementally, reused 484.68 GiB (96%)
104: 2025-11-18 01:46:25 INFO: transferred 15.86 GiB in 968 seconds (16.8 MiB/s)
104: 2025-11-18 01:46:25 INFO: adding notes to backup
104: 2025-11-18 01:46:25 INFO: Finished Backup of VM 104 (00:16:24)

105: 2025-11-18 01:46:25 INFO: Starting Backup of VM 105 (lxc)
105: 2025-11-18 01:46:25 INFO: status = running
105: 2025-11-18 01:46:25 INFO: CT Name: nextcloud
105: 2025-11-18 01:46:25 INFO: including mount point rootfs ('/') in backup
105: 2025-11-18 01:46:25 INFO: including mount point mp0 ('/media/nextcloud-data') in backup
105: 2025-11-18 01:46:25 INFO: backup mode: snapshot
105: 2025-11-18 01:46:25 INFO: ionice priority: 7
105: 2025-11-18 01:46:25 INFO: suspend vm to make snapshot
105: 2025-11-18 01:46:25 INFO: create storage snapshot 'vzdump'
105: 2025-11-18 01:48:08 INFO: resume vm
105: 2025-11-18 01:48:08 INFO: guest is online again after 103 seconds
105: 2025-11-18 01:48:08 INFO: creating Proxmox Backup Server archive 'ct/105/2025-11-18T00:46:25Z'
105: 2025-11-18 01:48:08 INFO: set max number of entries in memory for file-based backups to 1048576
105: 2025-11-18 01:48:08 INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp1078781_105/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --include-dev /mnt/vzsnap0/./media/nextcloud-data --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 105 --backup-time 1763426785 --entries-max 1048576 --repository root@pam@XXX:backup-local
105: 2025-11-18 01:48:08 INFO: Starting backup: ct/105/2025-11-18T00:46:25Z   
105: 2025-11-18 01:48:08 INFO: Client name: proxmox   
105: 2025-11-18 01:48:08 INFO: Starting backup protocol: Tue Nov 18 01:48:08 2025   
105: 2025-11-18 01:48:09 INFO: Downloading previous manifest (Mon Nov 17 02:12:52 2025)   
105: 2025-11-18 01:48:09 INFO: Upload config file '/var/tmp/vzdumptmp1078781_105/etc/vzdump/pct.conf' to 'root@pam@XXX:8007:backup-local' as pct.conf.blob   
105: 2025-11-18 01:48:09 INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@XXX:8007:backup-local' as root.pxar.didx   
105: 2025-11-18 01:49:10 INFO: processed 5.538 GiB in 1m, uploaded 0 B
105: 2025-11-18 01:50:10 INFO: processed 9.681 GiB in 2m, uploaded 0 B
105: 2025-11-18 01:51:10 INFO: processed 15.098 GiB in 3m, uploaded 0 B
105: 2025-11-18 01:52:10 INFO: processed 17.535 GiB in 4m, uploaded 0 B
105: 2025-11-18 01:53:10 INFO: processed 18.941 GiB in 5m, uploaded 0 B
105: 2025-11-18 01:54:10 INFO: processed 20.801 GiB in 6m, uploaded 0 B
105: 2025-11-18 01:55:10 INFO: processed 21.23 GiB in 7m, uploaded 0 B
105: 2025-11-18 01:56:10 INFO: processed 21.49 GiB in 8m, uploaded 0 B
105: 2025-11-18 01:57:10 INFO: processed 21.87 GiB in 9m, uploaded 0 B
105: 2025-11-18 01:58:10 INFO: processed 22.728 GiB in 10m, uploaded 0 B
[...]
105: 2025-11-18 05:28:10 INFO: processed 442.772 GiB in 3h 40m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:29:10 INFO: processed 443.668 GiB in 3h 41m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:30:10 INFO: processed 447.238 GiB in 3h 42m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:31:10 INFO: processed 448.48 GiB in 3h 43m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:32:10 INFO: processed 448.968 GiB in 3h 44m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:33:10 INFO: processed 452.859 GiB in 3h 45m 0s, uploaded 224.86 MiB
105: 2025-11-18 05:34:10 INFO: processed 455.778 GiB in 3h 46m 0s, uploaded 231.926 MiB
105: 2025-11-18 05:34:49 INFO: root.pxar: had to backup 551.261 MiB of 465.528 GiB (compressed 208.387 MiB) in 13599.36 s (average 41.508 KiB/s)
105: 2025-11-18 05:34:49 INFO: root.pxar: backup was done incrementally, reused 464.99 GiB (99.9%)
105: 2025-11-18 05:34:49 INFO: Uploaded backup catalog (22.747 MiB)
105: 2025-11-18 05:34:50 INFO: Duration: 13601.67s   
105: 2025-11-18 05:34:50 INFO: End Time: Tue Nov 18 05:34:50 2025   
105: 2025-11-18 05:34:50 INFO: adding notes to backup
105: 2025-11-18 05:34:51 INFO: cleanup temporary 'vzdump' snapshot
105: 2025-11-18 05:34:52 INFO: Finished Backup of VM 105 (03:48:27)

106: 2025-11-18 05:34:52 INFO: Starting Backup of VM 106 (lxc)
106: 2025-11-18 05:34:52 INFO: status = running
106: 2025-11-18 05:34:52 INFO: CT Name: database
106: 2025-11-18 05:34:52 INFO: including mount point rootfs ('/') in backup
106: 2025-11-18 05:34:52 INFO: backup mode: snapshot
106: 2025-11-18 05:34:52 INFO: ionice priority: 7
106: 2025-11-18 05:34:52 INFO: create storage snapshot 'vzdump'
106: 2025-11-18 05:34:52 INFO: creating Proxmox Backup Server archive 'ct/106/2025-11-18T04:34:52Z'
106: 2025-11-18 05:34:52 INFO: set max number of entries in memory for file-based backups to 1048576
106: 2025-11-18 05:34:52 INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp1078781_106/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 106 --backup-time 1763440492 --entries-max 1048576 --repository root@pam@XXX:backup-local
106: 2025-11-18 05:34:52 INFO: Starting backup: ct/106/2025-11-18T04:34:52Z   
106: 2025-11-18 05:34:52 INFO: Client name: proxmox   
106: 2025-11-18 05:34:52 INFO: Starting backup protocol: Tue Nov 18 05:34:52 2025   
106: 2025-11-18 05:34:52 INFO: Downloading previous manifest (Mon Nov 17 06:35:36 2025)   
106: 2025-11-18 05:34:52 INFO: Upload config file '/var/tmp/vzdumptmp1078781_106/etc/vzdump/pct.conf' to 'root@pam@XXX:8007:backup-local' as pct.conf.blob   
106: 2025-11-18 05:34:52 INFO: Upload directory '/mnt/vzsnap0' to 'root@pam@XXX:8007:backup-local' as root.pxar.didx   
106: 2025-11-18 05:35:11 INFO: root.pxar: had to backup 874.243 MiB of 5.107 GiB (compressed 205.425 MiB) in 19.30 s (average 45.289 MiB/s)
106: 2025-11-18 05:35:11 INFO: root.pxar: backup was done incrementally, reused 4.253 GiB (83.3%)
106: 2025-11-18 05:35:11 INFO: Uploaded backup catalog (1.229 MiB)
106: 2025-11-18 05:35:27 INFO: Duration: 35.19s   
106: 2025-11-18 05:35:27 INFO: End Time: Tue Nov 18 05:35:27 2025   
106: 2025-11-18 05:35:27 INFO: adding notes to backup
106: 2025-11-18 05:35:28 INFO: cleanup temporary 'vzdump' snapshot
106: 2025-11-18 05:35:28 INFO: Finished Backup of VM 106 (00:00:36)

Thanks a lot for your help.
 
For the VM backup, there is dirty bitmap tracking which therefore requires to only read and upload the changed data since the previous backup (about 15G in you case). For the container backups there is no dirty bitmap, so all data has to be re-read and re-chunked before deciding if it needs to be re-uploaded. You can use the change detection mode set to metadata to avoid this, see https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_ct_change_detection_mode