Backup lxc with exit code 2

Vittorio

Well-Known Member
Jul 24, 2019
90
5
48
60
Hi

I'm trying to backup an unprivileged lxc and I get an error

Code:
116: 2023-06-24 04:49:04 INFO: Starting Backup of VM 116 (lxc)
116: 2023-06-24 04:49:04 INFO: status = running
116: 2023-06-24 04:49:04 INFO: backup mode: stop
116: 2023-06-24 04:49:04 INFO: ionice priority: 7
116: 2023-06-24 04:49:04 INFO: CT Name: mariadb
116: 2023-06-24 04:49:04 INFO: including mount point rootfs ('/') in backup
116: 2023-06-24 04:49:04 INFO: stopping virtual guest
116: 2023-06-24 04:49:07 INFO: creating vzdump archive '/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tar.zst'
116: 2023-06-24 04:49:07 INFO: tar: /mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp: Cannot open: Permission denied
116: 2023-06-24 04:49:07 INFO: tar: Error is not recoverable: exiting now
116: 2023-06-24 04:49:07 INFO: restarting vm
116: 2023-06-24 04:49:10 INFO: guest is online again after 6 seconds
116: 2023-06-24 04:49:10 ERROR: Backup of VM 116 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd --rsyncable '--threads=1' >/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tar.dat' failed:
exit code 2

And this is the lxc config

Code:
arch: amd64
cores: 1
features: keyctl=1,nesting=1
hostname: mariadb
memory: 1024
net0: name=eth0,bridge=vmbr0,gw=192.168.1.10,hwaddr=5A:42:BB:78:D9:B0,ip=192.168.1.39/24,type=veth
onboot: 1
ostype: debian
rootfs: DATI:vm-116-disk-0,size=6G
startup: order=2,up=180
swap: 512
tags: proxmox-helper-scripts
unprivileged: 1
root@pve:/etc/pve/lxc#

What can I check?

Thanks
 
hello looks like a permission error on the nfs share "/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp: Cannot open: Permission denied".


check so that the proxmox host has read and write access to the "NFS-QNAP-Proxmox-Backup" share.
 
hello looks like a permission error on the nfs share "/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp: Cannot open: Permission denied".


check so that the proxmox host has read and write access to the "NFS-QNAP-Proxmox-Backup" share.
Strange because the backup job include a vm and an lxc both on same destination and the vm works, the lxc does not

Here is the full log

Code:
vzdump 106 116 --compress zstd --mode stop --quiet 1 --storage NFS-QNAP-Proxmox-Backup --mailnotification failure --mailto woody4165@gmail.com --notes-template '{{guestname}}' --prune-backups 'keep-last=3'


106: 2023-06-24 04:30:00 INFO: Starting Backup of VM 106 (qemu)
106: 2023-06-24 04:30:00 INFO: status = running
106: 2023-06-24 04:30:00 INFO: backup mode: stop
106: 2023-06-24 04:30:00 INFO: ionice priority: 7
106: 2023-06-24 04:30:00 INFO: VM Name: haos9.3
106: 2023-06-24 04:30:00 INFO: include disk 'scsi0' 'DATI:vm-106-disk-1' 42G
106: 2023-06-24 04:30:00 INFO: include disk 'efidisk0' 'DATI:vm-106-disk-0' 4M
106: 2023-06-24 04:30:00 INFO: stopping virtual guest
106: 2023-06-24 04:30:20 INFO: snapshots found (not included into backup)
106: 2023-06-24 04:30:20 INFO: creating vzdump archive '/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-qemu-106-2023_06_24-04_30_00.vma.zst'
106: 2023-06-24 04:30:20 INFO: starting kvm to execute backup task
106: 2023-06-24 04:30:21 INFO: started backup task '653d9045-d2fe-46ac-8100-543327b5ed1e'
106: 2023-06-24 04:30:21 INFO: resuming VM again after 21 seconds
106: 2023-06-24 04:30:24 INFO:   0% (406.0 MiB of 42.0 GiB) in 3s, read: 135.3 MiB/s, write: 101.9 MiB/s
106: 2023-06-24 04:30:27 INFO:   2% (902.6 MiB of 42.0 GiB) in 6s, read: 165.5 MiB/s, write: 104.3 MiB/s
106: 2023-06-24 04:30:31 INFO:   3% (1.3 GiB of 42.0 GiB) in 10s, read: 115.1 MiB/s, write: 97.8 MiB/s
106: 2023-06-24 04:30:36 INFO:   4% (1.7 GiB of 42.0 GiB) in 15s, read: 79.0 MiB/s, write: 68.4 MiB/s
106: 2023-06-24 04:30:40 INFO:   5% (2.1 GiB of 42.0 GiB) in 19s, read: 106.0 MiB/s, write: 79.8 MiB/s
106: 2023-06-24 04:30:44 INFO:   6% (2.6 GiB of 42.0 GiB) in 23s, read: 115.1 MiB/s, write: 101.7 MiB/s
106: 2023-06-24 04:30:48 INFO:   7% (3.0 GiB of 42.0 GiB) in 27s, read: 106.0 MiB/s, write: 93.2 MiB/s
106: 2023-06-24 04:30:52 INFO:   8% (3.4 GiB of 42.0 GiB) in 31s, read: 100.6 MiB/s, write: 95.4 MiB/s
106: 2023-06-24 04:30:55 INFO:   9% (3.8 GiB of 42.0 GiB) in 34s, read: 157.1 MiB/s, write: 110.7 MiB/s
106: 2023-06-24 04:30:58 INFO:  10% (4.3 GiB of 42.0 GiB) in 37s, read: 145.0 MiB/s, write: 112.6 MiB/s
106: 2023-06-24 04:31:01 INFO:  11% (4.6 GiB of 42.0 GiB) in 40s, read: 122.0 MiB/s, write: 94.8 MiB/s
106: 2023-06-24 04:31:07 INFO:  12% (5.1 GiB of 42.0 GiB) in 46s, read: 86.4 MiB/s, write: 76.3 MiB/s
106: 2023-06-24 04:31:12 INFO:  13% (5.5 GiB of 42.0 GiB) in 51s, read: 72.5 MiB/s, write: 63.5 MiB/s
106: 2023-06-24 04:31:17 INFO:  14% (5.9 GiB of 42.0 GiB) in 56s, read: 81.2 MiB/s, write: 68.9 MiB/s
106: 2023-06-24 04:31:34 INFO:  15% (6.3 GiB of 42.0 GiB) in 1m 13s, read: 27.3 MiB/s, write: 21.9 MiB/s
106: 2023-06-24 04:31:53 INFO:  16% (6.7 GiB of 42.0 GiB) in 1m 32s, read: 20.6 MiB/s, write: 19.8 MiB/s
106: 2023-06-24 04:32:16 INFO:  17% (7.1 GiB of 42.0 GiB) in 1m 55s, read: 18.9 MiB/s, write: 16.7 MiB/s
106: 2023-06-24 04:32:32 INFO:  18% (7.6 GiB of 42.0 GiB) in 2m 11s, read: 28.1 MiB/s, write: 22.7 MiB/s
106: 2023-06-24 04:32:48 INFO:  19% (8.0 GiB of 42.0 GiB) in 2m 27s, read: 26.5 MiB/s, write: 22.6 MiB/s
106: 2023-06-24 04:32:59 INFO:  20% (8.4 GiB of 42.0 GiB) in 2m 38s, read: 40.2 MiB/s, write: 29.5 MiB/s
106: 2023-06-24 04:33:11 INFO:  21% (8.9 GiB of 42.0 GiB) in 2m 50s, read: 35.6 MiB/s, write: 26.2 MiB/s
106: 2023-06-24 04:33:20 INFO:  22% (9.3 GiB of 42.0 GiB) in 2m 59s, read: 47.9 MiB/s, write: 39.2 MiB/s
106: 2023-06-24 04:33:38 INFO:  23% (9.7 GiB of 42.0 GiB) in 3m 17s, read: 24.4 MiB/s, write: 22.1 MiB/s
106: 2023-06-24 04:33:49 INFO:  24% (10.1 GiB of 42.0 GiB) in 3m 28s, read: 36.6 MiB/s, write: 27.7 MiB/s
106: 2023-06-24 04:34:02 INFO:  25% (10.6 GiB of 42.0 GiB) in 3m 41s, read: 36.5 MiB/s, write: 23.4 MiB/s
106: 2023-06-24 04:34:22 INFO:  26% (10.9 GiB of 42.0 GiB) in 4m 1s, read: 19.6 MiB/s, write: 16.1 MiB/s
106: 2023-06-24 04:34:31 INFO:  27% (11.3 GiB of 42.0 GiB) in 4m 10s, read: 46.3 MiB/s, write: 42.4 MiB/s
106: 2023-06-24 04:34:44 INFO:  28% (11.9 GiB of 42.0 GiB) in 4m 23s, read: 41.8 MiB/s, write: 27.4 MiB/s
106: 2023-06-24 04:34:53 INFO:  29% (12.2 GiB of 42.0 GiB) in 4m 32s, read: 35.4 MiB/s, write: 27.5 MiB/s
106: 2023-06-24 04:35:11 INFO:  30% (12.6 GiB of 42.0 GiB) in 4m 50s, read: 24.2 MiB/s, write: 19.1 MiB/s
106: 2023-06-24 04:35:21 INFO:  31% (13.1 GiB of 42.0 GiB) in 5m, read: 46.0 MiB/s, write: 34.0 MiB/s
106: 2023-06-24 04:35:30 INFO:  32% (13.4 GiB of 42.0 GiB) in 5m 9s, read: 43.1 MiB/s, write: 37.6 MiB/s
106: 2023-06-24 04:35:40 INFO:  33% (13.9 GiB of 42.0 GiB) in 5m 19s, read: 45.3 MiB/s, write: 35.8 MiB/s
106: 2023-06-24 04:35:52 INFO:  34% (14.3 GiB of 42.0 GiB) in 5m 31s, read: 33.8 MiB/s, write: 22.7 MiB/s
106: 2023-06-24 04:36:17 INFO:  35% (14.7 GiB of 42.0 GiB) in 5m 56s, read: 17.5 MiB/s, write: 15.9 MiB/s
106: 2023-06-24 04:36:32 INFO:  36% (15.1 GiB of 42.0 GiB) in 6m 11s, read: 29.0 MiB/s, write: 27.0 MiB/s
106: 2023-06-24 04:36:42 INFO:  37% (15.6 GiB of 42.0 GiB) in 6m 21s, read: 43.1 MiB/s, write: 34.5 MiB/s
106: 2023-06-24 04:36:53 INFO:  38% (16.0 GiB of 42.0 GiB) in 6m 32s, read: 39.9 MiB/s, write: 38.1 MiB/s
106: 2023-06-24 04:37:08 INFO:  39% (16.4 GiB of 42.0 GiB) in 6m 47s, read: 27.8 MiB/s, write: 24.5 MiB/s
106: 2023-06-24 04:37:25 INFO:  40% (16.8 GiB of 42.0 GiB) in 7m 4s, read: 27.5 MiB/s, write: 23.2 MiB/s
106: 2023-06-24 04:37:40 INFO:  41% (17.2 GiB of 42.0 GiB) in 7m 19s, read: 25.6 MiB/s, write: 19.9 MiB/s
106: 2023-06-24 04:37:51 INFO:  42% (17.7 GiB of 42.0 GiB) in 7m 30s, read: 47.8 MiB/s, write: 29.9 MiB/s
106: 2023-06-24 04:37:54 INFO:  44% (18.5 GiB of 42.0 GiB) in 7m 33s, read: 261.0 MiB/s, write: 46.1 MiB/s
106: 2023-06-24 04:38:04 INFO:  45% (18.9 GiB of 42.0 GiB) in 7m 43s, read: 42.8 MiB/s, write: 38.8 MiB/s
106: 2023-06-24 04:38:15 INFO:  46% (19.3 GiB of 42.0 GiB) in 7m 54s, read: 37.6 MiB/s, write: 34.1 MiB/s
106: 2023-06-24 04:38:36 INFO:  47% (19.8 GiB of 42.0 GiB) in 8m 15s, read: 21.9 MiB/s, write: 20.7 MiB/s
106: 2023-06-24 04:38:50 INFO:  48% (20.2 GiB of 42.0 GiB) in 8m 29s, read: 29.5 MiB/s, write: 27.1 MiB/s
106: 2023-06-24 04:39:11 INFO:  49% (20.6 GiB of 42.0 GiB) in 8m 50s, read: 20.0 MiB/s, write: 19.3 MiB/s
106: 2023-06-24 04:39:27 INFO:  50% (21.0 GiB of 42.0 GiB) in 9m 6s, read: 27.9 MiB/s, write: 25.0 MiB/s
106: 2023-06-24 04:39:34 INFO:  51% (21.5 GiB of 42.0 GiB) in 9m 13s, read: 72.5 MiB/s, write: 20.6 MiB/s
106: 2023-06-24 04:39:48 INFO:  52% (21.9 GiB of 42.0 GiB) in 9m 27s, read: 25.4 MiB/s, write: 20.2 MiB/s
106: 2023-06-24 04:39:56 INFO:  53% (22.3 GiB of 42.0 GiB) in 9m 35s, read: 53.0 MiB/s, write: 32.9 MiB/s
106: 2023-06-24 04:39:59 INFO:  54% (22.8 GiB of 42.0 GiB) in 9m 38s, read: 174.0 MiB/s, write: 34.8 MiB/s
106: 2023-06-24 04:40:15 INFO:  55% (23.2 GiB of 42.0 GiB) in 9m 54s, read: 23.3 MiB/s, write: 18.0 MiB/s
106: 2023-06-24 04:40:29 INFO:  56% (23.5 GiB of 42.0 GiB) in 10m 8s, read: 27.7 MiB/s, write: 20.0 MiB/s
106: 2023-06-24 04:40:48 INFO:  57% (24.0 GiB of 42.0 GiB) in 10m 27s, read: 24.0 MiB/s, write: 20.4 MiB/s
106: 2023-06-24 04:40:58 INFO:  58% (24.4 GiB of 42.0 GiB) in 10m 37s, read: 40.2 MiB/s, write: 33.1 MiB/s
106: 2023-06-24 04:41:09 INFO:  59% (24.9 GiB of 42.0 GiB) in 10m 48s, read: 46.5 MiB/s, write: 28.8 MiB/s
106: 2023-06-24 04:41:17 INFO:  60% (25.2 GiB of 42.0 GiB) in 10m 56s, read: 43.5 MiB/s, write: 32.7 MiB/s
106: 2023-06-24 04:41:29 INFO:  61% (25.7 GiB of 42.0 GiB) in 11m 8s, read: 39.3 MiB/s, write: 31.2 MiB/s
106: 2023-06-24 04:41:45 INFO:  62% (26.1 GiB of 42.0 GiB) in 11m 24s, read: 25.4 MiB/s, write: 18.7 MiB/s
106: 2023-06-24 04:41:53 INFO:  63% (26.5 GiB of 42.0 GiB) in 11m 32s, read: 53.9 MiB/s, write: 33.1 MiB/s
106: 2023-06-24 04:41:59 INFO:  64% (26.9 GiB of 42.0 GiB) in 11m 38s, read: 67.7 MiB/s, write: 37.1 MiB/s
106: 2023-06-24 04:42:14 INFO:  65% (27.3 GiB of 42.0 GiB) in 11m 53s, read: 29.0 MiB/s, write: 22.8 MiB/s
106: 2023-06-24 04:42:35 INFO:  66% (27.7 GiB of 42.0 GiB) in 12m 14s, read: 20.4 MiB/s, write: 15.2 MiB/s
106: 2023-06-24 04:42:50 INFO:  67% (28.2 GiB of 42.0 GiB) in 12m 29s, read: 29.2 MiB/s, write: 22.1 MiB/s
106: 2023-06-24 04:43:01 INFO:  68% (28.6 GiB of 42.0 GiB) in 12m 40s, read: 41.9 MiB/s, write: 29.8 MiB/s
106: 2023-06-24 04:43:13 INFO:  69% (29.0 GiB of 42.0 GiB) in 12m 52s, read: 35.0 MiB/s, write: 20.8 MiB/s
106: 2023-06-24 04:43:27 INFO:  70% (29.4 GiB of 42.0 GiB) in 13m 6s, read: 28.5 MiB/s, write: 21.3 MiB/s
106: 2023-06-24 04:43:46 INFO:  71% (29.8 GiB of 42.0 GiB) in 13m 25s, read: 23.3 MiB/s, write: 19.8 MiB/s
106: 2023-06-24 04:43:58 INFO:  72% (30.3 GiB of 42.0 GiB) in 13m 37s, read: 37.2 MiB/s, write: 28.1 MiB/s
106: 2023-06-24 04:44:14 INFO:  73% (30.8 GiB of 42.0 GiB) in 13m 53s, read: 33.2 MiB/s, write: 19.3 MiB/s
106: 2023-06-24 04:44:17 INFO:  75% (31.5 GiB of 42.0 GiB) in 13m 56s, read: 250.8 MiB/s, write: 17.2 MiB/s
106: 2023-06-24 04:44:36 INFO:  76% (31.9 GiB of 42.0 GiB) in 14m 15s, read: 22.5 MiB/s, write: 20.7 MiB/s
106: 2023-06-24 04:44:53 INFO:  77% (32.3 GiB of 42.0 GiB) in 14m 32s, read: 24.1 MiB/s, write: 22.3 MiB/s
106: 2023-06-24 04:45:18 INFO:  78% (32.8 GiB of 42.0 GiB) in 14m 57s, read: 19.7 MiB/s, write: 17.8 MiB/s
106: 2023-06-24 04:45:33 INFO:  79% (33.2 GiB of 42.0 GiB) in 15m 12s, read: 24.9 MiB/s, write: 23.4 MiB/s
106: 2023-06-24 04:46:04 INFO:  80% (33.6 GiB of 42.0 GiB) in 15m 43s, read: 14.0 MiB/s, write: 14.0 MiB/s
106: 2023-06-24 04:46:35 INFO:  85% (35.9 GiB of 42.0 GiB) in 16m 14s, read: 75.4 MiB/s, write: 11.1 MiB/s
106: 2023-06-24 04:46:38 INFO: 100% (42.0 GiB of 42.0 GiB) in 16m 17s, read: 2.0 GiB/s, write: 0 B/s
106: 2023-06-24 04:46:38 INFO: backup is sparse: 16.31 GiB (38%) total zero data
106: 2023-06-24 04:46:38 INFO: transferred 42.00 GiB in 977 seconds (44.0 MiB/s)
106: 2023-06-24 04:48:57 INFO: archive file size: 11.61GB
106: 2023-06-24 04:48:57 INFO: adding notes to backup
106: 2023-06-24 04:48:57 INFO: prune older backups with retention: keep-last=3
106: 2023-06-24 04:48:58 INFO: removing backup 'NFS-QNAP-Proxmox-Backup:backup/vzdump-qemu-106-2023_06_21-04_30_01.vma.zst'
106: 2023-06-24 04:49:04 INFO: pruned 1 backup(s) not covered by keep-retention policy
106: 2023-06-24 04:49:04 INFO: Finished Backup of VM 106 (00:19:04)

116: 2023-06-24 04:49:04 INFO: Starting Backup of VM 116 (lxc)
116: 2023-06-24 04:49:04 INFO: status = running
116: 2023-06-24 04:49:04 INFO: backup mode: stop
116: 2023-06-24 04:49:04 INFO: ionice priority: 7
116: 2023-06-24 04:49:04 INFO: CT Name: mariadb
116: 2023-06-24 04:49:04 INFO: including mount point rootfs ('/') in backup
116: 2023-06-24 04:49:04 INFO: stopping virtual guest
116: 2023-06-24 04:49:07 INFO: creating vzdump archive '/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tar.zst'
116: 2023-06-24 04:49:07 INFO: tar: /mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp: Cannot open: Permission denied
116: 2023-06-24 04:49:07 INFO: tar: Error is not recoverable: exiting now
116: 2023-06-24 04:49:07 INFO: restarting vm
116: 2023-06-24 04:49:10 INFO: guest is online again after 6 seconds
116: 2023-06-24 04:49:10 ERROR: Backup of VM 116 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd --rsyncable '--threads=1' >/mnt/pve/NFS-QNAP-Proxmox-Backup/dump/vzdump-lxc-116-2023_06_24-04_49_04.tar.dat' failed: exit code 2
 
I have this issue too, and my problem is definitely not a permission issue. My LXC is the only failing in about 30 vms/lxcs.