[SOLVED] lxc container backup suspend mode exit code 23

[zeusadm]

if i try make backup a lxc container in suspend mode i've got this error:
command 'rsync --stats -h -X -A --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/295998/root//./ /var/tmp/vzdumptmp296420_101' failed: exit code 23

if i make backup in stop mode - all works fine.

 

[dylanw]

Rsync's exit code 23 signifies a partial transfer due to error, but doesn't give much more information than that.
Could you provide the entire output for the task, to see if it has any more information?
Also the container config file could be helpful (pct config <ct_id>)

 

[zeusadm]

Rsync's exit code 23 signifies a partial transfer due to error, but doesn't give much more information than that.
Could you provide the entire output for the task, to see if it has any more information?
Also the container config file could be helpful (pct config <ct_id>)

root@prox1-ftm:~# vzdump 101 --mode suspend --storage local --compress lzo --node prox1-ftm --remove 0
INFO: starting new backup job: vzdump 101 --mode suspend --storage local --remove 0 --node prox1-ftm --compress lzo
INFO: filesystem type on dumpdir is 'zfs' -using /var/tmp/vzdumptmp510946_101 for temporary files
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2021-07-28 13:17:48
INFO: status = running
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: CT Name: test-ct
INFO: including mount point rootfs ('/') in backup
INFO: starting first sync /proc/499370/root/ to /var/tmp/vzdumptmp510946_101
ERROR: Backup of VM 101 failed - command 'rsync --stats -h -X -A --numeric-ids -aH --delete --no-whole-file --sparse --one-file-system --relative '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' /proc/499370/root//./ /var/tmp/vzdumptmp510946_101' failed: exit code 23
INFO: Failed at 2021-07-28 13:18:03
INFO: Backup job finished with errors
job errors
-------------
root@prox1-ftm:~# pct config 101
arch: amd64
cores: 1
hostname: test-ct
memory: 512
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=9E:1F:33:85:3B:CD,ip=dhcp,type=veth
ostype: ubuntu
rootfs: nfsproxmoxct:101/vm-101-disk-0.raw,size=8G
swap: 512
unprivileged: 1

 

[dylanw]

Could you try just running the failed rsync command with the flags -v --progress added, and the target directory (/var/tmp/vzdumptmp510946_101, in the above case) set somewhere you can write. This should show which file(s) are actually failing to transfer.
Note that the number in /proc/499370/root//./ above is the process id of the running container, and may have also changed, if the container has been stopped since.

 

[zeusadm]

Could you try just running the failed rsync command with the flags -v --progress added, and the target directory (/var/tmp/vzdumptmp510946_101, in the above case) set somewhere you can write. This should show which file(s) are actually failing to transfer.
Note that the number in /proc/499370/root//./ above is the process id of the running container, and may have also changed, if the container has been stopped since.

rsync: [generator] set_acl: sys_acl_set_file(var/log/journal, ACL_TYPE_ACCESS): Operation not supported (95)
rsync: [generator] set_acl: sys_acl_set_file(var/log/journal/64cffbdcdc0c462b8dab986582a1a7bb, ACL_TYPE_ACCESS): Operation not supported (95)
rsync: [receiver] set_acl: sys_acl_set_file(var/log/journal/64cffbdcdc0c462b8dab986582a1a7bb/.system.journal.27ZmIv, ACL_TYPE_ACCESS): Operation not supported (95)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]

 

[dylanw]

The problem here is that the target which is set does not support acls [1], which are in use for the files shown in the error. For suspend mode container backups setting the temporary target to a location which supports acls is a requirement (see Backup modes for Containers [2]). You will need to run the vzdump command with the '--tmpdir <path>' option where <path> is a directory with acl support enabled.

[1] https://linux.die.net/man/5/acl
[2] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_backup_modes

 

[zeusadm]

The problem here is that the target which is set does not support acls [1], which are in use for the files shown in the error. For suspend mode container backups setting the temporary target to a location which supports acls is a requirement (see Backup modes for Containers [2]). You will need to run the vzdump command with the '--tmpdir <path>' option where <path> is a directory with acl support enabled.

[1] https://linux.die.net/man/5/acl
[2] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#_backup_modes

thank you for your help!

 

[Zubin Singh Parihar]

The typical problem is that you are running ZFS without POSIX ACL Support.

The LXC container has ACL settings inside its filesystem and the 'snaphot' backup process that the Proxmox VE host runs is an rsync to the /var/tmp directory. If POSIX ACL is not turned on in the rpool/ROOT/pve-1 dataset (and it isn't by default for whatever strange reason, and the Proxmox devs should and hopefully will do that in the next release), then the rsync will fail.

TEST:

$ zfs get acltype rpool/ROOT/pve-1

if it returns:

NAME              PROPERTY  VALUE     SOURCE
rpool/ROOT/pve-1  acltype   off       default

That means ACL's are not on.




SOLUTION:

Enable ZFS POSIX ACLs:

$ zfs set acltype=posixacl rpool/ROOT/pve-1

Check it again:

$ zfs get acltype rpool/ROOT/pve-1

if it returns:

NAME              PROPERTY  VALUE     SOURCE
rpool/ROOT/pve-1  acltype   posix     local

then Success!



Now try that LXC Backup again!


Credit goes to this guy: @CH.illig --> https://forum.proxmox.com/members/ch-illig.36347/ for his post (in German - thank you Google Translate):
https://forum.proxmox.com/threads/lxc-backup-fehler-wegen-acl.129309/


I hope this helps!