Backup failed with exit code 2

S

SAM_N

New Member
Nov 16, 2019
3
0
1
Hello together,

I tried to backup a new container on my Proxmox VE Version 5.4-4.
The container is running an "Ubuntu 18.04"-System.
During the process, I got this error message with the exit code 2:

Code: 
INFO: starting new backup job: vzdump 100 --storage Backup --compress lzo --node pve --remove 0 --mode snapshot
INFO: Starting Backup of VM 100 (lxc)
INFO: status = running
INFO: CT Name: TEST
INFO: mode failure - some volumes do not support snapshots
INFO: trying 'suspend' mode instead
INFO: backup mode: suspend
INFO: ionice priority: 7
INFO: CT Name: TEST
INFO: temporary directory is on NFS, disabling xattr and acl support, consider configuring a local tmpdir via /etc/vzdump.conf
INFO: starting first sync /proc/22388/root// to /mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tmp
INFO: Number of files: 27,882 (reg: 21,436, dir: 2,851, link: 3,564, special: 31)
INFO: Number of created files: 27,881 (reg: 21,436, dir: 2,850, link: 3,564, special: 31)
INFO: Number of deleted files: 0
INFO: Number of regular files transferred: 21,417
INFO: Total file size: 1,487,873,500 bytes
INFO: Total transferred file size: 1,350,324,594 bytes
INFO: Literal data: 1,350,325,223 bytes
INFO: Matched data: 0 bytes
INFO: File list size: 851,921
INFO: File list generation time: 0.001 seconds
INFO: File list transfer time: 0.000 seconds
INFO: Total bytes sent: 1,352,280,166
INFO: Total bytes received: 434,222
INFO: sent 1,352,280,166 bytes  received 434,222 bytes  13,197,213.54 bytes/sec
INFO: total size is 1,487,873,500  speedup is 1.10
INFO: first sync finished (102 seconds)
INFO: suspend vm
INFO: starting final sync /proc/22388/root// to /mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tmp
INFO: Number of files: 27,882 (reg: 21,436, dir: 2,851, link: 3,564, special: 31)
INFO: Number of created files: 0
INFO: Number of deleted files: 0
INFO: Number of regular files transferred: 5
INFO: Total file size: 1,487,890,513 bytes
INFO: Total transferred file size: 130,606,735 bytes
INFO: Literal data: 25,292 bytes
INFO: Matched data: 130,581,443 bytes
INFO: File list size: 65,535
INFO: File list generation time: 0.001 seconds
INFO: File list transfer time: 0.000 seconds
INFO: Total bytes sent: 800,761
INFO: Total bytes received: 120,264
INFO: sent 800,761 bytes  received 120,264 bytes  167,459.09 bytes/sec
INFO: total size is 1,487,890,513  speedup is 1,615.47
INFO: final sync finished (5 seconds)
INFO: resume vm
INFO: vm is online again after 6 seconds
INFO: creating archive '/mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tar.lzo'
INFO: tar: ./var/cache/apt/archives/partial: Cannot open: Permission denied
INFO: tar: ./var/lib/apt/lists/partial: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/mysql: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/performance_schema: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/TEST: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/aria_log.00000001: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/aria_log_control: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/ib_logfile0: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/ib_logfile1: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/ibdata1: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/multi-master.info: Cannot open: Permission denied
INFO: tar: ./var/lib/mysql/tc.log: Cannot open: Permission denied
INFO: tar: ./var/lib/php/sessions/sess_j2cumhm9mag81otvk3urmljdd9: Cannot open: Permission denied
INFO: tar: ./var/lib/postfix/master.lock: Cannot open: Permission denied
INFO: tar: ./var/lib/snmp/snmpd.conf: Cannot open: Permission denied
INFO: tar: ./var/log/mysql: Cannot open: Permission denied
INFO: tar: ./var/log/auth.log: Cannot open: Permission denied
INFO: tar: ./var/log/mail.log: Cannot open: Permission denied
INFO: tar: ./var/log/syslog: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/active: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/bounce: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/corrupt: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/defer: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/deferred: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/flush: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/hold: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/incoming: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/maildrop: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/private: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/public: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/saved: Cannot open: Permission denied
INFO: tar: ./var/spool/postfix/trace: Cannot open: Permission denied
INFO: tar: ./var/spool/rsyslog: Cannot open: Permission denied
INFO: Total bytes written: 1118330880 (1.1GiB, 32MiB/s)
INFO: tar: Exiting with failure status due to previous errors
ERROR: Backup of VM 100 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tmp' ./etc/vzdump/pct.conf '--directory=/mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tmp' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' . | lzop >/mnt/pve/Backup/dump/vzdump-lxc-100-2019_11_16-13_30_12.tar.dat' failed: exit code 2
INFO: Backup job finished with errors
TASK ERROR: job errors

I changed between a manual backup and a time backup, but the backup still failed.
Maybe someone of you can help me with this problem.

Best regards,

SAM_N
 
Last edited:
hi,

is the container privileged or unprivileged?
 
timonych said:
For unprivileged container You can use STOP mode
Click to expand...

the backup mode has nothing to do with privileged/unprivileged. it depends on the underlying storage (if it supports snapshots or not)
 
oguz said:
the backup mode has nothing to do with privileged/unprivileged. it depends on the underlying storage (if it supports snapshots or not)
Click to expand...
NFS storage can consist of QCOW images
oguz said:
the backup mode has nothing to do with privileged/unprivileged. it depends on the underlying storage (if it supports snapshots or not)
Click to expand...
But if I would backup VM with RAM Image and Backup storage is NFS storage - there is no error.
If I would backup Unprivelleged LXC to NFS Storage it will failed in Snapshot/Suspend mode and have success in Stop mode.
 
timonych said:
NFS storage can consist of QCOW images
Click to expand...
yes, but containers use raw images on NFS

timonych said:
If I would backup Unprivelleged LXC to NFS Storage it will failed in Snapshot/Suspend mode and have success in Stop mode.
Click to expand...
again, this doesn't have anything to do with privileged/unprivileged.
 
I do have the same issue with an LXC container (unprivileged) on a local hw raid with LVM - backup destination is NFS.
Running version 6.1

Thanks,
screenie

oguz said:
hi,

is the container privileged or unprivileged?
Click to expand...
 
Hi,

I can confirm this - we are using:
proxmox-ve: 5.4-2
pve-manager: 5.4-13

My Containers are all located on LVM-THIN storages. All backups are done with snapshots.
I have changed some backup targets a few days before from local to a NFS storage. The local backups with snapshots had worked.
After the change to the NFS as backup target the snapshot backups of the unpriviledged containers failed with the "exit code 2" as described above in the first post.

When I "change" the container to privledged the backups to NFS are working again.

So it has to do with priviledged or not because my storage is LVM-THIN and local snapshot backups are working as well.

I think it is a bug...

Best Regards

Sh
 
Same here:
PVE: 6.4-13
Kernel Version: Linux 5.4.128-1-pve #1 SMP PVE 5.4.128-2 (Wed, 18 Aug 2021 16:20:02 +0200)

Failing LXC-Container: Ubuntu 20.4, unprivileged.

Target-Backup-Storage: NFS with vzdump backup files
Compression: ZSTD
Mode: Snapshot

Error-Message:
Bash: 
command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/vm-backup/dump/vzdump-lxc-110-2021_09_16-07_56_29.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd --rsyncable '--threads=1' >/mnt/pve/vm-backup/dump/vzdump-lxc-110-2021_09_16-07_56_29.tar.dat' failed: exit code 2
 
  • Like
Reactions: joeplaa
neuhausf said:
Same here:
PVE: 6.4-13
Kernel Version: Linux 5.4.128-1-pve #1 SMP PVE 5.4.128-2 (Wed, 18 Aug 2021 16:20:02 +0200)

Failing LXC-Container: Ubuntu 20.4, unprivileged.

Target-Backup-Storage: NFS with vzdump backup files
Compression: ZSTD
Mode: Snapshot

Error-Message:
Bash: 
command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/vm-backup/dump/vzdump-lxc-110-2021_09_16-07_56_29.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd --rsyncable '--threads=1' >/mnt/pve/vm-backup/dump/vzdump-lxc-110-2021_09_16-07_56_29.tar.dat' failed: exit code 2
Click to expand...
I run into the same issue as @neuhausf. A few lines above the error I see another line:

Bash: 
INFO: tar: /mnt/pve/nas-nfs/dump/vzdump-lxc-102-2021_11_20-01_00_02.tmp: Cannot open: Permission denied

Weird, as other backups do work. I couldn't find a correlation between excluded mountpoints, added mountpoints or not. Some LXC's backup successfully, others don't. See excerpt from the backup logs below. Then I look a little further and all unprivileged containers fail, all privileged succeed. I don't understand why though.

Is this a bug? Or a new setting I'm missing. It seemed to be introduced after I upgraded Proxmox to the latest version last weekend. Upgraded again today to version: pve-manager/7.1-5/6fe299a0 (running kernel: 5.11.22-5-pve) and the problem still persists.

Bash: 
102: 2021-11-20 01:00:02 INFO: Starting Backup of VM 102 (lxc)
102: 2021-11-20 01:00:02 INFO: status = running
102: 2021-11-20 01:00:02 INFO: CT Name: joeplaa-mailer
102: 2021-11-20 01:00:02 INFO: including mount point rootfs ('/') in backup
102: 2021-11-20 01:00:02 INFO: excluding volume mount point mp0 ('/var/lib/docker') from backup (disabled)
102: 2021-11-20 01:00:02 INFO: backup mode: snapshot
102: 2021-11-20 01:00:02 INFO: ionice priority: 7
102: 2021-11-20 01:00:02 INFO: create storage snapshot 'vzdump'
102: 2021-11-20 01:00:03 INFO: creating vzdump archive '/mnt/pve/nas-nfs/dump/vzdump-lxc-102-2021_11_20-01_00_02.tar.zst'
102: 2021-11-20 01:00:03 INFO: tar: /mnt/pve/nas-nfs/dump/vzdump-lxc-102-2021_11_20-01_00_02.tmp: Cannot open: Permission denied
102: 2021-11-20 01:00:03 INFO: tar: Error is not recoverable: exiting now
102: 2021-11-20 01:00:03 INFO: cleanup temporary 'vzdump' snapshot
102: 2021-11-20 01:00:03 ERROR: Backup of VM 102 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/nas-nfs/dump/vzdump-lxc-102-2021_11_20-01_00_02.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd --rsyncable '--threads=1' >/mnt/pve/nas-nfs/dump/vzdump-lxc-102-2021_11_20-01_00_02.tar.dat' failed: exit code 2

151: 2021-11-20 01:07:00 INFO: Starting Backup of VM 151 (lxc)
151: 2021-11-20 01:07:00 INFO: status = running
151: 2021-11-20 01:07:00 INFO: CT Name: jodiSonatype
151: 2021-11-20 01:07:00 INFO: including mount point rootfs ('/') in backup
151: 2021-11-20 01:07:00 INFO: excluding volume mount point mp0 ('/var/lib/docker') from backup (disabled)
151: 2021-11-20 01:07:00 INFO: excluding bind mount point mp1 ('/mnt/sonatype_cache') from backup (not a volume)
151: 2021-11-20 01:07:00 INFO: backup mode: snapshot
151: 2021-11-20 01:07:00 INFO: ionice priority: 7
151: 2021-11-20 01:07:00 INFO: create storage snapshot 'vzdump'
151: 2021-11-20 01:07:00 INFO: creating vzdump archive '/mnt/pve/nas-nfs/dump/vzdump-lxc-151-2021_11_20-01_07_00.tar.zst'
151: 2021-11-20 01:08:37 INFO: Total bytes written: 5879142400 (5.5GiB, 58MiB/s)
151: 2021-11-20 01:08:38 INFO: archive file size: 886MB
151: 2021-11-20 01:08:38 INFO: prune older backups with retention: keep-monthly=3, keep-weekly=4, keep-yearly=1
151: 2021-11-20 01:08:38 INFO: removing backup 'nas-nfs:backup/vzdump-lxc-151-2021_10_23-01_12_27.tar.zst'
151: 2021-11-20 01:08:38 INFO: pruned 1 backup(s) not covered by keep-retention policy
151: 2021-11-20 01:08:38 INFO: cleanup temporary 'vzdump' snapshot
151: 2021-11-20 01:08:38 INFO: Finished Backup of VM 151 (00:01:38)
 
Solved my problem by updating /etc/vzdump.conf as described here: https://forum.proxmox.com/threads/can-not-backup-lxc-unprivileged.71878/post-321844 and here: https://www.reddit.com/r/Proxmox/comments/9ucz3o/lxc_unprivileged_backup_task_failing/.

My vzdump.conf now looks like this:

INI: 
# vzdump default settings

tmpdir: /npool/tmp
#dumpdir: DIR
#storage: STORAGE_ID
#mode: snapshot|suspend|stop
#bwlimit: KBPS
#ionice: PRI
#lockwait: MINUTES
#stopwait: MINUTES
#stdexcludes: BOOLEAN
#mailto: ADDRESSLIST
#prune-backups: keep-INTERVAL=N[,...]
#script: FILENAME
#exclude-path: PATHLIST
#pigz: N
 
  • Like
Reactions: gregramsaran and timonych
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back