Problem backing up unprivileged container - permission denied, but only for docker files

C

catzilla

New Member
Apr 8, 2025
6
1
3
Hi. I am trying to back up a container, and it seems to be failing on docker files specifically for some reason.
Could someone please help resolve?

Thank you!

Please see the pastebins (logs are too long for a forum post):
https://pastebin.com/D3ewMxqR
https://pastebin.com/RdJBXm4C


1754099132874.png


Edit: Could this have anything to do with anything?

Code: 
cat /etc/pve/lxc/101.conf 
# uid map%3A from uid 0 map 1005 uids (in the ct) to the range starting 100000 (on the host), so 0..1004 (ct) %E2%86%92 100000..101004 (host)
# we map 1 uid starting from uid 1005 onto 1005, so 1005 %E2%86%92 1005
# we map the rest of 65535 from 1006 upto 101006, so 1006..65535 %E2%86%92 101006..165535
arch: amd64
cores: 6
features: nesting=1
hostname: CT101
memory: 10240
mp0: /tank,mp=/tank
net0: name=eth0,bridge=vmbr0,firewall=1,gw=192.168.1.1,hwaddr=BC:24:11:26:71:46,ip=192.168.1.6/24,ip6=dhcp,type=veth
onboot: 1
ostype: ubuntu
rootfs: local-zfs:subvol-101-disk-0,size=150G
swap: 1024
unprivileged: 1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.cgroup.devices.allow: c 189:* rwm
lxc.mount.entry: /dev/bus/usb dev/dri/usb none bind,optional,create=file
lxc.idmap: u 0 100000 1000
lxc.idmap: g 0 100000 1000
lxc.idmap: u 1000 1000 1
lxc.idmap: g 1000 1000 1
lxc.idmap: u 1001 101001 64530
lxc.idmap: g 1001 101001 64530
lxc.idmap: g 65534 165534 1
 
Last edited:
Tank is a 48 TB ZFS disk.

Code: 
root@pve-server:~# mount | grep tank
tank on /tank type zfs (rw,relatime,xattr,noacl,casesensitive)


Code: 
INFO: starting new backup job: vzdump 101 --mode stop --quiet 1 --fleecing 0 --storage external-backup --notes-template '{{guestname}}' --compress zstd
INFO: Starting Backup of VM 101 (lxc)
INFO: Backup started at 2025-08-01 00:00:10
INFO: status = running
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: CT101
INFO: including mount point rootfs ('/') in backup
INFO: excluding bind mount point mp0 ('/tank') from backup (not a volume)
INFO: stopping virtual guest
INFO: creating vzdump archive '/mnt/externaldrv/dump/vzdump-lxc-101-2025_08_01-00_00_04.tar.zst'
INFO: tar: ./var/lib/docker/overlay2/f4aefbc629d80d8a00620cb4f9e376b4569d308d60c84b193e2a1badb8dfc715/diff/home/ubuntu: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/3786d5e330ebe849437b9bb6d87c53d8d7fc4a9a9d4eb9c0a995dcc542871b5c/diff/home/ubuntu: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-502: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-958: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-575: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-1198: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-1101: Cannot open: Permission denied
<snip ~3k more of these>
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-1036: Cannot open: Permission denied
INFO: tar: ./var/lib/docker/overlay2/384e30b351d9a0d86149ea39be9bab9bb639cd960b03b004255580b2791c5145/diff/home/user-1041: Cannot open: Permission denied
INFO: tar: ./home/media: Cannot open: Permission denied
INFO: Total bytes written: 121920552960 (114GiB, 91MiB/s)
INFO: tar: Exiting with failure status due to previous errors
INFO: restarting vm
INFO: guest is online again after 1298 seconds
ERROR: Backup of VM 101 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:1000 -m g:0:100000:1000 -m u:1000:1000:1 -m g:1000:1000:1 -m u:1001:101001:64530 -m g:1001:101001:64530 -m g:65534:165534:1 -- tar cpf - --totals --one-file-system -p --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-file-ignored' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/externaldrv/dump/vzdump-lxc-101-2025_08_01-00_00_04.tmp' ./etc/vzdump/pct.conf ./etc/vzdump/pct.fw '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | zstd '--threads=1' >/mnt/externaldrv/dump/vzdump-lxc-101-2025_08_01-00_00_04.tar.dat' failed: exit code 2
INFO: Failed at 2025-08-01 00:21:50
INFO: Backup job finished with errors
INFO: notified via target `mail-to-root`
TASK ERROR: job errors


Code: 
root@CT101:/var/lib/docker/overlay2# ls -al
total 3026
drwx--x--- 371 root root 371 Aug  1 04:23 .
drwx--x---  12 root root  14 Aug  1 04:23 ..
drwx--x---   4 root root   7 Feb  7  2024 004d866c20e1192b09e9562383c8529811732215ba81aa429deefa7569d3073f
drwx--x---   4 root root   7 Feb  7  2024 00a1521fa1f82ac4a8b7b0fbecbf07fd5e8fdaf943db10b43557f87b606d0593
drwx--x---   3 root root   5 Aug  1  2024 01288dea26efacb62eabf31a3c64d4c049a58143864c57b0a697c89fb2dfb14a
drwx--x---   4 root root   7 Jul 18  2024 015b4671b67403c89052cf011f124b4ffccaca4fdd8d279a4bfafdd177dd2075
drwx--x---   4 root root   7 Aug 19  2024 0239501851f11f76de2716f680c9cb8d236fa6c65924cb5962ad67ea4e4c4c7c
drwx--x---   4 root root   7 Aug  1  2024 025aaf24c50d08d0b51bed7fdf8f37750e5827e389a406c1ee6a6de4f9821b23
drwx--x---   4 root root   7 Feb  7  2024 02c236ec204e80b2d019380303dc518e7c84fb239addb14f9d005b458b449cd0
drwx--x---   4 root root   7 Aug  1  2024 03673431488b6aa38274bad54a99fa7e66c28392c29e96067ed58ff0cf4af5cc
drwx--x---   4 root root   7 May 30 23:44 04a03ad6a312ffa0757b678db722581a4f876d3db2cf53cfffe4408b6ad09e86
drwx--x---   4 root root   7 Jul 13  2024 05b2333a4a20cc1d2c6ed2aea185a783ecad54d517a0d27865be9d03ffec2b82
drwx--x---   4 root root   7 Aug 19  2024 0684c2e4e7c7a50e545b6a5508e20a00f363e8c791193f8c7266b6268d492eb6
drwx--x---   4 root root   7 Dec  2  2023 06c5dbe55b267d3de850524abb1dcf0168410c92bf7b57ee0babe6403c83782a
drwx--x---   4 root root   7 Jul 13  2024 07402aa8ba8e319559f016d6c4d96c651a76cb8b7b88c59e3202ebf517d1e920
<snip>
drwx--x---   4 root root   7 Sep  2  2024 fe991954dfc9237702ef0ecf016a0add99410c576dd9bc274343234daeafd338
drwx--x---   3 root root   5 Nov 29  2023 ffe95e76aa82eecb782feb039943c005b6486d82f328dfffacd652490097cbcf
drwx------   2 root root 370 Jun 26 06:39 l


./home/media seems to be another problem:

Code: 
root@CT101:/home# ls -al
total 19
drwxr-xr-x  5 root      root       5 Jan 17  2025 .
drwxr-xr-x 20 root      root      25 Aug  1 04:21 ..
drwxr-x---  2 nobody    nogroup    5 Nov 27  2023 media
drwxr-xr-x  4 root      root       4 Jan 17  2025 pihole
drwxr-x---  6 powerwall powerwall 13 Aug 28  2024 powerwall

Code: 
root@CT101:/home# cat /etc/passwd | grep media
media:x:1000:1000:,,,:/home/media:/bin/bash
 
Last edited:
What do you mean "docker setup"? Any debugging steps I should take?

Edit: I think I get what you're saying. You would've been running docker straight on bare proxmox rather than a CT?

Ironically, I organized it this way to ease backups...
 
Last edited:
catzilla said:
So what is your recommendation? Convert the CT to a VM?
Click to expand...
I don't think you can really convert them. The Conversion, if any, might be done the other Way around (VM -> CT), but also in that Case it's probably easier to just install from Scratch and restore/migrate from Backup.

You need to DEPLOY a new VM using your preferred Linux Distro (e.g. Debian Bookworm).

I installed several of them using the ISO Image, then keep them around as "Templates" (do NOT convert to a real Template, I just call them vm-template-debian-bookworm-amd-64 etc so I can still update them, install additional Packages, etc).

I suggest you partition generously for the Root "/" Disk and use EXT4 for Root (I'd go with 64GiB nowadays), then for Docker (or Podman on Fedora, in my Case ;)) you just add another Virtual Disk (~ 1TB) and just format that as EXT4 (or XFS I guess, if you are confortable with that, no Experience with XFS lately). Of course I am assuming that you have some sort of RAID under the Hood on the Host probably with ZFS :) .

I prefer to standardize the Partition Layout for the Root "/" Disk as such with GPL Label:
- BIOS GRUB
- EFI on FAT32
- BOOT on EXT4
- ROOT on EXT4

(for a VM you might want to get rid of the separate BOOT, but that's my standard Partition Scheme I also use for the Host - with BOOT on mdadm RAID-1 though, since I encrypt ROOT using LUKS, it's made to be compatible to both UEFI and BIOS/CMS Boot)
 
  • Like
Reactions: catzilla
You must log in or register to reply here.
Top Bottom