Backup unprivileged containers to NFS storage?

Zyg0te

New Member
Mar 13, 2016
12
2
3
32
I have a NAS I use for Proxmox backups. However, backing up unprivileged containers to NFS storage fails immediately with a "Cannot open: Permission denied" error.

Whats the solution here? The Wiki mentions bind mounts points, but it's not clear to me how this can be used to allow the unprivilieged container to be backed up to an Proxmox NFS storage point.

Thanks
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
are you backing the container up using vzdump, or are you using some kind of backup software inside the container?
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
could you post the following:
  • pveversion -v
  • container configuration (/etc/pve/lxc/ID.conf)
  • storage configuration (/etc/pve/storage.cfg)
  • vzdump configuration (/etc/vzdump.conf)
  • backup log of such a failed backup
 

Zyg0te

New Member
Mar 13, 2016
12
2
3
32
Solved. It was related to NFS share permissions. Unpriviliged containers dont map to uid0 and only the root user has write access to the NFS share.
 
Last edited:
  • Like
Reactions: notaduck

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
Solved. It was related to NFS share permissions. Unpriviliged containers dont map to uid0 and only the root user has write access to the NFS share.

I think you should not run into this issue unless you have your tmpdir on NFS (which is a bad idea) and use suspend backups.
 

notaduck

Member
Nov 13, 2016
5
0
6
29
Solved. It was related to NFS share permissions. Unpriviliged containers dont map to uid0 and only the root user has write access to the NFS share.

Could you explain your solution ?! I'm standing in the exact same situation as you
 
Aug 15, 2016
27
0
1
40
Any news regarding this issue? I have the same problem whenever I
a) try to create an unprivileged container from a template stored on a NFS share (!the template is located on the NFS share!) or
b) run the (proxmox webgui) backup and try to store an unprivileged container on a NFS

Extract from log regarding
a)
extracting archive '/mnt/pve/pve_templates/template/cache/ubuntu-16.10-standard_16.10-1_amd64.tar.gz'
tar: /mnt/pve/pve_templates/template/cache/ubuntu-16.10-standard_16.10-1_amd64.tar.gz: Cannot open: Permission denied
tar: Error is not recoverable: exiting now

b)
INFO: creating archive '/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tar.lzo'
INFO: tar: /mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tmp: Cannot open: Permission denied
INFO: tar: Error is not recoverable: exiting now
ERROR: Backup of VM 101 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tmp' ./etc/vzdump/pct.conf '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | lzop >/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tar.dat' failed: exit code 2
INFO: Backup job finished with errors
TASK ERROR: job errors


Maybe changing permissions could solve this, but if I understood Fabian's comment ("I think you should not run into this issue unless...") right, I maybe do not have to extend my (standard) permissions?

Greets,
SImon
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
Any news regarding this issue? I have the same problem whenever I
a) try to create an unprivileged container from a template stored on a NFS share (!the template is located on the NFS share!) or
b) run the (proxmox webgui) backup and try to store an unprivileged container on a NFS

Extract from log regarding
a)
extracting archive '/mnt/pve/pve_templates/template/cache/ubuntu-16.10-standard_16.10-1_amd64.tar.gz'
tar: /mnt/pve/pve_templates/template/cache/ubuntu-16.10-standard_16.10-1_amd64.tar.gz: Cannot open: Permission denied
tar: Error is not recoverable: exiting now

this is expected - extraction happens as unprivileged user, so the unprivileged user needs at least read access to the archive.

b)
INFO: creating archive '/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tar.lzo'
INFO: tar: /mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tmp: Cannot open: Permission denied
INFO: tar: Error is not recoverable: exiting now
ERROR: Backup of VM 101 failed - command 'set -o pipefail && lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar cpf - --totals --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-xattr-write' --one-file-system '--warning=no-file-ignored' '--directory=/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tmp' ./etc/vzdump/pct.conf '--directory=/mnt/vzsnap0' --no-anchored '--exclude=lost+found' --anchored '--exclude=./tmp/?*' '--exclude=./var/tmp/?*' '--exclude=./var/run/?*.pid' ./ | lzop >/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tar.dat' failed: exit code 2
INFO: Backup job finished with errors
TASK ERROR: job errors

Maybe changing permissions could solve this, but if I understood Fabian's comment ("I think you should not run into this issue unless...") right, I maybe do not have to extend my (standard) permissions?

Like I said, unless your tmpdir is on NFS and not accessable for the unprivileged user, you should be fine.

Code:
INFO: tar: /mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-101-2017_04_07-16_40_42.tmp: Cannot open: Permission denied
indicates your tmpdir is on NFS (because you haven't configured one explicitly in /etc/vzdump.conf).
 
  • Like
Reactions: user843
Aug 15, 2016
27
0
1
40
Hi Fabian,

thank you very much for your reply. Now I edited /etc/vzdump.conf, so that tmpdir points to /var/tmp and I am able to do backups.

But restoring the backup as unprivileged container is nor possible:
extracting archive '/mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-200-2017_05_01-15_07_04.tar.lzo'
tar: /mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-200-2017_05_01-15_07_04.tar.lzo: Cannot open: Permission denied
tar: Error is not recoverable: exiting now
TASK ERROR: command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- tar xpf /mnt/pve/BoSiX1515-NFS/dump/vzdump-lxc-200-2017_05_01-15_07_04.tar.lzo --totals --sparse --numeric-owner --acls --xattrs '--xattrs-include=user.*' '--xattrs-include=security.capability' '--warning=no-xattr-write' -C /var/lib/lxc/200/rootfs --skip-old-files --anchored --exclude './dev/*'' failed: exit code 2


I assume, this is again because of the unprivileged user, right? Can you please tell me, how I can create a NFS share where this user is able to read/write? Is this a setting I have to configure on NFS Server side?

Thank you in advance,
Simon
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
yes, the unprivileged user with id 65536 needs to be able to read the backup archive.
 
Aug 15, 2016
27
0
1
40
Hi Fabian,

thank you again. I can understand, that the user needs access to the restored file, once it is restored. But I do not understand, why I have to give an unprivileged user rights for a restore process?

That's why I prefer to doublecheck here, before I extend user rights on my NFS.

Greets,
Simon
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
Hi Fabian,

thank you again. I can understand, that the user needs access to the restored file, once it is restored. But I do not understand, why I have to give an unprivileged user rights for a restore process?

That's why I prefer to doublecheck here, before I extend user rights on my NFS.

Greets,
Simon

because the extraction happens as unprivileged user - so it needs access to the archive that will be extracted ;) you can also move the backup from NFS to somewhere else first if you are concerned with giving that user permissions on your NFS server, or you can use the CLI and pipe the archive into pct restore, passing "-" as archive path so that it reads from stdin.
 
Aug 15, 2016
27
0
1
40
Hi Fabian,

OK, I think this is a way, I can use it. Thank you :)

Are there any plans to change this behaviour, so that a privileged user can also extract and does a chown (or whatever) after extraction?

Greets,
Simon
 

fabian

Proxmox Staff Member
Staff member
Jan 7, 2016
4,652
735
133
Hi Fabian,

OK, I think this is a way, I can use it. Thank you :)

Are there any plans to change this behaviour, so that a privileged user can also extract and does a chown (or whatever) after extraction?

Greets,
Simon

chowning is not really feasible.. but what should be possible is to read/open the archive as actual root, and pass it to the extraction process running with mapped users. maybe you could file an entry over at https://bugzilla.proxmox.com for this?
 

Pavel Šajnar

New Member
Jun 6, 2019
1
0
1
41
In my setup I backup to Synology NAS. On NAS NFS settings when I change permissions (Squash) mapping to: Map all users to admin, it then works.
 

Tyrone Johnson

New Member
Apr 8, 2019
1
0
1
60
Hello, sorry but I need to open this old thread again, but I'm still having this same problem for containers. I've changed the permissions mapping as noted above, but no luck. The backup stops due to lacking priveledge. Is there a workaround somehow? thanks
 

starkruzr

Active Member
I may be having a related issue. I backed up containers from a previous cluster and in the current cluster UIDs and GIDs are different. When I try to restore to a Ceph RBD pool, I get this:

Code:
extracting archive '/mnt/pve/aquarii-rtechvol/dump/vzdump-lxc-121-2020_03_22-22_59_35.tar.gz'
tar: ./var/spool/postfix/dev/random: Cannot mknod: Operation not permitted
tar: ./var/spool/postfix/dev/urandom: Cannot mknod: Operation not permitted
tar: ./home/RTECH.RTI/jtd/.zsh_history: Cannot change ownership to uid 600201106, gid 600200513: Invalid argument
tar: ./home/RTECH.RTI/jtd/.local/share/nano: Cannot change ownership to uid 600201106, gid 600200513: Invalid argument

And a bunch more of those tar errors. I can't remember if the container was privileged or unprivileged, maybe the config file will help someone figure it out:

Code:
arch: amd64
cores: 1
hostname: Alexandria
memory: 2048
nameserver: 192.168.9.20 192.168.9.21
net0: name=eth0,bridge=vmbr0,gw=192.168.9.1,hwaddr=4E:26:E2:39:F6:4A,ip=192.168.9.35/24,ip6=auto,type=veth
onboot: 1
ostype: ubuntu
rootfs: aquarii-rtechvol:121/vm-121-disk-0.raw,size=60G
searchdomain: rtech.rti
swap: 2048

aquarii-rtechvol is an NFS mount. Not sure how to get around any of this.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE and Proxmox Mail Gateway. We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!