[SOLVED] Permission denied (os error 13) backup connect failed: command error: unable to create backup group

[Jarvar]

Please help.
I used to be able to make backups to this particular PBS server.
I had to create a new VM and make backups.
This is the error I am getting when I try to do that.

ERROR: VM 108 qmp command 'backup' failed - backup connect failed: command error: unable to create backup group "/usbpool002/dataset002/store001/ns/csd-mis-001/vm/108" - Permission denied (os error 13)

Any help would be much appreciated.
Thank you!

 

[Chris]

Hi,
please verify that the datastore backing path on the proxmox backup server has the correct owner and group. Both should be backup. Did you recently change the zpool backing this datastore?

 

[Jarvar]

Hi,
please verify that the datastore backing path on the proxmox backup server has the correct owner and group. Both should be backup. Did you recently change the zpool backing this datastore?

Hello Chris. I don’t think I’ve ever changed the defaults on the system. How do I check that the owner and group is backup? Is root default? Should have all been created and setup with root@pam
It was working and just stopped.
I didn’t change the zpool backing.

 

[Chris]

Hello Chris. I don’t think I’ve ever changed the defaults on the system. How do I check that the owner and group is backup? Is root default? Should have all been created and setup with root@pam

What I meant was on the filesystem, not the owner of the PBS user with which the backups were created. Check via ls -l /usbpool002/dataset002/store001 and eventually also the further path components of the path in the error.

Edit: According to the code, this exact error should happen when the final comonent of the path fails to be created, so check the permissions of ls -l /usbpool002/dataset002/store001/ns/csd-mis-001/vm

 

[Jarvar]

What I meant was on the filesystem, not the owner of the PBS user with which the backups were created. Check via ls -l /usbpool002/dataset002/store001 and eventually also the further path components of the path in the error.

Edit: According to the code, this exact error should happen when the final comonent of the path fails to be created, so check the permissions of ls -l /usbpool002/dataset002/store001/ns/csd-mis-001/vm

Thank you for clarifying.
This is the output
ls -l /usbpool002/dataset002/store001/ns/csd-mis-001/vm total 9 drwxr-xr-x 26 backup backup 27 Oct 10 22:30 107

 

[Jarvar]

Wondering if this would fail also if there is a Garbage collection and Verify job running when trying to backup?
I wouldn't think so since sometimes those tasks can take a long time depending on the type of storage used.

 

[Chris]

Wondering if this would fail also if there is a Garbage collection and Verify job running when trying to backup?
I wouldn't think so since sometimes those tasks can take a long time depending on the type of storage used.

No,
locks are in place to guarantee consistency in case of other concurrent tasks taking place. This error stems most definetly from some permission issue while creating the backup group directory.

ls -l /usbpool002/dataset002/store001/ns/csd-mis-001/vm total 9 drwxr-xr-x 26 backup backup 27 Oct 10 22:30 107

Sorry, while typing the command I lost the -a flag, add it so that also the ownership and permissions of the directory itself an the parent directory are shown, ls -la /usbpool002/dataset002/store001/ns/csd-mis-001/vm.

Also check if the proxmox-backup-proxy runs as backup user via ps aux | grep proxmox-backup

 

[Jarvar]

No,
locks are in place to guarantee consistency in case of other concurrent tasks taking place. This error stems most definetly from some permission issue while creating the backup group directory.


Sorry, while typing the command I lost the -a flag, add it so that also the ownership and permissions of the directory itself an the parent directory are shown, ls -la /usbpool002/dataset002/store001/ns/csd-mis-001/vm.

Also check if the proxmox-backup-proxy runs as backup user via ps aux | grep proxmox-backup

ls -la /usbpool002/dataset002/store001/ns/csd-mis-001/vm total 10 drwxr-xr-x 3 root root 3 Apr 4 2023 . drwxr-xr-x 3 backup backup 3 Apr 4 2023 .. drwxr-xr-x 26 backup backup 27 Oct 10 22:30 107


And the results of the ps aux | grep proxmox-backup:

ps aux | grep proxmox-backup root 860 2.1 0.2 378300 20320 ? Ssl Oct05 205:50 /usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-api backup 926 1.4 0.8 2164212 72496 ? Ssl Oct05 133:02 /usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-proxy root 3912445 0.0 0.0 6372 644 pts/0 S+ 10:12 0:00 grep proxmox-backup


Thank you so much

 

[Chris]

drwxr-xr-x 3 root root 3 Apr 4 2023 .

as you can see, the folder /usbpool002/dataset002/store001/ns/csd-mis-001/vm is indeed owned by user and group root instead of backup. Perform a

chown backup:backup /usbpool002/dataset002/store001/ns/csd-mis-001/vm

and you should be fine.

This however was most definetly a manual change/intervention from someone on your side, Proxmox Backup Server services will not write these directories as root user.

 

[Jarvar]

as you can see, the folder /usbpool002/dataset002/store001/ns/csd-mis-001/vm is indeed owned by user and group root instead of backup. Perform a

chown backup:backup /usbpool002/dataset002/store001/ns/csd-mis-001/vm

and you should be fine.

This however was most definetly a manual change/intervention from someone on your side, Proxmox Backup Server services will not write these directories as root user.

Thank you. I tried as you recommended and will attempt a backup when office hours are over to limit any hiccups.
Thank you very much for assisting me here.

 

[Jarvar]

@

as you can see, the folder /usbpool002/dataset002/store001/ns/csd-mis-001/vm is indeed owned by user and group root instead of backup. Perform a

chown backup:backup /usbpool002/dataset002/store001/ns/csd-mis-001/vm

and you should be fine.

This however was most definetly a manual change/intervention from someone on your side, Proxmox Backup Server services will not write these directories as root user.

I mess something up, I tried to create a container instead and bindmount the zpool. It takes a lot less resources but messes with permissions.
https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
I followed that and now I'm having some issues again. I'm trying to convert everything back.
Could you help please?

It would be great to get a container with PBS running for a lighter footprint though.

ls -la /usbpool002/dataset021/store021/ns/csd-mis-003/vm total 68 drwxr-xr-x 8 backup backup 8 Oct 10 13:17 . drwxr-xr-x 3 backup backup 3 Jun 3 07:57 .. drwxr-xr-x 29 1005 1005 30 Oct 17 22:00 107 drwxr-xr-x 9 1005 1005 10 Oct 18 05:00 108 drwxr-xr-x 4 1005 1005 5 Oct 16 18:33 109 drwxr-xr-x 5 1005 1005 6 Oct 16 15:34 201 drwxr-xr-x 3 1005 1005 4 Jun 3 13:07 204 drwxr-xr-x 3 1005 1005 4 Jun 3 13:08 401

 

[Chris]

@

I mess something up, I tried to create a container instead and bindmount the zpool. It takes a lot less resources but messes with permissions.
https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
I followed that and now I'm having some issues again. I'm trying to convert everything back.
Could you help please?

It would be great to get a container with PBS running for a lighter footprint though.

ls -la /usbpool002/dataset021/store021/ns/csd-mis-003/vm total 68 drwxr-xr-x 8 backup backup 8 Oct 10 13:17 . drwxr-xr-x 3 backup backup 3 Jun 3 07:57 .. drwxr-xr-x 29 1005 1005 30 Oct 17 22:00 107 drwxr-xr-x 9 1005 1005 10 Oct 18 05:00 108 drwxr-xr-x 4 1005 1005 5 Oct 16 18:33 109 drwxr-xr-x 5 1005 1005 6 Oct 16 15:34 201 drwxr-xr-x 3 1005 1005 4 Jun 3 13:07 204 drwxr-xr-x 3 1005 1005 4 Jun 3 13:08 401

Well if the owner/group of the files and folder in the datastore were initially owned by the user backup on the host, they will now have to be owned by the same user backup inside the container. So on the host, the ownership of these has be adapted according to the uid/gid mapping, which by default is a positive shift by 100000. I assume you did not remap the ownership accordingly.

Please share the configuration of your LXC by running pct conf <VMID> --current as well as the output of id backup, the latter executed inside the container.

 

[Jarvar]

Well if the owner/group of the files and folder in the datastore were initially owned by the user backup on the host, they will now have to be owned by the same user backup inside the container. So on the host, the ownership of these has be adapted according to the uid/gid mapping, which by default is a positive shift by 100000. I assume you did not remap the ownership accordingly.

Please share the configuration of your LXC by running pct conf <VMID> --current as well as the output of id backup, the latter executed inside the container.

Good morning @Chris
I started a new thread instead of continuing on this one unless it's related.
https://forum.proxmox.com/threads/r...n-external-drive-with-zpool-from-host.135173/
I appreciate your help.
Thank you