Immich LXC fails to backup

S

Skipping1895

New Member
Jul 14, 2024
4
0
1
Proxmox Virtual Environment 8.2.7

I am trying to do a backup of my Immich lxc. But it fails all the time.
I do it with "stop mode" since snapshot is not supported because the Immich data is stored on NAS via NFS.
Why does it fail and what to do to correct it?

Code: 
INFO: starting new backup job: vzdump 113 --fleecing 0 --storage backups --quiet 1 --notes-template '{{guestname}}' --mode stop --prune-backups 'keep-last=1,keep-monthly=1,keep-weekly=1'
INFO: Starting Backup of VM 113 (lxc)
INFO: Backup started at 2024-12-06 01:00:05
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 7
INFO: CT Name: immich
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/root/immich-storage') in backup
INFO: creating Proxmox Backup Server archive 'ct/113/2024-12-06T00:00:05Z'
INFO: set max number of entries in memory for file-based backups to 1048576
INFO: run: lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup --crypt-mode=none pct.conf:/var/tmp/vzdumptmp1336341_113/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --include-dev /mnt/vzsnap0/./root/immich-storage --skip-lost-and-found --exclude=/tmp/?* --exclude=/var/tmp/?* --exclude=/var/run/?*.pid --backup-type ct --backup-id 113 --backup-time 1733443205 --entries-max 1048576 --repository backupUser@pbs@192.168.1.109:backups --ns backups
INFO: Starting backup: [backups]:ct/113/2024-12-06T00:00:05Z
INFO: Client name: pve
INFO: Starting backup protocol: Fri Dec  6 09:45:13 2024
INFO: Downloading previous manifest (Fri Oct 11 21:55:09 2024)
INFO: Upload config file '/var/tmp/vzdumptmp1336341_113/etc/vzdump/pct.conf' to 'backupUser@pbs@192.168.1.109:8007:backups' as pct.conf.blob
INFO: Upload directory '/mnt/vzsnap0' to 'backupUser@pbs@192.168.1.109:8007:backups' as root.pxar.didx
INFO: catalog upload error - stream error received: stream no longer needed
INFO: unclosed encoder dropped
INFO: closed encoder dropped with state
INFO: unfinished encoder state dropped
INFO: Error downloading .didx from previous manifest: stream closed because of a broken pipe
INFO: Error: connection closed because of a broken pipe
ERROR: Backup of VM 113 failed - command 'lxc-usernsexec -m u:0:100000:65536 -m g:0:100000:65536 -- /usr/bin/proxmox-backup-client backup '--crypt-mode=none' pct.conf:/var/tmp/vzdumptmp1336341_113/etc/vzdump/pct.conf root.pxar:/mnt/vzsnap0 --include-dev /mnt/vzsnap0/./ --include-dev /mnt/vzsnap0/./root/immich-storage --skip-lost-and-found '--exclude=/tmp/?*' '--exclude=/var/tmp/?*' '--exclude=/var/run/?*.pid' --backup-type ct --backup-id 113 --backup-time 1733443205 --entries-max 1048576 --repository backupUser@pbs@192.168.1.109:backups --ns backups' failed: exit code 255
INFO: Failed at 2024-12-06 09:45:16
INFO: Backup job finished with errors
TASK ERROR: job errors
 
If I'm not wrong, it looks like the previous backup was about 2 months ago. Did you verify this backup? Do other backups in the same datastore succeed? Did you check both filesystems?
 
Hello,
Mp0 mounting way may be the problem.
- Shutdown your LXC.
- Edit /etc/pve/lxc/113.conf and change the line mp0: /xxx,mp=/yyy by lxc.mount.entry: /xxx /yyy none bind 0 0
Replace xxx and yyy by yours paths.
- Start the LXC and test the proper functioning.
- If OK, shutdown again your LXC and test to backup it.
 
Al_mageddon said:
- Edit /etc/pve/lxc/113.conf and change the line mp0: /xxx,mp=/yyy by lxc.mount.entry: /xxx /yyy none bind 0 0
Click to expand...
Sorry for changing thr issue (though may be related to the solution): are there any plans to support bind mounts via UI instead of having to edit manually the conf file?
 
omgs said:
Sorry for changing thr issue (though may be related to the solution): are there any plans to support bind mounts via UI instead of having to edit manually the conf file?
Click to expand...
You got it working?

I'm on PVE 7, and I don't have the line mp0... weird.
 
The syntax starting with <code>mpX</code> is for mount points. The <code>lxc.mount.entry</code> and the <code>mpX: /..... </code> are equivalent.
 
Hi,
omgs said:
The syntax starting with <code>mpX</code> is for mount points. The <code>lxc.mount.entry</code> and the <code>mpX: /..... </code> are equivalent.
Click to expand...
please note that Proxmox VE will not know about the mount point if you use lxc.mount.entry and thus ignores it for certain operations. If you don't have a specific reason, use mpX keys for managing mount points. You can set them via e.g. pct set 111 -mp2 /root/example,mp=/foo on the commandline.
 
fiona said:
Hi,

please note that Proxmox VE will not know about the mount point if you use lxc.mount.entry and thus ignores it for certain operations. If you don't have a specific reason, use mpX keys for managing mount points. You can set them via e.g. pct set 111 -mp2 /root/example,mp=/foo on the commandline.
Click to expand...
I never touch the config files in PVE 7.

I’m on the last PVE 7.

Let me see if I can figure out how to change it to mpX
 
  • Like
Reactions: charleslcso
fiona said:
You cannot add bind mounts via UI, but you can use the CLI command pct set like I wrote.

P.S. Proxmox VE 7 is end of life since July, see
https://pve.proxmox.com/wiki/FAQ
https://pve.proxmox.com/wiki/Upgrade_from_7_to_8
Click to expand...
Hi Fiona,

Allow me to play safe by confirming everything is correct.

The container having issues in PBS backup has ID of 123.

It has 3 mount points, all are from another server. All mount points stay connected 24/7.

I need to issue the command you've given me as below in the PVE host, not within the container 123:

* pct set 123 -mp2 root@pbs.local:/mnt/remote/1,mp=/mnt/local/1
* pct set 123 -mp2 root@pbs.local:/mnt/remote/2,mp=/mnt/local/2
* pct set 123 -mp2 root@pbs.local:/mnt/remote/3,mp=/mnt/local/3

First of all, do I need to umount the above first?

And is the syntax correct?
 
charleslcso said:
Hi Fiona,

Allow me to play safe by confirming everything is correct.

The container having issues in PBS backup has ID of 123.

It has 3 mount points, all are from another server. All mount points stay connected 24/7.

I need to issue the command you've given me as below in the PVE host, not within the container 123:

* pct set 123 -mp2 root@pbs.local:/mnt/remote/1,mp=/mnt/local/1
* pct set 123 -mp2 root@pbs.local:/mnt/remote/2,mp=/mnt/local/2
* pct set 123 -mp2 root@pbs.local:/mnt/remote/3,mp=/mnt/local/3

First of all, do I need to umount the above first?

And is the syntax correct?
Click to expand...
You can only bind-mount local paths with this. It's not possible to specify a different server here. So you need to first mount your remote file system on the host and then you can bind mount with the path on the host into the container. Also you are using mp2 always, so you will just overwrite that setting.
 
  • Like
Reactions: Johannes S
fiona said:
You can only bind-mount local paths with this. It's not possible to specify a different server here. So you need to first mount your remote file system on the host and then you can bind mount with the path on the host into the container. Also you are using mp2 always, so you will just overwrite that setting.
Click to expand...
Got ya!

I got bind-mount working using new mpN.

Now I realise the ownership becomes nobody:nogroup from within the container. This might be an issue.

Also, what's the best way to make sure the bind-mounts are always mounted (healthy) inside the container?
 
charleslcso said:
Now I realise the ownership becomes nobody:nogroup from within the container. This might be an issue.
Click to expand...
See:
fiona

Post in thread 'Permissions with migrated LVM volume'

Hi,
while the container is down, you can use pct mount <ID> to mount it in /var/lib/lxc/<ID>/rootfs on the host. Then you can go there, check the current owner. Is it 1000 on the host? To unmount it, use pct unmount <ID>.

Now, you can either:
  • chown to an ID in the container's range starting from 100000, i.e. add the user ID for the user in the container that should own the files.
  • Make the host user ID that is currently owning the files available in the container via mapping (assuming it's 1000 on the host)
    Code: 
    lxc.idmap: u 0 100000 1000
lxc.idmap: g 0 100000...

See: https://pve.proxmox.com/wiki/Unprivileged_LXC_containers
charleslcso said:
Also, what's the best way to make sure the bind-mounts are always mounted (healthy) inside the container?
Click to expand...
I think startup will fail if they can't be bind-mounted. Of course you need to ensure that the shares are available network-wise/on the host.
 
  • Like
Reactions: charleslcso
A little trick I use to manage situations mounting unprivileged containers on the host, I create a host user (ang group) called lxc-root with id 100000. The same principle stands for lxc-www-data. From within the guest, sometimes you can't chown files inside the container because of the uid, and have to do this from the host, using these "lxc-*" users to avoid errors.
 
You must log in or register to reply here.
Top Bottom