Backup Client Fails with EINVAL: Invalid argument

[MeCJay12]

I've moved all my VM backups to PBS but now that I'm trying to move my file backups to PBS I'm having trouble. My backup client runs successfully periodically but also likes to fail with the vey generic "EINVAL: Invalid argument" error and a file. The file that it errors on will change from run to run. Any idea what's going on here?

Starting backup: [files]:host/Crontab/2024-10-24T13:07:58Z
Client name: Crontab
Starting backup protocol: Thu Oct 24 09:07:58 2024
Downloading previous manifest (Mon Oct 14 20:43:51 2024)
Upload directory '/data/' to 'backup@pbs@pbsLabB:8007:backup' as data.pxar.didx
unclosed encoder dropped
closed encoder dropped with state
unfinished encoder state dropped
catalog upload error - channel closed
Error: error at "TV Shows/Young Sheldon/Season 05/Young Sheldon - S05E01 - One Bad Night and Chaos of Selfish Desires.mkv"

Caused by:
    EINVAL: Invalid argument

 

[Chris]

Hi,
please provide some more information in order to assist.

The output of proxmox-backup-manager version --verbose and the proxmox-backup-client version, as well as the task log for the backup job from the PBS server are of interest. Also, the exact command line used for invocation of the backup job in you cron job.

Further, do check the systemd journal for errors by dumping the journal for the timespan around the failing backup job, you can achieve this by journactl --since <DATETIME> --until <DATETIME> > journal.txt and add it as attachment.

Further, what underlying filesystem is used for the target datastore? I suspect this to be rather a server side storage issue, EINVAL errors are typically encountered when the underlying datastore storage is not supported, e.g. [0,1]

[0] https://forum.proxmox.com/threads/virtualized-pbs-with-shared-mount.100455/
[1] https://forum.proxmox.com/threads/cant-start-backup-process-on-pbs.154623/

 

[MeCJay12]

My PBS is a VM running on a PVE server. The backup storage is an ext4 formatted VirtIO SCSI virtual disk. I recognize that running PBS in a VM isn't recommended but the footprint of three new servers and disks to do a full 3-2-1 backup is cost prohibitive.

Proxmox Backup Server Version

user@pbsLabB:~$ sudo proxmox-backup-manager version --verbose
[sudo] password for user:
proxmox-backup                    3.2.0        running kernel: 6.8.8-2-pve
proxmox-backup-server             3.2.7-1      running version: 3.2.6
proxmox-kernel-helper             8.1.0
proxmox-kernel-6.8                6.8.12-2
proxmox-kernel-6.8.8-2-pve-signed 6.8.8-2
proxmox-kernel-6.8.4-3-pve-signed 6.8.4-3
proxmox-kernel-6.8.4-2-pve-signed 6.8.4-2
ifupdown2                         3.2.0-1+pmx9
libjs-extjs                       7.0.0-4
proxmox-backup-docs               3.2.7-1
proxmox-backup-client             3.2.7-1
proxmox-mail-forward              0.2.3
proxmox-mini-journalreader        1.4.0
proxmox-offline-mirror-helper     0.6.7
proxmox-widget-toolkit            4.2.3
pve-xtermjs                       5.3.0-3
smartmontools                     7.3-pve1
zfsutils-linux                    2.2.6-pve1

Proxmox Backup Client Version & Script run to backup

root@Crontab:/# proxmox-backup-client version
client version: 3.2.7
root@Crontab:/# cat /scripts/pbcFilesBackup.sh
#!/bin/bash

export PBS_REPOSITORY="backup@pbs@pbsLabB:backup"
export PBS_PASSWORD='password'
export XDG_RUNTIME_DIR=/run/user/1000

proxmox-backup-client backup data.pxar:/data/ veeam.pxar:/Veeam/ --ns files

Full journalctl but the only thing unusual I found was that it exited with the following

Oct 24 21:51:48 pbsLabB proxmox-backup-proxy[606]: backup ended and finish failed: backup ended but finished flag is not set.
Oct 24 21:51:48 pbsLabB proxmox-backup-proxy[606]: removing unfinished backup
Oct 24 21:51:48 pbsLabB proxmox-backup-proxy[606]: removing backup snapshot "/mnt/disk/ns/files/host/Crontab/2024-10-24T13:07:58Z"
Oct 24 21:51:49 pbsLabB proxmox-backup-proxy[606]: TASK ERROR: backup ended but finished flag is not set.

 

[Chris]

What filesystem is used at the sources, /data and /Veeam? Further, please do provide the systemd journal by some other means, which does not require me to login via a google account, thank you!

 

[MeCJay12]

Sorry, fixed permission on the link. You can open it without a Google account now.

/data and /Veeam are CIFS shares mounted in Ubuntu then bind mounted into the Docker container that runs PBC.

 

[Chris]

Sorry, fixed permission on the link. You can open it without a Google account now.

Thanks!

/data and /Veeam are CIFS shares mounted in Ubuntu then bind mounted into the Docker container that runs PBC.

What docker image and parameters are you using and what are the mount parameters for the CIFS shares? Latter you can get from the output of mount executed on Ubuntu. Also check the syslog on your Ubuntu host, container and CIFS share host, I do not expect anything of interest to be there, but it is worth to check nevertheless.