vzdump --exclude-path syntax command-line

[Riesling.Dry]

Hi there,
I want to backup a VE/CT to an external HD.
Just the core system, OS (Deb11) w. basic setting, but without the data (plex media lib w. ~10TB)
After reading wiki, I started the dump via command-line and get:

INFO: starting new backup job: vzdump 333 --ionice 1 --storage extern --exclude-path /opt/plexmedia/

As the vzdump-lxc-333-2024_05_27-12_04_23.dat file has grown in 3 hrs. > 1TB and is still growing, the backup still running, it seems, it does not consider the "--exclude-path /opt/plexmedia/"-option, hence my question:

Is the syntax correct?

Thanks for your feedback,
~R.

 

[fiona]

Hi,
AFAIK, that should work. Can you please share the full backup task log and output of pveversion -v and pct config 333? If the backup has finished since then, you can check with tar --list -f /path/to/your/backup.tar | grep plexmedia if the files have been excluded.

 

[Riesling.Dry]

Hi Fiona,
thanks for your reply.
it should, but it obviously doesn't, as the backup ist still running and has meanwhile grown to 5.3T:

167837705 5.3T -rw-r--r-- 1 root 5.3T May 28 13:39 vzdump-lxc-333-2024_05_27-12_04_23.dat

Soon, when it will run out of disk-space, backup will fail and vanish - that is why I will stop it now.

Seems this one is not up-to-date:

Spoiler: pveversion -v

pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.143-1-pve)
pve-manager: 7.4-17 (running version: 7.4-17/513c62be)
pve-kernel-5.15: 7.4-11
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.15.143-1-pve: 5.15.143-1
pve-kernel-5.15.136-1-pve: 5.15.136-1
pve-kernel-5.15.131-1-pve: 5.15.131-2
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-1-pve: 5.11.22-2
ceph-fuse: 15.2.13-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4.3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-2
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.7
libpve-storage-perl: 7.4-3
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.6-1
proxmox-backup-file-restore: 2.4.6-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.2
proxmox-widget-toolkit: 3.7.3
pve-cluster: 7.3-3
pve-container: 4.4-6
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-4~bpo11+2
pve-firewall: 4.3-5
pve-firmware: 3.6-6
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-2
qemu-server: 7.4-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.14-pve1

Spoiler: pct config 333

pct config 333
arch: amd64
cores: 8
features: fuse=1,nesting=1
hostname: jellyfin
lock: backup
memory: 12288
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.0.0.1,hwaddr=0A:7B:60:3A:74:E8,ip=10.3.3.3/24,type=veth
onboot: 0
ostype: debian
parent: vzdump
rootfs: local-zfs:subvol-333-disk-0,size=20000G
swap: 2048
unprivileged: 1

How else could I move the 10TB VE to a new host and what would be the fastest option?
I treid PBS, but it fails on restore...
That is why I wanted to backup only the working System (already treid to rebuild from scratch, but that didnt work out) and later rsync or scp the media (which lasted like a week over LAN - thats why wanted to try to backup to ext. HD, move the ext. HD to new node and restore, but I run out of disk-space)...

Regards,
~R.

 

[gfngfn256]

--exclude-path /opt/plexmedia/

You don't provide the actual command you entered for the backup & its excludes.

But maybe try with one of the following:

--exclude-path '/opt/plexmedia/'
--exclude-path /opt/plexmedia
--exclude-path '/opt/plexmedia'
--exclude-path 'opt/plexmedia'
--exclude-path opt/plexmedia

You may get lucky.

 

[fiona]

Hi Fiona,
thanks for your reply.
it should, but it obviously doesn't, as the backup ist still running and has meanwhile grown to 5.3T:

167837705 5.3T -rw-r--r-- 1 root 5.3T May 28 13:39 vzdump-lxc-333-2024_05_27-12_04_23.dat

Soon, when it will run out of disk-space, backup will fail and vanish - that is why I will stop it now.

Seems this one is not up-to-date:

Spoiler: pveversion -v

pveversion -v
proxmox-ve: 7.4-1 (running kernel: 5.15.143-1-pve)
pve-manager: 7.4-17 (running version: 7.4-17/513c62be)
pve-kernel-5.15: 7.4-11
pve-kernel-5.13: 7.1-9
pve-kernel-5.11: 7.0-10
pve-kernel-5.15.143-1-pve: 5.15.143-1
pve-kernel-5.15.136-1-pve: 5.15.136-1
pve-kernel-5.15.131-1-pve: 5.15.131-2
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.11.22-7-pve: 5.11.22-12
pve-kernel-5.11.22-1-pve: 5.11.22-2
ceph-fuse: 15.2.13-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx4
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4.3
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.4-2
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-3
libpve-rs-perl: 0.7.7
libpve-storage-perl: 7.4-3
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.6-1
proxmox-backup-file-restore: 2.4.6-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-offline-mirror-helper: 0.5.2
proxmox-widget-toolkit: 3.7.3
pve-cluster: 7.3-3
pve-container: 4.4-6
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-4~bpo11+2
pve-firewall: 4.3-5
pve-firmware: 3.6-6
pve-ha-manager: 3.6.1
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-2
qemu-server: 7.4-4
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.14-pve1

Spoiler: pct config 333

pct config 333
arch: amd64
cores: 8
features: fuse=1,nesting=1
hostname: jellyfin
lock: backup
memory: 12288
net0: name=eth0,bridge=vmbr0,firewall=1,gw=10.0.0.1,hwaddr=0A:7B:60:3A:74:E8,ip=10.3.3.3/24,type=veth
onboot: 0
ostype: debian
parent: vzdump
rootfs: local-zfs:subvol-333-disk-0,size=20000G
swap: 2048
unprivileged: 1

How else could I move the 10TB VE to a new host and what would be the fastest option?
I treid PBS, but it fails on restore...
That is why I wanted to backup only the working System (already treid to rebuild from scratch, but that didnt work out) and later rsync or scp the media (which lasted like a week over LAN - thats why wanted to try to backup to ext. HD, move the ext. HD to new node and restore, but I run out of disk-space)...

Regards,
~R.

Hmm, so the issue seems to be the trailing slash in the exclude path. I tried it locally and it works with --exclude-path /opt/plexmedia instead.

 

[fiona]

You don't provide the actual command you entered for the backup & its excludes.

But maybe try with one of the following:

--exclude-path '/opt/plexmedia/'
--exclude-path /opt/plexmedia
--exclude-path '/opt/plexmedia'
--exclude-path 'opt/plexmedia'
--exclude-path opt/plexmedia

You may get lucky.

Mostly for future readers: without a leading slash, it will also exclude any sub-directories matching the pattern (i.e. /some/other/opt/plexmedia), so please be careful!

 

[Riesling.Dry]

I used leading and trailing slash: vzdump 333 --ionice 1 --storage extern --exclude-path /opt/plexmedia/ see OP

acc. https://pve.proxmox.com/pve-docs/vzdump.1.html

--exclude-path <array>
Exclude certain files/directories (shell globs). Paths starting with / are anchored to the container’s root, other paths match relative to each subdirectory.

so I used the leading / to also exclude any sub-directories matching the pattern (i.e. /opt/plexmedia/movies/, /opt/plexmedia/tv/, etc.)

Will try again without the trailing / (and update host beforehand ;°)

 

[fiona]

For PBS backups or suspend mode backups (which use rsync), using a trailing slash does work (it will exclude only directories that match, but not files).

For non-PBS stop/snapshot mode backups however, a pattern with a trailing slash will not match at all, neither exclude files nor directories. This is because the pattern is passed to tar and that won't match with a trailing slash.

We'll see if the behavior can be aligned or otherwise, mention this in the documentation.

EDIT: it's actually only tar that behaves differently

 

[Riesling.Dry]

Thank you @fiona !
The traling slash was the cause of the problem - without it, it worked!
Many thanks for your help
Cheers,
~R.

Spoiler: result w./o. the trailing '/'

INFO: starting new backup job: vzdump 333 --ionice 1 --exclude-path /opt/plexmedia --storage extern10tb
INFO: Starting Backup of VM 333 (lxc)
INFO: Backup started at 2024-05-28 15:37:02
INFO: status = stopped
INFO: backup mode: stop
INFO: ionice priority: 1
INFO: CT Name: jellyfin
INFO: including mount point rootfs ('/') in backup
INFO: creating vzdump archive '/mnt/extern/dump/vzdump-lxc-333-2024_05_28-15_37_02.tar'
INFO: Total bytes written: 25136936960 (24GiB, 17MiB/s)
INFO: archive file size: 23.41GB
INFO: Finished Backup of VM 333 (00:24:33)
INFO: Backup finished at 2024-05-28 16:01:35
INFO: Backup job finished successfully
INFO: notified via target `mail-to-root`
TASK OK

 

[fiona]

I sent patches to warn about this (during backup and in the docs). Changing it right now seems like it could trip up some users, so that is probably better done during a (major) release to be able to mention it prominently in the release notes.

 

[Riesling.Dry]

Thank you Fiona,
glad to know it was not my dullness but an actual flaw - do you give bugbounties?!
Cheers,
~R.