Backup problem (catalog_encode_u64)

[pmuch]

Hi.
I get error :
INFO: Error: error at "": catalog_encode_u64 failed - value >= 2^63
On some files - and the backup fails.

I hav this problem on one LXC container, all other containers and VMs go OK.

 

[fabian]

does this container have any files/directories out of the ordinary? the failing method is used to encode names of files/directories, size and mtime for regular files, and tables of directory contents.. could you include the output of pveversion -v?

 

[pmuch]

The files are rather ordinary, it's owncloud server. The volume is big, 3TB, and in 90% full.

It's on cluster, ceph storage, pveverison on node with problematic container:

proxmox-ve: 6.2-1 (running kernel: 5.4.60-1-pve)
pve-manager: 6.2-11 (running version: 6.2-11/22fb4983)
pve-kernel-5.4: 6.2-6
pve-kernel-helper: 6.2-6
pve-kernel-5.4.60-1-pve: 5.4.60-1
pve-kernel-5.4.55-1-pve: 5.4.55-1
pve-kernel-5.4.44-1-pve: 5.4.44-1
pve-kernel-5.4.41-1-pve: 5.4.41-1
ceph: 14.2.11-pve1
ceph-fuse: 14.2.11-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.4
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-1
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-6
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
openvswitch-switch: 2.12.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-10
pve-cluster: 6.1-8
pve-container: 3.1-13
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.1-3
pve-qemu-kvm: 5.0.0-13
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-14
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve1

pveverison -v on node running backup:

proxmox-ve: 6.2-1 (running kernel: 5.4.55-1-pve)
pve-manager: 6.2-11 (running version: 6.2-11/22fb4983)
pve-kernel-5.4: 6.2-6
pve-kernel-helper: 6.2-6
pve-kernel-5.3: 6.1-6
pve-kernel-5.4.60-1-pve: 5.4.60-2
pve-kernel-5.4.55-1-pve: 5.4.55-1
pve-kernel-5.4.44-1-pve: 5.4.44-1
pve-kernel-5.4.41-1-pve: 5.4.41-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
pve-kernel-5.3.18-3-pve: 5.3.18-3
pve-kernel-5.3.18-2-pve: 5.3.18-2
ceph: 14.2.11-pve1
ceph-fuse: 14.2.11-pve1
corosync: 3.0.4-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: residual config
ifupdown2: 3.0.0-1+pve2
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.16-pve1
libproxmox-acme-perl: 1.0.5
libpve-access-control: 6.1-2
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.2-1
libpve-guest-common-perl: 3.1-3
libpve-http-server-perl: 3.0-6
libpve-storage-perl: 6.2-6
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.3-1
lxcfs: 4.0.3-pve3
novnc-pve: 1.1.0-1
openvswitch-switch: 2.12.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-12
pve-cluster: 6.1-8
pve-container: 3.1-13
pve-docs: 6.2-5
pve-edk2-firmware: 2.20200531-1
pve-firewall: 4.1-2
pve-firmware: 3.1-3
pve-ha-manager: 3.1-1
pve-i18n: 2.2-1
pve-qemu-kvm: 5.0.0-13
pve-xtermjs: 4.7.0-2
qemu-server: 6.2-14
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-2
zfsutils-linux: 0.8.4-pve1

 

[fabian]

okay - so I can reproduce this (or another similar issue) with an mtime that's in the past.. can you check whether this container contains such files?

 

[pmuch]

I got this error on directory..
ls -l in this directory shows:
-rw-r--r-- 1 www-data www-data 127 Dec 13 1901 conv.conf
drwxr-xr-x 2 www-data www-data 4096 Sep 10 08:57 sflfs
-rw-r--r-- 1 www-data www-data 204302 Dec 13 1901 t_history.dat

 

[fabian]

thanks for confirming! filed https://bugzilla.proxmox.com/show_bug.cgi?id=2998 for tracking.

 

[ZenServer]

Hello i ve got the same issue with a LCX server (debian owncloud server)

"catalog_encode_u64 failed - value >= 2^63"

 

[infinityM]

Hey Guys, We're also running into this problem.
Is there a fix for it?

 

[fabian]

https://lists.proxmox.com/pipermail/pbs-devel/2020-October/001141.html but not yet finally reviewed and applied

 

[ZenServer]

https://lists.proxmox.com/pipermail/pbs-devel/2020-October/001141.html but not yet finally reviewed and applied

Thanks for this information. Backup server is great

 

[infinityM]

any idea when this will be fixed?

 

[fabian]

when the patch I linked in my last post was reviewed, applied, the proxmox-backup packages got a version bump and went through our internal testing. in other words - when it's done. probably won't take months, but probably won't happen in a single day either

 

[ZenServer]

Problem solved with last update !

 

[ZenServer]

but probably won't happen in a single day either

and at last.... they did ! ;-)