Error when deleting backup from datastore

[BigErchie]

Hi all,

I've come across the following error when deleting backups in PBS. Can't find any reference to this in the forum and wondered if it was a known issue or just my config?

proxmox-backup: 1.0-4
proxmox-backup-server: 1.0.1-1
pve-kernel-5.4: 6.2-7
pve-kernel-helper: 6.2-7
pve-kernel-5.4.65-1-pve: 5.4.65-1
pve-kernel-5.4.44-2-pve: 5.4.44-2
ifupdown2: 3.0.0-1+pve3
libjs-extjs: 6.0.1-10
proxmox-backup-docs: 1.0.1-1
proxmox-backup-client: 1.0.1-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.3-10
pve-xtermjs: 4.7.0-2
smartmontools: 7.1-pve2
zfsutils-linux: 0.8.4-pve2

Thanks in advance.

 

[t.lamprecht]

Hi!

Can you post the output of ls -laR /mnt/freenas/pbs/prodesk1/ct/104/2020-11-11T15:03:48Z (please recheck the path I just typed it in a hurry)?

 

[BigErchie]

Thanks Thomas. The dir looks to be empty.

root@pbs:~# ls -laR /mnt/freenas/pbs/prodesk1/ct/104/2020-11-11T15\:03\:48Z/
'/mnt/freenas/pbs/prodesk1/ct/104/2020-11-11T15:03:48Z/':
total 1
drwxr-xr-x 2 root backup 2 Nov 11 15:25 .
drwxr-xr-x 3 root backup 4 Nov 11 15:02 ..

I'm also seeing this in the web UI, size is simply just spinning but seems to indicate the snapshot is still there.

 

[BigErchie]

Hi @t.lamprecht , just wondering if you had any further thoughts on this given that PBS v1.0 has now been released?

 

[t.lamprecht]

Hi,

this rather just seems a single left over directory, you could rmdir it for it to be gone and not listed by the API/web interface anymore.

Or is this issue still occurring for new/other backups?

 

[BigErchie]

It's happening for all backups even the new ones that I'm still creating. I appear to be able to continue to create them and restore from the ones that have been created.

The problem seems to be that the software is not properly removing the directory during a manual delete or a prune/GC.

I thought it was perhaps a permissions error on my NFS share but this has not changed and the only thing that has is PBS has been updated. I presume that this is not something that is being widely reported?

 

[t.lamprecht]

I thought it was perhaps a permissions error on my NFS share but this has not changed and the only thing that has is PBS has been updated. I presume that this is not something that is being widely reported?

Ah, this is backed by a NFS? Yes, there's a bug @dcsapak could reproduce here only very recently (yesterday or so) with your described effects.

The underlying issue is how flocks (file locks) are handled by NFS, which breaks the final removal - we'll look into a fix for that.

 

[BigErchie]

Thanks @t.lamprecht , glad to know it's not something I have somehow managed to break tinkering with things. I guess I can switch to SMB for the time being until a fix is in place