Proxmox Backup Server 2.1-1 server with two zfs partitions, 10TB and 12TB, forming a datastore that is 100% used. I deleted some larger backups, but the space was not freed up. Everything that is on the disks, I no longer need, how to clean the disks and make the space available again for use?
Proxmox Backup Server with disks full
- Thread starter thomasddiego
- Start date
Hi,
do you have a garbage collection scheduled? Deleting the backups will only free the index files, not the chunks containing the actual data. Only by runnning a garbage collection job, you will free up non-referenced chunks, which are older that >24 hours.
And with 100% full pools the GC will probably fail because of insufficient space. Make sure to disable all backup jobs so it won't fill up the few MBs you might be able to free um by deleting indexes.
Gargabe starts and always stops pointing to different chunks
2023-08-21T12:33:07-03:00: starting garbage collection on store BackupUnit
2023-08-21T12:33:07-03:00: Start GC phase1 (mark used chunks)
2023-08-21T13:04:19-03:00: marked 1% (3 of 218 index files)
2023-08-21T13:12:42-03:00: marked 2% (5 of 218 index files)
2023-08-21T13:24:28-03:00: marked 3% (7 of 218 index files)
2023-08-21T13:29:49-03:00: marked 4% (9 of 218 index files)
2023-08-21T13:31:16-03:00: marked 5% (11 of 218 index files)
2023-08-21T13:41:49-03:00: marked 6% (14 of 218 index files)
2023-08-21T13:41:50-03:00: TASK ERROR: update atime failed for chunk/file "/mnt/datastore/BackupUnit/.chunks/e437/e437495ad2cc01deca02e9bb0a06869ee69dd7da4eb723ff55ced3e91cf8bcd7" - EPERM: Operation not permitted
The disk space of the VM where the Proxmox Backup OS is installed is ok, only the datastore where the backups are saved is having a problem.
Any other suggestion?
2023-08-21T12:33:07-03:00: starting garbage collection on store BackupUnit
2023-08-21T12:33:07-03:00: Start GC phase1 (mark used chunks)
2023-08-21T13:04:19-03:00: marked 1% (3 of 218 index files)
2023-08-21T13:12:42-03:00: marked 2% (5 of 218 index files)
2023-08-21T13:24:28-03:00: marked 3% (7 of 218 index files)
2023-08-21T13:29:49-03:00: marked 4% (9 of 218 index files)
2023-08-21T13:31:16-03:00: marked 5% (11 of 218 index files)
2023-08-21T13:41:49-03:00: marked 6% (14 of 218 index files)
2023-08-21T13:41:50-03:00: TASK ERROR: update atime failed for chunk/file "/mnt/datastore/BackupUnit/.chunks/e437/e437495ad2cc01deca02e9bb0a06869ee69dd7da4eb723ff55ced3e91cf8bcd7" - EPERM: Operation not permitted
The disk space of the VM where the Proxmox Backup OS is installed is ok, only the datastore where the backups are saved is having a problem.
Any other suggestion?
Attachments
So you still have 218 index files referencing chunks. It might be worth to delete those not needed anymore before running the garbage collection.
The EPERM when updating the atime might be a sideffect of the zfs pool running out of space, as copy on write requires some space left to work, so freeing up a few more MB by deleting additional index files might be enough to fix this.
If you don't care about the backups on this datastore anymore, as stated in your initial post, you could also delete the datastore completely and recreate a new one after removing all the
.chunks and index files.And for the future I would highly recommend you set a quota like
zfs set quota=9T YourPool so you can't fill up your pool more than 90% by accident.Good morning everybody,
I'm coming back here just to report that all the tips helped to resolve the issue.
I deleted the indexes of the backups already made, then performed garbage collection, after many hours, all the datastore space was freed. I have already reconfigured which VMs should make backups and the first run has already completed successfully.
I want to thank Chris and Dunuin for their tips and suggestions.
I'm coming back here just to report that all the tips helped to resolve the issue.
I deleted the indexes of the backups already made, then performed garbage collection, after many hours, all the datastore space was freed. I have already reconfigured which VMs should make backups and the first run has already completed successfully.
I want to thank Chris and Dunuin for their tips and suggestions.