PBS Datastore 100% full

M

mare19

Member
Jul 20, 2021
44
1
13
29
Hi,

My PBS Datastore is tottaly full and I can't get free space.

Here the output of df -h :
1698674301188.png

Any suggestion?

Thanks
 
Hi,
unfortunately you will have to free space on the mounted filesystem or increase it's capacity. What storage backend is used for the datastore?
 
Chris said:
Hi,
unfortunately you will have to free space on the mounted filesystem or increase it's capacity. What storage backend is used for the datastore?
Click to expand...

HI Chris

I deleted some some vm Groups but it didn't help. I created a ZFS pool RAID z1 if you mean that?
 
mare19 said:
HI Chris

I deleted some some vm Groups but it didn't help. I created a ZFS pool RAID z1 if you mean that?
Click to expand...
deleting the groups will not clean up much space, as that only removes the index files but not the chunks referencing the data. If you are still able to run a garbage collection, that might help, if some chunks have been marked for cleanup more than 24h ago.
 
Chris said:
deleting the groups will not clean up much space, as that only removes the index files but not the chunks referencing the data. If you are still able to run a garbage collection, that might help, if some chunks have been marked for cleanup more than 24h ago.
Click to expand...
Garbage collection because no space left, can I deleted them manually? Or any other thing I can do?
 
You cannot really cleanup the chunks by yourself, as you don't know which chunks are still referenced by backups and which not. An option would be to remove data from other datasets on the same zpool or files not related to the proxmox backup datastore. If that is not possible, then you are unfortunately out of luck. You will either have to expand the zpool or backup/copy your datastore to some other, larger storage and reconfigure the storage backing path in the config.
 
Last edited:
  • Like
Reactions: mare19
mare19 said:
Garbage collection because no space left, can I deleted them manually? Or any other thing I can do?
Click to expand...
You don't know which chunk belongs to what backup snapshot. So by randomly deleting chunks you will probably corrupt most of your backups snapshots.

Then there aren't many options if you can't run a GC.

If you don't care about losing all your backups you could just delete the datastore folder and start from scratch with a new datastore.

If you care about your backups you can only:
1.) buy more disks and add them to the pool so you get more space
2.) mount a bigger filesystem (NFS/SMB share or USB disk), copy the whole datastore to it, point PBS to the new datastore path and then run a GC
3.) mount some additional storage, move some of the chunks to the new storage and then symlink them to your datastore, run the GC and later move them back
So all options require having some additional storage.

For the future, you should really set up some quotas. A ZFS pool shouldn't be filled too much anyway as it will become slow when full. Usually recommended is not to fill it more than 80-90%. So I would set a 90% quota so you can't fill it more than 90% by accident. If you then will it to 90% where everything will stop working you could increase the quota again to something like 95% so you get some free space so you could at least run the GC to free up some space.
 
Last edited:
  • Like
Reactions: mare19
Thank you guys

I will delete the datastore folder and start from scratch with a new datastore.

- And the quotas are a good idea, in the future I will set it up like this
 
You must log in or register to reply here.
Top Bottom