PBS Datastore 100% full

mare19

Member
Jul 20, 2021
43
1
13
28
Hi,

My PBS Datastore is tottaly full and I can't get free space.

Here the output of df -h :
1698674301188.png

Any suggestion?

Thanks
 
Hi,
unfortunately you will have to free space on the mounted filesystem or increase it's capacity. What storage backend is used for the datastore?
 
Hi,
unfortunately you will have to free space on the mounted filesystem or increase it's capacity. What storage backend is used for the datastore?

HI Chris

I deleted some some vm Groups but it didn't help. I created a ZFS pool RAID z1 if you mean that?
 
HI Chris

I deleted some some vm Groups but it didn't help. I created a ZFS pool RAID z1 if you mean that?
deleting the groups will not clean up much space, as that only removes the index files but not the chunks referencing the data. If you are still able to run a garbage collection, that might help, if some chunks have been marked for cleanup more than 24h ago.
 
deleting the groups will not clean up much space, as that only removes the index files but not the chunks referencing the data. If you are still able to run a garbage collection, that might help, if some chunks have been marked for cleanup more than 24h ago.
Garbage collection because no space left, can I deleted them manually? Or any other thing I can do?
 
You cannot really cleanup the chunks by yourself, as you don't know which chunks are still referenced by backups and which not. An option would be to remove data from other datasets on the same zpool or files not related to the proxmox backup datastore. If that is not possible, then you are unfortunately out of luck. You will either have to expand the zpool or backup/copy your datastore to some other, larger storage and reconfigure the storage backing path in the config.
 
Last edited:
  • Like
Reactions: mare19
Garbage collection because no space left, can I deleted them manually? Or any other thing I can do?
You don't know which chunk belongs to what backup snapshot. So by randomly deleting chunks you will probably corrupt most of your backups snapshots.

Then there aren't many options if you can't run a GC.

If you don't care about losing all your backups you could just delete the datastore folder and start from scratch with a new datastore.

If you care about your backups you can only:
1.) buy more disks and add them to the pool so you get more space
2.) mount a bigger filesystem (NFS/SMB share or USB disk), copy the whole datastore to it, point PBS to the new datastore path and then run a GC
3.) mount some additional storage, move some of the chunks to the new storage and then symlink them to your datastore, run the GC and later move them back
So all options require having some additional storage.

For the future, you should really set up some quotas. A ZFS pool shouldn't be filled too much anyway as it will become slow when full. Usually recommended is not to fill it more than 80-90%. So I would set a 90% quota so you can't fill it more than 90% by accident. If you then will it to 90% where everything will stop working you could increase the quota again to something like 95% so you get some free space so you could at least run the GC to free up some space.
 
Last edited:
  • Like
Reactions: mare19
Thank you guys

I will delete the datastore folder and start from scratch with a new datastore.

- And the quotas are a good idea, in the future I will set it up like this
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!