[SOLVED] PBS full - Login fail with GUI but not with SSH

[LGTMozart]

Hi!

My PBS backups 4 VMs of my PVE and I need to restore one VM. But sometimes I have "PBS: error fetching datastores - 400 bad request (500)" in the PVE's GUI.
I think it is because my PBS is full (I'm awaiting new disks). I have also difficulties to login with the Web interface, I get "login failed - please retry". My login and password is good because I can login with ssh.
Yesterday I logged with success on the web interface and I "forget" some backups and keep only the last one for each VMs. But I didn't see changes on disks' occupation so I start a "garbage collection". Today I can't connect to the web interface.

How can I see (in ssh) if the garbage collection is still runing?

I tried "zfs list -t all" and get:
NAME USED AVAIL REFER
rpool 1.05T 0B 1.05T
rpool/ROOT 1.11G 0B 96K
rpool/ROOT/pbs-1 1.11G 0B 1.11G

I think it means my disk is sill full. How can I emptied it to gain stability to perform my VM's restoration?

Debian 10 (buster)
I didn't find how to get my PBS' version

 

[Dunuin]

GC won't work without some free space. You should set a future ZFS quota so you can`t end up in such a case in the first place, where your PBS is basically bricked.

Your best bet would be to copy your datastore folder to a bigger filesystem, Change the "path" of your datastore to the new location and then run the GC to trim some old backups snapshots. The trimmed datastore folder could then be copied back.

 

[LGTMozart]

Thanks Dunuin. I recently read about the ZFS quota. It should be required during the install (or I missed something).

If I'm lucky, monday I will have received my new disks and add them to my PBS (I have to find how) then I will do like you said.

 

[Dunuin]

I recently read about the ZFS quota. It should be required during the install (or I missed something).

No that is totally up to you to set it up. Same with many other important things like postfix and zfs-zed for email notifications on currupted backups and failed disks, verify jobs to check backups for integrity, crateting tokens and defining roles for ransomware protection, sync jobs to second PBS for worst case, ...

 

[LGTMozart]

No that is totally up to you to set it up. Same with many other important things like postfix and zfs-zed for email notifications on currupted backups and failed disks, verify jobs to check backups for integrity, crateting tokens and defining roles for ransomware protection, sync jobs to second PBS for worst case, ...

ok I will have a look when I will access the GUI.
Thank you.

 

[Dunuin]

Setting up ZFS quotas, zfs-zed, postfix has to be done from CLI.

 

[LGTMozart]

I'm working on ZFS quotas now.

zfs list
returns:


So must my command be (for a 90% quota):
zfs set quota=2.835G rpool ?

zfz-zed seems to be active, the /etc/zfs/zed.d/zed.rc file contains the uncomment line ZED_EMAIL_ADDR="root" and in the GUI I wrote an e-mail for root@pam
I didn't find what I have to do with postfix.
In the GUI on my Datastore options, Notify is set to 'Verify=Error, Sync=Error, GC=error' and Notify User is set to 'root@pam'. Is it enough?

 

[Dunuin]

So must my command be (for a 90% quota):
zfs set quota=2.835G rpool ?

Yes

zfz-zed seems to be active, the /etc/zfs/zed.d/zed.rc file contains the uncomment line ZED_EMAIL_ADDR="root" and in the GUI I wrote an e-mail for root@pam
I didn't find what I have to do with postfix.
In the GUI on my Datastore options, Notify is set to 'Verify=Error, Sync=Error, GC=error' and Notify User is set to 'root@pam'. Is it enough?

You need a mail server running on your PBS/PVE to be able to send emails. PVE/PBS comes with postfix, but you will have to set it up in order to be able to send emails. How to do that depends ob the requirements of your mail provider where you create a SMTP sattelite system for.

 

[LGTMozart]

Thank you for the answers.

 

[LGTMozart]

Hi Dunuin.

I had a lot of things to do and I almost forget finishing my work on pbs... So I just tried the command:
zfs set quota=2.835G rpool

But it didn't work. It returned the error "cannot set property for 'rpool': size is less than current used or reserved space"
I don't understand my mistake. Can you help me again ?

zfs list
returns:

 

[Dunuin]

Can you help me again ?

Sorry, should be zfs set quota=2.844T rpool not zfs set quota=2.835G rpool for a 90% quota.

 

[LGTMozart]

It worked! Thank you!