PBS 3.1-2 wrong datastore information sshfs

[popallo]

Hello everyone !

To explain the context, I use a hetzner storage box for storing the backups I make for different PVE nodes. It works very well.
Previously, I mounted the storage box in CIFS but as I sometimes had disconnections, this generated numerous entries in kern.log and I found myself having to restart the PBS.

In short, I used SSHFS instead of CIFS and I no longer have this problem. The assembly is carried out via auto.fs by following this tutorial:
https://community.hetzner.com/tutorials/setup-autofs-mount-storagebox

It works but I end up with an inconsistent display on PBS.

In fact, it tells me that the datastore is 250TB while I have a 1TB storage box.



A "df -h" also shows that the mount point is 1tb.



I suspect that the storage boxes are CEPH type storage or equivalent, so for my part I just obtain a quota, but I cannot find why the display under PBS is incorrect.

I had 1TB displayed in CIFS.

Anyone have an idea or a clue?

Note that the PBS works in an LXC container with the FUSE option.

Any help or clue will be highly appreciated thanks.

 

[ggoller]

Hi!
could you try executing `stat -f .` on the datastore and post the result?

 

[popallo]

Hi!
could you try executing `stat -f .` on the datastore and post the result?

Thank you for the feedback.

Of course, here is the result of the command:

backup@pbs:/mnt/datastore# stat -f .

   File: "."

     ID: 0 Name: 255 Type: fuseblk

Block size: 131072 Fundamental block size: 512

Blocks: Total: 2074776029 Free: 748854998 Available: 748854998

Inodes: Total: 750095161 Free: 748854998

 

[ggoller]

Hi,
we calculate the total/used/available storage using the output of `stat -f .` (eg Blocks Total * Block size).
We could try to patch this, so that the size is calculated using Blocks Total * Fundamental Block Size—which in your case results to `1062.28 GB`— which is roughly correct (this could also be kinda random, because the `Fundamental Block Size` in ceph was (ie is in your case probably) the PAGE_CACHE_SIZE).
Although it seems that this will not be possible anymore as ceph now sets Fundamental Block Size = Block Size. (Bug: https://tracker.ceph.com/issues/3793, kernel commit: https://git.kernel.org/pub/scm/linu.../?id=92a49fb0f79f3300e6e50ddf56238e70678e4202).

I think this looks more like this ceph bug: https://tracker.ceph.com/issues/55090, which would be an issue regarding the quote.

Out of curiosity, could you post the output of `df -a` on the datastore?

 

[popallo]

Thank you for taking the time to add all these details.

Here is the result of df -ah:

datastore:                             990G  633G  358G  64% /mnt/datastore

We see here that I have what is expected.

 

[ggoller]

Hmm, I looked into the source code of `df` and it seems like they also use the `f_frsize` (Fragmented block size) parameter instead of the `f_bsize` (Block size). I'll get out a patch now and see what the others say

 

[popallo]

Hmm, I looked into the source code of `df` and it seems like they also use the `f_frsize` (Fragmented block size) parameter instead of the `f_bsize` (Block size). I'll get out a patch now and see what the others say

Great news

And if I can be a crash test server or beta tester, I would be very pleased.

 

[ggoller]

just fyi, the patch is on the mailing list already: https://lists.proxmox.com/pipermail/pbs-devel/2024-January/007676.html
I hope it's gonna get applied soon!

 

[popallo]

just fyi, the patch is on the mailing list already: https://lists.proxmox.com/pipermail/pbs-devel/2024-January/007676.html
I hope it's gonna get applied soon!

Hello, thanks for the follow-up! excellent news I can't wait to be able to apply it and I will be sure to come and give feedback on how well it works .

 

[unam]

Hi folks,

I also have the problem and I'm very happy to find out that it's already solved

Would you have any news about your patch integration @ggoller ?

Regards,

 

[ggoller]

Hi!
this has been merged!
commit: https://git.proxmox.com/?p=proxmox.git;a=commit;h=78bf05a458592ffe9a6a741bdd2d226bc5d84e25

 

[popallo]

Hi!
this has been merged!
commit: https://git.proxmox.com/?p=proxmox.git;a=commit;h=78bf05a458592ffe9a6a741bdd2d226bc5d84e25

Hi @ggoller and thks for the feedback.

Do you think there is something to do after the update to apply the effects of the patch?

I ask this because I have just updated the pbs and unfortunately, after restarting the server, I still see the same information concerning the storage (see my comments above).

After apt update && apt upgrade :

Setting up proxmox-widget-toolkit (4.1.5) ...
Setting up proxmox-backup-docs (3.1.5-1) ...
Setting up proxmox-backup-client (3.1.5-1) ...
Setting up proxmox-backup-server (3.1.5-1) ...

$ df -h

pbs:                             968G  868G  101G  90% /mnt/pbs

Under pbs:



Thank you in advance for your help.

 

[ggoller]

The patch has only been merged into the current master branch. The next release would be in a few (2-3) weeks. Otherwise you can also clone the source code and build it yourself!

 

[popallo]

The patch has only been merged into the current master branch. The next release would be in a few (2-3) weeks. Otherwise you can also clone the source code and build it yourself!

Ok thks for the feedback! I will wait for the next release so .

 

[unam]

Hi guys,

I just upgraded to latest 3.2.2 but the problem does not looks to be solved.
Or maybe the patch was not integrated in the current release ?

Regards,

 

[ggoller]

Hi!
the patch should be integrated in this release...
Could you please post some information about the problematic datastore? Backing storage, `stat -f .` output, if ceph or zfs the quota etc.

 

[popallo]

Hi, same thing for me. Latest version of PBS (3.2.2) and same problem with wrong datastore size under pbs.

With df -h :

5.0T  953G  4.1T  19%

Under PBS :


With stat -f:

stat -f .
  File: "."
    ID: 0        Namelen: 255     Type: fuseblk
Block size: 131072     Fundamental block size: 512
Blocks: Total: 10587981953 Free: 8590751412 Available: 8590751412
Inodes: Total: 8591775918 Free: 8590751412

The datastore is a Hetzner storage box, mount with sshfs for me.

 

[unam]

Hi,

Any news about this bug ? How could we help to fix it ?

Regards,

 

[ggoller]

Hi!
sorry for the late response. We sadly forgot to bump the librust-proxmox-sys library and thus this commit (https://git.proxmox.com/?p=proxmox.git;a=commit;h=78bf05a458592ffe9a6a741bdd2d226bc5d84e25) has not been pulled in in the latest release.

 

[unam]

Hi, thanks for you reply.

Is there any hope that this commit will be in the next release ?