Datacenter Summary - Storage Usage calculation algorithm

[Cha0s]

Hello,

How does Proxmox calculate the storage usage in the Datacenter Summary section?


I am running 3 nodes, with 4x3.8TB SSDs per node used in a Ceph Cluster (3 replicas - standard/default ceph installation).
I've configured the dashboard to only show the storage for ceph for a single node so that it doesn't add up multiple times the same storage.

The last 2 days I haven't created or changed any VM (50 running VMs). But I noticed that the storage space keeps changing.

For example (there are copy/pastes over the last 2 days):

Storage
7.67 TiB of 11.95 TiB


Storage
7.76 TiB of 11.99 TiB


Storage
7.66 TiB of 11.90 TiB


Storage
7.60 TiB of 11.80 TiB


Storage
7.96 TiB of 12.06 TiB


Storage
8.02 TiB of 12.10 TiB


Storage
8.05 TiB of 12.13 TiB

Obvisouly the used space will change over time. But how is it that the total space keeps changing? And not only changing but increasing!
What am I missing?

 

[wolfgang]

Hi,
you can set what storages are included the calculation.
Click on the gear and then you can select the storages.

 

[Cha0s]

Thank you but I've already done that.

I've configured the dashboard to only show the storage for ceph for a single node so that it doesn't add up multiple times the same storage.

Still, this doesn't answer my question.

How does Proxmox calculate the storage usage in the Datacenter Summary section?

Not which storages are included. But how those included storages are calculated.

 

[wolfgang]

But how is it that the total space keeps changing?

What do you mean with this?
Do you mean the free space gets less?

Ceph is a thin-allocated storage and so you have to trim the vdisk to release the allocated blocks.

 

[Cha0s]

No, I mean the total space gets more.

See my quote in my first post.
These copy/pastes are from the dashboard over a period of 24 hours.

At one point for example it was:
Storage
8.02 TiB of 12.10 TiB

And then at another point later on:
Storage
8.05 TiB of 12.13 TiB


I get that the used space will change over time. I am not asking about that. I am asking about the total space (the "of 12.13TiB" part). How can the total space increase when there is no new space available to the cluster?

 

[wigor]

I think the "total space" is a sum up from ceph available space. That is - as i understand - not totally fixed, because of the fill-grade of each single osd and the possibility to place new data regarding the crush algo.

 

[gvard]

I've seen an increase on total space too. @wigor, is what you mentioned documented somewhere to verify it?

 

[wigor]

Hey,
i don´t think it´s somewhere "documented". It´s more in the nature ceph allocates space in objects in placement groups. You can have a full cluster with lot of osds only at 80% and only one or a few osd at 95% or so. Then your cluster is observed as "full". Free = 0.
And if you rebalance then, there possible all osds will be at e.g. 82%. And so Free > 0.
Sorry i have no link, it´s only my view of ceph i can give you here.

 

[psionic]

Hi,
you can set what storages are included the calculation.
Click on the gear and then you can select the storages.

What gear are you referring to?

My Cluster summary seems to be showing only one node instead of all 4...

 

[wigor]

i found this:

(This is why Ceph extrapolates the remaining available space in the cluster not from the average utilization of OSDs but from the most full one instead.)

https://ceph.io/community/the-first-telemetry-results-are-in/