Clarity on Storage Info

S

Seed

Renowned Member
Oct 18, 2019
109
63
68
124
I have an HDD pool for Ceph that consists of 12x10TB disks that are spread across 3 nodes so 4x10TB in each.

In datacenter summary I see this:

Screen Shot 2020-01-24 at 9.25.59 PM.png

but in datacenter >> ceph >> performance I see this:

Screen Shot 2020-01-24 at 9.26.49 PM.png

The above "Usage" seems accurate as I am using 3 part replication and currently have about 17TB of data so 16x3 or 51 or so. Why is "Storage" so far off?
 
Click on your user menu in the upper right hand corner.

Select "My Settings"

Select the storage you would like to see displayed on the datacenter summary page.

With "nothing" selected, I'm not sure what it "defaults" to, but mine looks weird too. If I select my ceph pools, then the numbers line up between ceph and datacenter view.
 
Thanks Allan,

The usage still doesn't line up and is even more confusing...

This is just the HDD pools (Host1, Host2, Host3) on the hosts that actually have the disks which is 12x10TB (before proxmox upgrade), and it states I have 10TB more than I do disks...

Screen Shot 2020-01-25 at 11.01.23 AM.png

Here is the OSD list and config:

Screen Shot 2020-01-25 at 11.55.52 AM.png Screen Shot 2020-01-25 at 11.54.17 AM.png

I just upgraded Proxmox to 6.1 and reset the view so it's now default with no Dashboard Storage selected in the My Settings and get this now:

Screen Shot 2020-01-25 at 12.03.35 PM.png

Then if i select Host1, Host2, Host 3 which is where the 12 10TB HDDs are I get this, 4 disks across 3 hosts:

Screen Shot 2020-01-25 at 12.05.29 PM.png

And here is a host view of the pool-hdd. It thinks it only has 30TB. :/

Screen Shot 2020-01-25 at 12.24.15 PM.png
So I have no idea what's going on....
 

Attachments

  • Screen Shot 2020-01-25 at 11.59.49 AM.png
    Screen Shot 2020-01-25 at 11.59.49 AM.png
    23 KB · Views: 1
Last edited:
I agree, definitely some fishy graphs/charts going on there. Mine are weird too.

Instead of selecting the hosts, try selecting the relevant pool instead. That at least seems to get the datacenter view to align to the ceph view.

In my cluster, the "host view" doesn't seem to be obviously linked to anything either. My hosts have different amounts of storage installed, and yet, this summary appears the same across all of them, and wrong quantities for the selected pool anyway.
 
AllanM said:
I agree, definitely some fishy graphs/charts going on there. Mine are weird too.

Instead of selecting the hosts, try selecting the relevant pool instead. That at least seems to get the datacenter view to align to the ceph view.

In my cluster, the "host view" doesn't seem to be obviously linked to anything either. My hosts have different amounts of storage installed, and yet, this summary appears the same across all of them, and wrong quantities for the selected pool anyway.
Click to expand...

You have to select the pools per host and when I choose the 3 pools that are on host1, host2, host3. it's wrong..

Screen Shot 2020-01-25 at 11.40.00 PM.png
 
Ok so i wiped out my main data volume and reset the values in the UI so nothing is selected and it shows like this:

Screen Shot 2020-02-10 at 11.29.15 PM.png

This makes no sense and is pretty annoying. Usage is accurate, Storage is making stuff up like our President.
 
i did explain this sometime ago in another thread but i will again here:

the storage counter counts by default:

the 'local' storage for each node
and all other storages only once

since there is not a nice way to get which storages are local to which node and which are shared...
 
dcsapak said:
i did explain this sometime ago in another thread but i will again here:

the storage counter counts by default:

the 'local' storage for each node
and all other storages only once

since there is not a nice way to get which storages are local to which node and which are shared...
Click to expand...

no offense, but this still doesn’t make any sense. No where, anywhere in this system am I using anywhere close “25TB” of data. I know what’s on ALL my disks on ALL nodes and it’s about 500GB.

it’s a bug...

....
 
can you post the output of 'pvesh get /cluster/resources --type storage --output-format json-pretty' ?
 
So this looks to include nfs mounts I guess, but the total still doesn't make sense. If it's going to factor in "everything", it should be closer to 200TB of storage.

Also, even if i choose the HDD pool of the "hosts" that actually have the HDD disks, it's 20TB short for the total. 3 hosts @ 4 10TB disks in each.

Screen Shot 2020-02-11 at 10.29.34 AM.png

Code: 
[
   {
      "disk" : 40462450871,
      "id" : "storage/host3/pool-hdd",
      "maxdisk" : 37857406222519,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-hdd",
      "type" : "storage"
   },
   {
      "disk" : 40448471223,
      "id" : "storage/host1/pool-hdd",
      "maxdisk" : 37857459351735,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-hdd",
      "type" : "storage"
   },
   {
      "disk" : 40479228087,
      "id" : "storage/hp1/pool-hdd",
      "maxdisk" : 37857355890871,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-hdd",
      "type" : "storage"
   },
   {
      "disk" : 40468041911,
      "id" : "storage/hp2/pool-hdd",
      "maxdisk" : 37857411813559,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-hdd",
      "type" : "storage"
   },
   {
      "disk" : 40448471223,
      "id" : "storage/host2/pool-hdd",
      "maxdisk" : 37857459351735,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-hdd",
      "type" : "storage"
   },
   {
      "disk" : 0,
      "id" : "storage/host3/local-lvm",
      "maxdisk" : 64063799296,
      "node" : "host3",
      "shared" : 0,
      "status" : "available",
      "storage" : "local-lvm",
      "type" : "storage"
   },
   {
      "disk" : 0,
      "id" : "storage/host1/local-lvm",
      "maxdisk" : 64063799296,
      "node" : "host1",
      "shared" : 0,
      "status" : "available",
      "storage" : "local-lvm",
      "type" : "storage"
   },
   {
      "disk" : 6406379,
      "id" : "storage/hp1/local-lvm",
      "maxdisk" : 64063799296,
      "node" : "hp1",
      "shared" : 0,
      "status" : "available",
      "storage" : "local-lvm",
      "type" : "storage"
   },
   {
      "disk" : 6406379,
      "id" : "storage/hp2/local-lvm",
      "maxdisk" : 64063799296,
      "node" : "hp2",
      "shared" : 0,
      "status" : "available",
      "storage" : "local-lvm",
      "type" : "storage"
   },
   {
      "disk" : 0,
      "id" : "storage/host2/local-lvm",
      "maxdisk" : 64063799296,
      "node" : "host2",
      "shared" : 0,
      "status" : "available",
      "storage" : "local-lvm",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host3/nas-iso",
      "maxdisk" : 446566629376,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-iso",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host1/nas-iso",
      "maxdisk" : 446566629376,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-iso",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/hp1/nas-iso",
      "maxdisk" : 446566629376,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-iso",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/hp2/nas-iso",
      "maxdisk" : 446566629376,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-iso",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host2/nas-iso",
      "maxdisk" : 446566629376,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-iso",
      "type" : "storage"
   },
   {
      "disk" : 51186208955,
      "id" : "storage/host3/pool-nvme",
      "maxdisk" : 2835628658875,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-nvme",
      "type" : "storage"
   },
   {
      "disk" : 51186208955,
      "id" : "storage/host1/pool-nvme",
      "maxdisk" : 2835628658875,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-nvme",
      "type" : "storage"
   },
   {
      "disk" : 51186208955,
      "id" : "storage/hp1/pool-nvme",
      "maxdisk" : 2835628658875,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-nvme",
      "type" : "storage"
   },
   {
      "disk" : 51186208955,
      "id" : "storage/hp2/pool-nvme",
      "maxdisk" : 2835628658875,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-nvme",
      "type" : "storage"
   },
   {
      "disk" : 51186208955,
      "id" : "storage/host2/pool-nvme",
      "maxdisk" : 2835628658875,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-nvme",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host3/nas-snippets",
      "maxdisk" : 446566629376,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-snippets",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host1/nas-snippets",
      "maxdisk" : 446566629376,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-snippets",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/hp1/nas-snippets",
      "maxdisk" : 446566629376,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-snippets",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/hp2/nas-snippets",
      "maxdisk" : 446566629376,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-snippets",
      "type" : "storage"
   },
   {
      "disk" : 8096579584,
      "id" : "storage/host2/nas-snippets",
      "maxdisk" : 446566629376,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-snippets",
      "type" : "storage"
   },
   {
      "disk" : 26214566723584,
      "id" : "storage/host3/nas-vzdump",
      "maxdisk" : 30706267455488,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-vzdump",
      "type" : "storage"
   },
   {
      "disk" : 26214587695104,
      "id" : "storage/host1/nas-vzdump",
      "maxdisk" : 30706267455488,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-vzdump",
      "type" : "storage"
   },
   {
      "disk" : 26214490177536,
      "id" : "storage/hp1/nas-vzdump",
      "maxdisk" : 30706267455488,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-vzdump",
      "type" : "storage"
   },
   {
      "disk" : 26214490177536,
      "id" : "storage/hp2/nas-vzdump",
      "maxdisk" : 30706267455488,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-vzdump",
      "type" : "storage"
   },
   {
      "disk" : 26214587695104,
      "id" : "storage/host2/nas-vzdump",
      "maxdisk" : 30706267455488,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "nas-vzdump",
      "type" : "storage"
   },
   {
      "disk" : 3637555200,
      "id" : "storage/host3/local",
      "maxdisk" : 29194506240,
      "node" : "host3",
      "shared" : 0,
      "status" : "available",
      "storage" : "local",
      "type" : "storage"
   },
   {
      "disk" : 4130074624,
      "id" : "storage/host1/local",
      "maxdisk" : 29194506240,
      "node" : "host1",
      "shared" : 0,
      "status" : "available",
      "storage" : "local",
      "type" : "storage"
   },
   {
      "disk" : 3355860992,
      "id" : "storage/hp1/local",
      "maxdisk" : 29194506240,
      "node" : "hp1",
      "shared" : 0,
      "status" : "available",
      "storage" : "local",
      "type" : "storage"
   },
   {
      "disk" : 3271143424,
      "id" : "storage/hp2/local",
      "maxdisk" : 29194506240,
      "node" : "hp2",
      "shared" : 0,
      "status" : "available",
      "storage" : "local",
      "type" : "storage"
   },
   {
      "disk" : 3700834304,
      "id" : "storage/host2/local",
      "maxdisk" : 29194506240,
      "node" : "host2",
      "shared" : 0,
      "status" : "available",
      "storage" : "local",
      "type" : "storage"
   },
   {
      "disk" : 33295069057,
      "id" : "storage/host3/pool-ssd",
      "maxdisk" : 476231561089,
      "node" : "host3",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-ssd",
      "type" : "storage"
   },
   {
      "disk" : 33295069057,
      "id" : "storage/host1/pool-ssd",
      "maxdisk" : 476232183681,
      "node" : "host1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-ssd",
      "type" : "storage"
   },
   {
      "disk" : 33295069057,
      "id" : "storage/hp1/pool-ssd",
      "maxdisk" : 476232183681,
      "node" : "hp1",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-ssd",
      "type" : "storage"
   },
   {
      "disk" : 33295069057,
      "id" : "storage/hp2/pool-ssd",
      "maxdisk" : 476232183681,
      "node" : "hp2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-ssd",
      "type" : "storage"
   },
   {
      "disk" : 33295069057,
      "id" : "storage/host2/pool-ssd",
      "maxdisk" : 476232183681,
      "node" : "host2",
      "shared" : 1,
      "status" : "available",
      "storage" : "pool-ssd",
      "type" : "storage"
   }
]
 
Last edited:
but i get exaclty there when i count

by default only the 'local' storage (literally the storage named 'local') gets counted for each node, all others only once
(i'll send a patch today that improves that a bit, since we have the information there if a storage is shared or not)
with that i come to 24TB/66TB which is about what you got (this depends on the order of the api result)

and i do not know which storages you have selected in your last screenshot, but i get to those numbers
if i count 3 hosts (storage pool-hdd, btw why is it marked shared? it clearly is not) , note that 10TB disks are not 10TiB but 9.09TiB * 4 = 36.38TiB per host *3 ~109TiB minus some overhead for lvm etc...
 
Cool yeah you’re right I forget about the loss of each drive with that many drives and sizes, it ends up being significant.

thanks for the patch

C
 
You must log in or register to reply here.

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Quick Navigation

Home Get Subscription Wiki Downloads Proxmox Customer Portal About

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!
Community platform by XenForo® © 2010-2024 XenForo Ltd.
Top Bottom
Back