I have configured 3 node proxmox with equal number of OSDs and storage. The storage use keeps increasing despite of the allocated space. The used percentage grows from 99 percent to 100 percent. Why is this happening?
CEPH storage use grows rapidly despite of space allocation
- Thread starter Sujith Arangan
- Start date
No I was just looking at the summary. But still how can the dedicated storage increase consuming the disk space?
Space allocated where?
Can you please show the output of the following commands within
[CODE][/CODE] tags for better readability?ceph -sceph dfpveceph pool ls --noborder(make sure the terminal is large enough or pipe it into a file, as any content that won't fit will be cut offceph osd df tree
Hello Please find the output below. I removed some of the VMs a week ago but after one week again the storage reaches 100 %. I am wondering how the VM storage balloons?
root@node-1:~# ceph -s
cluster:
id: f4bc1d34-85cc-4686-b923-52977765ec61
health: HEALTH_ERR
1 backfillfull osd(s)
1 full osd(s)
3 nearfull osd(s)
4 pool(s) full
services:
mon: 3 daemons, quorum orca1,orca2,orca3 (age 4w)
mgr: orca1(active, since 10w)
mds: 1/1 daemons up
osd: 6 osds: 6 up (since 4w), 6 in (since 4w)
data:
volumes: 1/1 healthy
pools: 4 pools, 193 pgs
objects: 822.04k objects, 3.1 TiB
usage: 9.2 TiB used, 1.2 TiB / 10 TiB avail
pgs: 193 active+clean
root@node-1:~# ceph df
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
ssd 10 TiB 1.2 TiB 9.2 TiB 9.2 TiB 88.24
TOTAL 10 TiB 1.2 TiB 9.2 TiB 9.2 TiB 88.24
--- POOLS ---
POOL ID PGS STORED OBJECTS USED %USED MAX AVAIL
.mgr 1 1 27 MiB 7 82 MiB 100.00 0 B
ORCA-CEPH-FS_data 2 32 40 GiB 10.24k 120 GiB 100.00 0 B
ORCA-CEPH-FS_metadata 3 32 6.4 MiB 24 19 MiB 100.00 0 B
ORCA-CEPH-VM 4 128 3.0 TiB 811.77k 9.1 TiB 100.00 0 B
root@orca1:~# pveceph pool ls --noborder
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
Name Size Min Size PG Num min. PG Num Optimal PG Num PG Autosca
.mgr 3 2 1 1 1 on
ORCA-CEPH-FS_data 3 2 32 32 on
ORCA-CEPH-FS_metadata 3 2 32 16 16 on
ORCA-CEPH-VM 3 2 128 128 on
root@node-1:~# ceph osd df tree
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME
-1 10.47958 - 10 TiB 9.2 TiB 9.2 TiB 94 KiB 23 GiB 1.2 TiB 88.24 1.00 - root default
-3 3.49319 - 3.5 TiB 3.1 TiB 3.1 TiB 32 KiB 7.8 GiB 420 GiB 88.25 1.00 - host orca1
0 ssd 1.74660 1.00000 1.7 TiB 1.5 TiB 1.5 TiB 13 KiB 3.8 GiB 262 GiB 85.36 0.97 90 up osd.0
1 ssd 1.74660 1.00000 1.7 TiB 1.6 TiB 1.6 TiB 19 KiB 4.0 GiB 159 GiB 91.13 1.03 103 up osd.1
-5 3.49319 - 3.5 TiB 3.1 TiB 3.1 TiB 30 KiB 7.5 GiB 421 GiB 88.24 1.00 - host orca2
2 ssd 1.74660 1.00000 1.7 TiB 1.5 TiB 1.5 TiB 11 KiB 3.8 GiB 217 GiB 87.86 1.00 92 up osd.2
3 ssd 1.74660 1.00000 1.7 TiB 1.5 TiB 1.5 TiB 19 KiB 3.7 GiB 204 GiB 88.62 1.00 101 up osd.3
-7 3.49319 - 3.5 TiB 3.1 TiB 3.1 TiB 32 KiB 7.6 GiB 421 GiB 88.24 1.00 - host orca3
4 ssd 1.74660 1.00000 1.7 TiB 1.7 TiB 1.7 TiB 19 KiB 3.8 GiB 89 GiB 95.01 1.08 100 up osd.4
5 ssd 1.74660 1.00000 1.7 TiB 1.4 TiB 1.4 TiB 13 KiB 3.8 GiB 331 GiB 81.48 0.92 93 up osd.5
TOTAL 10 TiB 9.2 TiB 9.2 TiB 97 KiB 23 GiB 1.2 TiB 88.24
MIN/MAX VAR: 0.92/1.08 STDDEV: 4.25
root@node-1:~#
Last edited: