LVM storage 100% used

[cartwright]

Hello,

I am having issues with a volume group with 5 disks in it. For some reason, they are showing full and I cannot seem to figure out why nor a way to clear whatever LVM thinks is on there. These are 5 brand new disks that have nothing on them. I am trying to create a volume group called fileserver with logical volume data.

outputs of pvdisplay, vgdisplay, and lvdisplay. Help would be appreciated.

root@pve:~# pvdisplay
--- Physical volume ---
PV Name /dev/nvme0n1p3
VG Name pve
PV Size 232.63 GiB / not usable 1.16 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 59554
Free PE 4057
Allocated PE 55497
PV UUID FmqoY3-gExH-LfmW-aNPs-E6h6-e3g9-ByRhza

--- Physical volume ---
PV Name /dev/sda
VG Name fileserver
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 953861
Free PE 0
Allocated PE 953861
PV UUID lj7VMC-04OK-foMK-iEqx-eevi-E1kP-CI4E2R

--- Physical volume ---
PV Name /dev/sdb
VG Name fileserver
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 953861
Free PE 0
Allocated PE 953861
PV UUID bolJgp-X9Wp-M1xz-jBzH-qTzn-ELLb-c0nBDA

--- Physical volume ---
PV Name /dev/sdc
VG Name fileserver
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 953861
Free PE 0
Allocated PE 953861
PV UUID Sfuszd-fjtS-PaqV-IQXY-NaJs-6dx1-lHsD0R

--- Physical volume ---
PV Name /dev/sdd
VG Name fileserver
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 953861
Free PE 0
Allocated PE 953861
PV UUID rZcmzB-4KSz-Rys6-UxBt-e3vd-ytgf-lDataW

--- Physical volume ---
PV Name /dev/sde
VG Name fileserver
PV Size 3.64 TiB / not usable 3.84 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 953861
Free PE 905
Allocated PE 952956
PV UUID UIIdfQ-ny1X-V2i6-8ihC-WUyF-rUSP-za2nKW

root@pve:~# vgdisplay
--- Volume group ---
VG Name pve
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 7
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 232.63 GiB
PE Size 4.00 MiB
Total PE 59554
Alloc PE / Size 55497 / 216.79 GiB
Free PE / Size 4057 / 15.85 GiB
VG UUID jTvGHO-Iisf-m3At-jy1d-IhCx-u4k8-wF9ASm

--- Volume group ---
VG Name fileserver
System ID
Format lvm2
Metadata Areas 5
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 5
Act PV 5
VG Size 18.19 TiB
PE Size 4.00 MiB
Total PE 4769305
Alloc PE / Size 4768400 / 18.19 TiB
Free PE / Size 905 / 3.54 GiB
VG UUID DlgAV9-eF9I-Dxlv-5G4o-7bkz-L0ed-ljcdHf

root@pve:~# lvdisplay
--- Logical volume ---
LV Path /dev/pve/swap
LV Name swap
VG Name pve
LV UUID zvouKA-G4sg-Vez1-Z3uM-doxp-QNOS-38XkqW
LV Write Access read/write
LV Creation host, time proxmox, 2017-08-23 05:13:28 -0500
LV Status available
# open 2
LV Size 8.00 GiB
Current LE 2048
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1

--- Logical volume ---
LV Path /dev/pve/root
LV Name root
VG Name pve
LV UUID vlyO0Z-uw05-8YUm-3VYS-Bp3l-zKtk-ouqayG
LV Write Access read/write
LV Creation host, time proxmox, 2017-08-23 05:13:28 -0500
LV Status available
# open 1
LV Size 58.00 GiB
Current LE 14848
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0

--- Logical volume ---
LV Name data
VG Name pve
LV UUID G3fUbL-HFfL-q2K4-REMp-Lwkv-EjCy-03Zwq5
LV Write Access read/write
LV Creation host, time proxmox, 2017-08-23 05:13:28 -0500
LV Pool metadata data_tmeta
LV Pool data data_tdata
LV Status available
# open 0
LV Size 150.64 GiB
Allocated pool data 0.00%
Allocated metadata 0.44%
Current LE 38563
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:4

--- Logical volume ---
LV Path /dev/fileserver/data
LV Name data
VG Name fileserver
LV UUID EmSPgn-Bh06-3ChI-WDFF-5Dnq-4hjx-PHf6OA
LV Write Access read/write
LV Creation host, time pve, 2017-09-20 04:45:58 -0500
LV Status available
# open 0
LV Size 18.19 TiB
Current LE 4768400
Segments 5
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:5

 

[dcsapak]

LV Size 18.19 TiB

you have a lv 'data' in the vg 'fileserver' which takes up your 18.19TiB

 

[cartwright]

I will try to remove that whenever I get a chance and see if that shows the LVM storage I add as empty now however in the pvdisplay I see it saying that the drives themselves say Allocatable yes (but full). I had thought that is why my lv data inside of vg fileserver is claiming to be full.

 

[cartwright]

You were exactly right - thank you so much!!!

 

[stra4d]

What was the solution here? I am experiencing the same issue, have added new disks, they show as empty via 'df -v' but full in the web GUI.

 

[cartwright]

What was the solution here? I am experiencing the same issue, have added new disks, they show as empty via 'df -v' but full in the web GUI.

yeah basically i didn't need to create the LV and I could attach it with all of my storage. Than being said proxmox wasn't a great solution for what I was trying to do so I went with Debian+docker using LVM. Easier and less resource intensive.

Edit: this helped a lot

https://www.howtoforge.com/linux_lvm

 

[stra4d]

It works only if there isn't a logical volume?

 

[cartwright]

yeah I didn't need logical volumes I just wanted a pool of storage and would make my own directories at the fileserver level that i would be attaching the volume group too. think of logical volumes as mini "disks" that you expand and such out of your volume group which is likely an aggregate of multiple disks.

I'm no expert here though.

 

[jeffwadsworth]

What was the solution here? I am experiencing the same issue, have added new disks, they show as empty via 'df -v' but full in the web GUI.

Note the size of the Volume Group "Fileserver" and the size of the Logical Volume "Data" in the case above. It allocates all the space.

 

[stra4d]

I guess my understanding of the logical volumes is not complete. I was under the impression that you needed a physcial volume, and then a logical volume for LVM (my experience comes from CentOS where you take the physical disk, add a physical volume then a logical volume and then your system can use that space). How does the LVM disk/setup function without a logical volume?

 

[jeffwadsworth]

The logical volume sits on top of the volume group. You must have a volume group before you have any logical volumes.

 

[stra4d]

Ok I figured out my issue - I had the info in the storage.cfg file twice somehow.

dir: local
        path /var/lib/vz
        content backup,vztmpl,iso

lvm: localLVM
        vgname pve
        shared 0
        content images,rootdir