Cannot extend volume group ( over iscsi ).

[Pigi_102]

Hello all.
I have a cluster ( 3 nodes ) for testing purpose, connected to a TrueNas, experimenting several storage backend.
One of those storage is an iscsi lun where I have created an LVM.
The lun ( FWIW ) is seen from nodes as /dev/sdc and has not been partitioned ( I'm quite sure I did everything from the GUI... )
Now I have expanded the LUN from 20G to 40G and this can be seen from vgs:

root@pve1:~# vgdisplay vg-iscsi
  --- Volume group ---
  VG Name               vg-iscsi
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  8
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               19.99 GiB
  PE Size               4.00 MiB
  Total PE              5118
  Alloc PE / Size       0 / 0   
  Free  PE / Size       5118 / 19.99 GiB
  VG UUID               nvaG5c-pyq1-YRJC-n9H4-Yasl-QQvy-hMwsVP

and as you can see, the free space has been "seen" from the sdc side but not from the vg side.
Being that there is not a ( real ) lvm over it, I cannot do an lvextend -l100%FREE as I would in a normal lvm.
I've tried to move over this lvm a disk from a virtual machine sligltly bigger than the actual size, but smaller than the "extended" size and I got a
"Volume group "vg-iscsi" has insufficient free space (5118 extents): 6656 required"

How am I supposed to use this new space ?
I rather not delete and create the lvm as I think it'ìs not the right way to do this...

Thanks in advance.


Pigi_102

 

[bbgeek17]

Hi @Pigi_102 ,
You may want to look at this article : https://kb.blockbridge.com/technote/proxmox-lvm-shared-storage/
While it does not specifically address extension case, it does illustrate the LVM structure. You will notice that before the VG there is something called Physical Volume. As you extend your available space, you need to do it at each layer.

There are many articles online that will guide you, for example: https://azvise.com/2020/09/03/linux-extend-lvm-partition-after-resizing-disk/

Cheers

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Pigi_102]

hello and thanks for your reply.
I know quite well how an lvm works, ( I'm RHCSA and RHCE from long time .... ) the physical disk has been extended and that's why the PE free are
Free PE / Size 5118 / 19.99 GiB

what's missing is how to get those Free PE seen by proxmox being that there is not a logical volume to extend. Only lvm to be created when creating a disk.... but it seems that lvm(metad ? ) is not able to understand that.
Easiest way would be to destroy the VG and recreate it but it sound a bit hard to me

 

[bbgeek17]

There must be some confusion then.

and as you can see, the free space has been "seen" from the sdc side

We don't see that, as you did not include that output (lsscsi -ss ; lsblk /dev/sdc)

Now I have expanded the LUN from 20G to 40G and this can be seen from vgs

Thats not what your output appears to show:
VG Size: 19.99 GiB - The total size of the volume group.
PE Size: 4.00 MiB - Physical Extent (the smallest allocatable unit) is 4 MiB in size.
Alloc PE / Size: 0 / 0 - No PEs have been allocated yet, meaning no Logical Volumes (LVs) have been created in this VG
Free PE / Size: 5118 / 19.99 GiB - All 5118 PEs (5118*4=19GiB) are available for allocation.

VGs are _always_ based on PVs. Do a pvs/pvdisplay to see them. You can place PV directly on raw device, i.e. /dev/sdc. Most people partition the /dev/sdc first, meaning that you have to extend the partition first.

Good luck!

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[bbgeek17]

I think you are still confused.
- You said that you are working with /dev/sdc
- You said that you are working vg-iscsi implying that it is based on /dev/sdc

Your latest output shows :
sdb / 40G and it is the basis for vg-iscsi
sdc / 60G and it is the basis for vg-bck

You also misunderstand what "Free PE" means - they are already part of the VG, they are just not used. This makes sense as there is nothing on vg-iscsi, as confirmed by pvs
I would take a step back and carefully review everything again...

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[Pigi_102]

No, sorry. My fault. I've removed the post as I have taken the data after a reboot and son device name has changed.
I'm rebuild the environment as it was to start clean and will post as soon as I have all as before.
Sorry again.