[SOLVED] How to expand LVM after adding storage space to RAID5

[vale.maio2]

Hi all, I've been searching for a solution for the past week and I'm still running around circles.

I'm running ProxMox 7.2-11 on a Dell T710. The server used to have 8 1GB drives in RAID5 (total space 7TB). When installing ProxMox I've created a 7TB LVM, which I then split into smaller partitions and gave them to my main VM (Lubuntu, if it makes any difference).

I've now swapped the drives on the bare metal with 8 4TB drives still in RAID5, for a total of 28TB (the PERC H700 took care of rebuilding the RAID). On the GUI under disks, I can see the 28TB RAID disk just fine, and lsblk can see that too:

root@server:~# lsblk /dev/sdb
NAME                    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdb                       8:16   0 25.5T  0 disk
|-data-vm--100--disk--0 253:7    0  150G  0 lvm
|-data-vm--100--disk--1 253:8    0 1000G  0 lvm
|-data-vm--100--disk--3 253:9    0 1000G  0 lvm
|-data-vm--100--disk--5 253:10   0 1000G  0 lvm
|-data-vm--100--disk--6 253:11   0 1000G  0 lvm
|-data-vm--100--disk--2 253:12   0  2.3T  0 lvm
root@server:~#

However, when I'm going to the main VM in ProxMox GUI, I can't create a new partition for it, as it still thinks the drive is 7TB in size. From the ProxMox CLI, parted can't even see any partitions on the disk, and I honestly don't remember how I created the various partitions in use:

root@server:~# parted /dev/sdb
GNU Parted 3.4
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Error: /dev/sdb: unrecognised disk label
Model: DELL PERC H700 (scsi)                                            
Disk /dev/sdb: 28.0TB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:
(parted)

How do I go about using the remaining 21TB on the disk? I don't care about downtime (this is just my personal home server), but unfortunately I can't wipe it and start from scratch.

 

[bbgeek17]

Run pvs/lvs/lvs to get a better understanding of the structure.

The google search you are looking for "extend lvm after disk resize"
i.e. https://azvise.com/2020/09/03/linux-extend-lvm-partition-after-resizing-disk/

Read and execute carefully, try to understand why you are running the suggested commands.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[vale.maio2]

Run pvs/lvs/lvs to get a better understanding of the structure.

The google search you are looking for "extend lvm after disk resize"
i.e. https://azvise.com/2020/09/03/linux-extend-lvm-partition-after-resizing-disk/

Read and execute carefully, try to understand why you are running the suggested commands.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Thank you very much for the answer, however that's where I'm getting stuck. Those instructions basically say to expand the partition with parted, which is easy enough. However, parted does not recognize any partition on the disk (parted output in my original post), despite being able to see them in ProxMox GUI.

 

[bbgeek17]

Its possible to not use partition in the first place, which seems like what you did. However we wont know until you provide the information from above commands.
You will probably have to skip some steps and modify others, which is why I suggested you understand the commands prior to running them.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[vale.maio2]

Its possible to not use partition in the first place, which seems like what you did. However we wont know until you provide the information from above commands.
You will probably have to skip some steps and modify others, which is why I suggested you understand the commands prior to running them.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

Sorry I didn't mean anything by it, I just wasn't home so couldn't run the command. Anyway, here's the output:

root@server:~# pvs
  PV         VG   Fmt  Attr PSize    PFree
  /dev/sda3  pve  lvm2 a--  <223.07g <16.00g
  /dev/sdb   data lvm2 a--     6.36t  <7.00g
root@server:~# lvs
  LV            VG   Attr       LSize    Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vm-100-disk-0 data -wi-ao----  150.00g
  vm-100-disk-1 data -wi-ao---- 1000.00g
  vm-100-disk-2 data -wi-ao----    2.30t
  vm-100-disk-3 data -wi-ao---- 1000.00g
  vm-100-disk-5 data -wi-ao---- 1000.00g
  vm-100-disk-6 data -wi-ao---- 1000.00g
  data          pve  twi-aotz--  140.45g             5.32   1.37
  root          pve  -wi-ao----   55.75g
  swap          pve  -wi-ao----    8.00g
  vm-101-disk-0 pve  Vwi-aotz--    8.00g data        93.49
root@server:~#

So I'm guessing I need to either expand the lvm2 /dev/sdb or create a new one, is that correct?

 

[bbgeek17]

you missed the "vgs", but its not critical.

Your entire disk, without partitions, is part of physical volume. As you can see from "pvs" - pvs does not yet recognize that your disk is larger. Only 16g is free.

Here is an example:

We create and attach 16GiB disk:

[centos@features-client ~]$ bb vss provision -c 16GiB --with-disk
b== Created vss: service-1 (VSS1863194C40626440)

== VSS: service-1 (VSS1863194C40626440)
label                 service-1
serial                VSS1863194C40626440
uuid                  47c27035-8e74-40bc-af07-bbe7c76bb41d
created               2022-09-21 13:48:11 -0400
status                online
current time          2022-09-21T13:48-04:00

[centos@features-client ~]$ bb host attach
==============================================================================
service-1/disk-1 attached (read-write) to features-client.localnet as /dev/sda
==============================================================================

We now provision LVM on top of the disk:

[root@features-client centos]# lsblk   
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  16G  0 disk
vda    253:0    0   8G  0 disk
`-vda1 253:1    0   8G  0 part /
vdb    253:16   0   1G  0 disk [SWAP]

[root@features-client centos]# pvcreate /dev/sda
  Physical volume "/dev/sda" successfully created.

[root@features-client centos]# vgcreate vg1 /dev/sda
  Volume group "vg1" successfully created


[root@features-client centos]# lvcreate -l100%FREE -n lv1 vg1
  Logical volume "lv1" created.

[root@features-client centos]# pvs
  PV         VG  Fmt  Attr PSize   PFree
  /dev/sda   vg1 lvm2 a--  <16.00g    0

[root@features-client centos]# vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  vg1   1   1   0 wz--n- <16.00g    0

[root@features-client centos]# lvs
  LV   VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lv1  vg1 -wi-a----- <16.00g

we now expand the disk, notice pvs does not reflect the change immediatelly:

 bb disk resize -c 32GiB
== Resized disk disk-1 (DSK1963194C4062644F): 16.0GiB --> 32.0GiB

[root@features-client centos]# lsblk
NAME      MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda         8:0    0  32G  0 disk
`-vg1-lv1 252:0    0  16G  0 lvm 
vda       253:0    0   8G  0 disk
`-vda1    253:1    0   8G  0 part /
vdb       253:16   0   1G  0 disk [SWAP]

[root@features-client centos]# pvs
  PV         VG  Fmt  Attr PSize   PFree
  /dev/sda   vg1 lvm2 a--  <16.00g    0

Now we resize the PV:

[root@features-client centos]# pvresize /dev/sda
  Physical volume "/dev/sda" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized

[root@features-client centos]# pvs
  PV         VG  Fmt  Attr PSize   PFree
  /dev/sda   vg1 lvm2 a--  <32.00g 16.00g

And finally the LV:

root@features-client centos]#  lvextend -l +100%FREE /dev/vg1/lv1 
  Size of logical volume vg1/lv1 changed from <16.00 GiB (4095 extents) to <32.00 GiB (8191 extents).
  Logical volume vg1/lv1 successfully resized.

Whether this is exactly what you want to do - depends on your goals and needs. There are many different ways to slice this disk.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[vale.maio2]

Sorry, I'm getting a bb command not found error straight away, and I'm not sure what that command is supposed to be so I can't google it.

root@server:~# bb vss provision -c 16GiB --with-disk
-bash: bb: command not found

By the way, here's the vgs output:

root@server:~# vgs
  VG   #PV #LV #SN Attr   VSize    VFree
  data   1   6   0 wz--n-    6.36t  <7.00g
  pve    1   4   0 wz--n- <223.07g <16.00g

 

[bbgeek17]

ok.

Just run:
pvresize /dev/sdb

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[vale.maio2]

ok.

Just run:
pvresize /dev/sdb

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

That was it, hidden in plain sight. I've seen that command so many times, but it never clicked that it was exactly what I needed. I can see now that the total space for the PV is 28TB, and I can finally expand the existing LVs and/or create new ones.

Thank you so much!