LVM disk signaling about to fail - need to replace and want to be sure I'll do this correctly

[Chris Olive]

30 years in IT and I never had a drive fail until 2010. Since 2010, seems like every time I turn around I've got a failed drive. Anyway...

Just started using Proxmox and LOVE it, but of COURSE... But my hardware decides it's got to bork on me. I'm using relatively new hardware with a nice motherboard, SATA drives, etc. GRUB decided to bork on me and I have to boot the entire server using a USB for now. I'll address GRUB later now that I can still boot, but I'm beside myself.

The more pressing issue is that local storage had almost NO drive space after my single node installation. No room to upload ISOs and download and store LXC templates. The solution seemed simple enough, Just add a 750gb SATA drive I had laying around, extend the rootfs where all the ISOs and templates were landing and all was well. Love LVM.

Until that 750gb drive started giving me errors. Now I need to replace it, and I GUESS I'll break down and buy a new drive.

SEEMS like all I'd need to do is (a) add my new drive to the existing PV, (b) issue a pvmove /dev/bad_drive, and (c) then vgreduce pve /dev/bad_drive. But I'm also reading that doing this on a live volume is bad. Anyone done this before? I think if this was just a straight Linux box, I'd be a bit more comfortable, but with this having Proxmox working and I'm now dependent on the volumes, VMs and containers running here, I want to double check.

Here you can see what I did. /dev/sda3 was the original 2TB drive I installed Proxmox on. the rootfs (/dev/mapper/pve-root) was almost full out of the gate, so I added /dev/sdb1 and extended /dev/pve/root and left 357gb in reserve. /dev/sdb1 is going bad and I want to replace.

root@pve:~# pvs
  PV         VG  Fmt  Attr PSize   PFree 
  /dev/sda3  pve lvm2 a--   <1.82t      0
  /dev/sdb1  pve lvm2 a--  698.63g 357.50g
root@pve:~# vgs
  VG  #PV #LV #SN Attr   VSize VFree 
  pve   2  22   0 wz--n- 2.50t 357.50g
root@pve:~# lvs /dev/pve/root
  LV   VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root pve -wi-ao---- 453.50g                                                   
root@pve:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
udev                   16G     0   16G   0% /dev
tmpfs                 3.2G   26M  3.2G   1% /run
/dev/mapper/pve-root  446G   96G  331G  23% /
tmpfs                  16G   43M   16G   1% /dev/shm
tmpfs                 5.0M     0  5.0M   0% /run/lock
tmpfs                  16G     0   16G   0% /sys/fs/cgroup
/dev/fuse              30M   20K   30M   1% /etc/pve
tmpfs                 3.2G     0  3.2G   0% /run/user/0

Also, just anticipating... I have two 6gb/s SATA slots, of which /dev/sdb1 (the drive I need to place) is in one. If I add another SATA drive to pvmove over... I don't want it staying in the 3gb/s slot; I want it to replace /dev/sdb1 in the other 6gb/s slot. So I'm anticipating the drive designation will change. I haven't worked out yet how I would address. Maybe someone can address this as well?

I like LVM but I haven't had to muck with it to this extent (no pun intended) and now it's on a system that otherwise is producing wonders for me. The ability to deploy containers as readily as I can on Proxmox has me otherwise over the moon and I can't bork this.

TIA!

 

[wolfgang]

Hi

But I'm also reading that doing this on a live volume is bad. Anyone done this before?

I would do such things always offline - or have a backup from the important data.

I think if this was just a straight Linux box

Proxmox VE is a Debian 10 based so it is a normal Linux Box where you can do what you can do on Debian.

I think the easiest way is to copy the disk with dd and the resize the lv.

 

[Chris Olive]

Well, since the LV that was extended was /dev/pve/root, there's really no way of doing this offline that I know of other than booting from a nice recovery system, doing the reduce, etc. from there, then bringing PM back up again. Not sure why I didn't see this other than I've been underwater lately.