Disks and partitions best practices questions

[josechu]

Hi,

I've a proxmox homelab running on one minipc with 1 nVME disk 128g. The initial installation I did there some time ago, via CLI, did the partitions on its own as I dont remember choosing much along the process:


Inside the 126,9g LVM:

root@proxmox:~# vgs
  VG  #PV #LV #SN Attr   VSize   VFree
  pve   1  11   0 wz--n- 118.23g 14.75g

Whith the usual content:

root@proxmox:~# lvs
  LV            VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data          pve twi-aotz-- <53.93g             56.37  2.57
  root          pve -wi-ao---- <39.56g
  swap          pve -wi-ao----   8.00g
  vm-100-disk-0 pve Vwi-aotz--   4.00m data        0.00
  vm-100-disk-1 pve Vwi-aotz--  32.00g data        36.94
  vm-101-disk-0 pve Vwi-aotz--   4.00g data        55.64
  vm-102-disk-0 pve Vwi-aotz--   4.00g data        51.66
  vm-103-disk-0 pve Vwi-aotz--   1.00g data        49.16
  vm-104-disk-0 pve Vwi-aotz--   8.00g data        98.23
  vm-105-disk-0 pve Vwi-aotz--   8.00g data        56.50
  vm-106-disk-0 pve Vwi-aotz--   4.00g data        35.40

Proxmox created the 2 LVMs inside:


local, 41,49g, for iso and ct images
local-lvm, 57,90g, for VM disks and CT volumes


Now, my needs are changing and I will probably need more space... planning to add more VMs, and one OpenMediaVault or similar NAS software to be able to have shared filesystems among different VMs/LXCs...

So Im adding a 1 TB new nVME (exact same model as the 128g one, so same performance I assume, but with more capacity), my questions are:
1. does it worth to reinstall from scratch all Proxmox in the new 1tb?
1a. If yes, what is the best approach for partitions? (I would also maintain the 128g disk... probably for any dedicated VM? i dont know whats the best use... or attached PCI passthrough for the NAS software?
2. Or is it just better to maintain Proxmox as it is in the 128g nvme, and just add the new 1tb disk and create there new partitions for Proxmox to use?
2.1 For redoing or adding new partitions, whats the best practice? keep for example volumes/images in the first disk, removing local-lvm and enlarging local; and then create in the 1tb disk a huge local-lvm for vm/ct disks?

Let's see whats the best approach, i would like to avoid reinstalling from scratch in case it doesnt add anything positive in terms of performance or capabilities.

 

[josechu]

Hi, no replies at all ... but, i update here just in case this is helpful for anyone else. Finally I found some time today and decided to reinstall from scratch on the 1tb nvme. Pretty easy process as I have an external PBS node with all vms/lxcs backups. So in around 1h I did the HW change and also the PVE install and instances restoring.

During the installation process, there was a dialog box asking around max size of local partition, as now I have a 1tb disk i have put there 100g as is mainly for vm images, ct templates... and some local backups (my main and usual backups are in external PBS). I was expecting than the rest of the disk (around 899g) would be automatically used for local-lvm (vm disks...etc), but no, its only using 105g.
I see that the new partition table is using the whole disk:


LVM:


LVM Thin:


Some more data:

root@pve:~# vgs
  VG  #PV #LV #SN Attr   VSize    VFree
  pve   1  11   0 wz--n- <952.87g 740.87g
root@pve:~# lvs
  LV            VG  Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  data          pve twi-aotz-- <98.00g             26.73  2.41
  root          pve -wi-ao----  96.00g
  swap          pve -wi-ao----  16.00g
  vm-100-disk-0 pve Vwi-aotz--   4.00m data        14.06
  vm-100-disk-1 pve Vwi-aotz--  32.00g data        30.92
  vm-101-disk-0 pve Vwi-aotz--   4.00g data        36.86
  vm-102-disk-0 pve Vwi-aotz--   4.00g data        53.45
  vm-103-disk-0 pve Vwi-aotz--   1.00g data        44.45
  vm-104-disk-0 pve Vwi-aotz--   8.00g data        92.40
  vm-105-disk-0 pve Vwi-aotz--   8.00g data        43.86
  vm-106-disk-0 pve Vwi-a-tz--   4.00g data        33.48

So... is this done on purpose to have available disk to make other stuff like LVM snapshots...etc? I'd say that the better approach would be to extend now LVM thin from 105g to maximum available. Am I right? or bad idea because of...?

It would be nice to get some feedback around best practices and how this should me dimensioned and partitioned. And also, if my plan to extend thin LVM to maximum is good... then the how-to I guess I'll find on the forum as I have seen some related threads.

 

[Kingneutron]

I would have recommended to keep using your original boot disk, as long as the disk wasn't dying.

You don't need to use the whole disk for LVM, you can limit the installer to say ~256GB, give root FS 40GB and partition the rest of the disk for ZFS. Then you shouldn't have to mess around with resizing

 

[louie1961]

I have done it several different ways, experimenting along the way. I currently have one Proxmox that has a single NVMe drive (1 TB I believe) and everything is installed on the one drive. I also have a node with two NVMe ports, so I put in two identical NVMe drives and mirrored the installation. I also have a third Proxmox node where I have two spinning drives and a bunch of SATA SSD drives. I have installed Proxmox on a set of 256GB drives in a mirrored array then have used separate SSDs in a ZFS mirror (added after install) to run the VMs and CTs, and passed through the spinners to truenas scale. At one time I also had NVMe drives in a pcie adapter (Asus Hyper M.2 adapter).

You can add new storage pools after install, and you will in fact want to experiment to see what works best for you. I generally keep my important data on a separate Synology NAS and share to my Proxmox VMs and CTs using NFS or SMB shares. Same for VM and CT backups. They get stored on Synology. The only stuff I generally keep on my Proxmox nodes is Proxmox itself and the actual running VMs or CTs.