My first Proxmox-install: How to best set up the disks given beginner's hardware limitations?

[trollbot13]

Dear experts,
I am going to take the plunge soon and install my very first Proxmox system.
Target hardware is an not-so-new mini-PC (M900 tiny) with 16 GB RAM, one NVMe-SSD (128 GB) and a SATA-HDD (256 GB), all of it used consumer-grade.
Intended usage is as a small home server in my home network for file sharing, media server, and other "minor" stuff.
The data for that will reside on an external btrfs-formatted USB disk.

After reading the documentation and on the forum, I believe that for my first steps and to save on wear and tear of the internal SSD, it would be best to use ext4 and LVM for my initial setup.
Things may change later, once I gain experience, speed becomes more important, a second node joins in, ...
In this I would prefer to have everything "system" on the SSD for fast response times, and swap and VM / containers on the HDD.
I am not yet very much concerned with boot time of the guest systems and would like their logs not to eat up the SSD cells.

The docs state "The installer creates a Volume Group (VG) called pve, and additional LogicalVolumes (LVs) called root, data, and swap, if ext4 or xfs is used.", so everything would end up either on a single disk or spread around over both - situations I would like to avoid for the reasons above.

So, in your experience, is there any sense in what I am trying to do?
And if so: How do I convince the installer not to put everything into a single VG?
From what I understand, the only reason to use LVM in my situation at all is Proxmox's standard of setting up the VM and containers as virtual disks using LVM functionality?

If my ideas are total nonsense: What would you suggest instead?
I would have liked to use btrfs, like I do on my PC, but read a lot here about problems, required workarounds, tech-preview status etc., so I decided to stick with "the standard" - and ZFS with 16 GB RAM would probably not solve my issues either.

Thanks a lot in advance for your help with my first steps in the realm of Proxmox.

 

[Onslow]

Welcome to the Forum, @trollbot13 !
Be warned that I'm not an expert ;-).

I don't remember if the installer allows this (you'd have to enter the "Advanced options" and see).

If not, after installing you can remove the thin-pool LV (called data) and extend the root LV and use it e.g. for backups.

And on the second disk create another PV and VG and a thin pool. Then create a storage using this pool.

See https://pve.proxmox.com/pve-docs/ for details.
Good luck!

 

[odhiambo]

I found this video very useful. I am as new as 2 weeks with Proxmox!
https://www.youtube.com/watch?v=KMNS_JoHFhg&t=1197s - Just ignore the netbird bit.

 

[trollbot13]

Welcome to the Forum, @trollbot13 !
Be warned that I'm not an expert ;-).

I don't remember if the installer allows this (you'd have to enter the "Advanced options" and see).

If not, after installing you can remove the thin-pool LV (called data) and extend the root LV and use it e.g. for backups.

And on the second disk create another PV and VG and a thin pool. Then create a storage using this pool.

See https://pve.proxmox.com/pve-docs/ for details.
Good luck!

Thanks for the friendly greeting and your reply.
Will take a look if the installer allows for splitting the VG as you describe. Have not worked with LVM before, but should be able to manage.
One question, though: How would Proxmox know to use the manually created "data"- and "swap"-LV on the second disk, instead of looking for them in the original location? They are in a different PV and VG, after all - or is only the name of the LV relevant?

 

[trollbot13]

I found this video very useful. I am as new as 2 weeks with Proxmox!
https://www.youtube.com/watch?v=KMNS_JoHFhg&t=1197s - Just ignore the netbird bit.

Thanks for the video, will take a look.
Glad to see I an not the only newbie here.
How do you like Proxmox so far? Any pitfalls to report that I could avoid?

 

[Onslow]

Will take a look if the installer allows for splitting the VG as you describe. Have not worked with LVM before, but should be able to manage.

Not splitting the VG in fact. The original VG (called "pve") should still use only the first disk (SSD in your setup).
And the second disk (HDD in your setup) is to be used for the second VG (you can call it whatever you want, e.g. vg2).

One question, though: How would Proxmox know to use the manually created "data"- and "swap"-LV on the second disk, instead of looking for them in the original location? They are in a different PV and VG, after all - or is only the name of the LV relevant?

Sorry, I forgot you also wanted to move swap to the HDD.
You can remove also "swap" LV in the "pve" VG.
And create it in the vg2. No, the name of the LV on its own is not relevant nor sufficient.
Of course change the swap's location in the operating system.

For using the new "data" thin-pool you must create a storage pool
(https://pve.proxmox.com/pve-docs/chapter-pvesm.html#_storage_pools).

I advise that after fresh installation you copy all the /etc (and directories below it) to some other place, like a USB stick - to have a reference pattern. Generally you want to recreate the configurations but in other places / disks.

 

[trollbot13]

Of course change the swap's location in the operating system.

Stupid me - of course I need to mount the LV via fstab.
Now, seeing this: Is there any reason at all to include swap in a LV?
Swap is fixed size anyway, so why not put it separately on the HDD and use the rest of the disk for the second VG, "data" only?
If I had my way, there would be no LV on the SSD either, since it only houses "normal system" files that can well live on any btrfs or ext4 filesystem.
As far as I understood the documentation, the only reason for using LVM here is that the virtual disks are easier to place into a LV - please correct me if I am wrong.
And in my setup any virtual disk would wind up on the HDD in the "data" LV.
So neither the system-SSD nor swap need LVM in any way - if only there was a way to convince Proxmox to use plain ext4 or btrfs for root.
Is there?

 

[Onslow]

Now, seeing this: Is there any reason at all to include swap in a LV?

E.g. for flexibility.

Swap is fixed size anyway,

Not quite. You can later swapoff, change its size, mkswap, swapon.

so why not put it separately on the HDD and use the rest of the disk for the second VG, "data" only?

Again, for flexibility. Yon can later change your mind, remove the swap LV, make it elsewhere and use the released space for something else.

If I had my way, there would be no LV on the SSD either, since it only houses "normal system" files that can well live on any btrfs or ext4 filesystem.

As above. BTW, you can occupy a VG gradually. I.e., not give all the space for root LV, but step by step, having a reserve for something other when you need it.

As far as I understood the documentation, the only reason for using LVM here is that the virtual disks are easier to place into a LV - please correct me if I am wrong.

Not only. See that link about storage types.

And in my setup any virtual disk would wind up on the HDD in the "data" LV.
So neither the system-SSD nor swap need LVM in any way - if only there was a way to convince Proxmox to use plain ext4 or btrfs for root.
Is there?

This I'm not sure. But the modern approach is using LVM also for root "partition", not only in hypervisors.

P.S. I forgot about one more advantage (because it may be not available in a computer with no empty slots).

Having LVM, you can expand a LV (and the filesystem on it) using more than one physical drive!
You just add another PV to the VG and you have more space also for existing LVs.

 

[trollbot13]

E.g. for flexibility.

I agree ... although in my (probably special) case LVM seems just an extra layer without any significant benefit, as I don't need any flexibility.
The SSD will be filled with the system's files, keeping 8 GB on HDD (iirc the recommendation is half the physical RAM in Proxmox?) reserved for swap won't hurt even in the long run, and if anything, I will need to swap out the complete HDD one day for a larger one (or SSD), so no adding of anything to any LVM-construct anyway. All that flexibility LVM brings is pretty much wasted on me (as of now) and the only advantage (compared to btrfs) is that it's block-oriented (can store large raw images) and non-cow, so high-frequency changes are potentially not as problematic as they might be for btrfs.
I do understand, though, that most other people will most likely benefit much more from the storage options Proxmox offers, so the design choices seem sound. It's my use case that currently does not really fit the solution, as it does not require something that flexible and "professional level".
Nevertheless I would like to give it a try (and gain room for later "growth"), just as simple as possible for my first steps. Don't want to get frustrated and give up, just because the OS is "too good for me" (and ZFS too challenging for both my bare metal and brain).
So thanks again for helping me along. I guess there is quite a learning curve waiting for me.

 

[Onslow]

Don't want to get frustrated and give up

LVM is quite easy to understand when you imagine that VG is something like a disk and LV is something like a disk partition.
And VG is built of physical volume(s) which can be physical disk(s) or partition(s) of physical disk(s).

As a benefit you are not restricted by the order (a sequence) of LVs, contrary to being restricted by the order of partitions (you can't expand the not last partition without moving the last partition beforehand, which can't be done without rebooting to some "live CD" tool).

LVs and filesystems on them can be expanded without unmounting them and without stopping the operating system.