Best practice for setting up a Node with only one drive?

[David511]

I have set up a Datacenter Cluster with two nodes and one Backup Server with a reverse proxy and several webservers, each on its own VM. Each node is running on a re-purposed embedded firewall device with only space for a single 512G drive (a M.2 SSD). The Proxmox Backup Server is running on a server with one SSD and three 4TB drives set up as ZFS RAID Z1 for ~7TB available.

With the default partitions on both nodes, and not being able to add additional physical SSD's, I have not found a way to use the benefits of shared storage. Also, with the VMs all located on LVM-Thin, could not shrink an oversized VM, even after using gpart to shrink the partition inside the VM; qemu-image resize did not work for me with LVM-Thin.

I am wondering if there is a better way to set up the storage partitions, volumes etc... perhaps using only 100GB for the LVM partition and then using the unallocated space on the drive for a different partition, storage pool or...? I am only using about 20GB for VMs right now. Could it be better to have a different partition plan or to rebuild nodes with a different storage system? Appreciate any ideas or suggestions. Most of the examples I read on the forums talk about adding more drives, but I am pretty much limited to one physical drive per node....

Current node setup (x2):

 

[LnxBil]

but I am pretty much limited to one physical drive per node..

Best practise is to not have such a setup, in any case. Get better hardware.

Imagine what will happen if one node fails. You need to have worst-case answers for this before going into production with such a setup:
- when was the last backup? What were the current changes? how big is the data loss?
- what is the time-to-recovery?

 

[David511]

Great points about better hardware, but in this use case I think the hardware is acceptable. I recently shutdown and sold the full-size rack servers (but kept the rack UPS, of course :0) and am glad to be using less electricity with these repurposed low power units. The backups are run nightly, and I have extra units available if I want to add a third node and make it a high reliability cluster. The websites have low traffic usage and are static HTML, and changes are weekly or monthly and so I think a failure of the single drive on a node is not a big issue, and time-to-recovery of one day is acceptable.

So, with the low power systems that I have, is there a better way to set up the partitions on the one drive?

 

[bobmc]

I don't think you can shrink a VM disk, the resize option only ever allows you to increase the size. You can technically use LVReduce to shrink logical volumes but not if the filesystem is mounted. As this is the only drive in your system that's not going to be easy. You'd have to boot from live media.

You can specify the disk usage during the Proxmox install if you only wanted to allocate, say 200G and retain 300G for other uses so that might be your best option

 

[David511]

That's along the lines of what I am thinking... so I think I'll try setting up third node, but with a non-default partition scheme (maybe 100GB for the OS and some additional VMs and CTs and then, what to do with the rest of the disk...? ...use LVM and LVM-Thin, or...?

From reading the PVE documentation, it looks like a local LVM and LVM-Thin cannot be shared, but maybe ZFS can be shared? And the Storage ZFS page says "It is recommended to create an extra ZFS file system to store your VM images".... which I did not see when initially building the nodes...

 

[Dunuin]

That's along the lines of what I am thinking... so I think I'll try setting up third node, but with a non-default partition scheme (maybe 100GB for the OS and some additional VMs and CTs and then, what to do with the rest of the disk...? ...use LVM and LVM-Thin, or...?

From reading the PVE documentation, it looks like a local LVM and LVM-Thin cannot be shared, but maybe ZFS can be shared? And the Storage ZFS page says "It is recommended to create an extra ZFS file system to store your VM images".... which I did not see when initially building the nodes...

ZFS can share the space dynamically, LVM+LVM-Thin not (or only the guests inside the LVM-Thin pool).