Once i have ZFS - what to do with local-lvm? bonus question regarding cluster creation and recovery

Iacov

Member
Jan 24, 2024
38
0
6
hey

my current setup is one node with a 500gb nvme and a 500gb sata ssd
i use the nvme for the proxmox installation and it was partitioned in roughly 100gb local and 380gb local-lvm
the sata ssd was formatted in xfs and i use it for local backups

i'm thinking about setting up a second node and form a 2-node cluster (with a pi being a q-device) and giving it a 500gb nvme and 500gb sata ssd as well

the more and more i read about it, ZFS seems to be a "better stuited" file system than lvm-thin - and cold migrating a vm without manually moving configs etc seem to require ZFS, right?

but what it local-lvm/lvm-thin used for once you get started with ZFS? it tages nearly 3/4 of my available space and what else could i use it before besides VM images?
what is your recommendation for repurposing this space?

and bonus question - how do i "migrate" from my current lvm-thin setup to ZFS?
backup my vms, delete them, form the cluster, create ZFS on each node and then re-import the vms using the zfs-storage?

and bonus bonus question: i once had an issue with my proxmox installation and had to reformat it. if that ever happens again, could i simply mount the zfx-sata-ssd and be good? or would that mess up the cluster? (i do not intend to use any HA functionality!)

thank you for your time and experience!
 
my current setup is one node with a 500gb nvme and a 500gb sata ssd
i use the nvme for the proxmox installation and it was partitioned in roughly 100gb local and 380gb local-lvm
the sata ssd was formatted in xfs and i use it for local backups

i'm thinking about setting up a second node and form a 2-node cluster (with a pi being a q-device) and giving it a 500gb nvme and 500gb sata ssd as well

the more and more i read about it, ZFS seems to be a "better stuited" file system than lvm-thin - and cold migrating a vm without manually moving configs etc seem to require ZFS, right?

but what it local-lvm/lvm-thin used for once you get started with ZFS? it tages nearly 3/4 of my available space and what else could i use it before besides VM images?
what is your recommendation for repurposing this space?
Save yourself all the work, frustration, data loss and downtime by running all storage with redundancy. I would wipe the nodes and create a ZFS mirror with those NVMe + SATA SSDs. SSDs are consumables that will fail sooner or later (and when not buying enteprise grade SSDs for ZFS its more towards sooner) so mirror them.

if that ever happens again, could i simply mount the zfx-sata-ssd and be good?
No, a guest consists of virtual disk + config files. The config files are stored on the system disk and are then lost unless you backup your "/etc" folder or run a cluster where the configs would be synced across all nodes.

and bonus question - how do i "migrate" from my current lvm-thin setup to ZFS?
Backup + Restore.

or would that mess up the cluster? (i do not intend to use any HA functionality!)
Yes, you would have to remove that failed node from the cluster and rejoin.
 
Last edited:

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!