LVM vs LVM-Thin on Shared Fibre Channel LUN (3-Node Cluster) – Best Practice?

P

pvpaulo

Active Member
Jun 15, 2022
60
1
28
Hello everyone,


I have a 3-node Proxmox cluster connected to a SAN via dedicated Fibre Channel (single FC switch).


There is a 2TB LUN presented to all three nodes, configured with multipath on each host, and I intend to use it as shared storage.


Previously, I used iSCSI + multipath + LVM (thick) as shared storage, and the environment worked very well, especially for live migration.


Now that I am migrating to Fibre Channel + multipath, I am evaluating whether I should continue using traditional LVM (thick) or switch to LVM-Thin to benefit from more efficient VM snapshots and backup flexibility.


My questions are:


  1. Are there any known limitations or risks when using LVM-Thin on a shared LUN accessed by multiple cluster nodes?
  2. Can a thin pool shared between three nodes present any locking or concurrency issues?
  3. Is there any official recommendation or established best practice for FC + multipath environments regarding the use of LVM-Thin versus traditional LVM?
  4. For production-critical environments, is it generally safer to keep LVM thick and delegate snapshot functionality to the SAN instead?
  5. What specific precautions should be taken to avoid overprovisioning risks when using LVM-Thin on a shared Fibre Channel LUN?

I would like to confirm best practices before standardizing this setup in production.


Link documentation : https://pve.proxmox.com/wiki/Storage
 
Last edited:
pvpaulo said:
Previously, I used iSCSI + multipath + LVM (thick) as shared storage, and the environment worked very well, especially for live migration.
Now that I am migrating to Fibre Channel + multipath, I am evaluating whether I should continue using traditional LVM (thick) or switch to LVM-Thin to benefit from more efficient VM snapshots and backup flexibility.
My questions are:
Are there any known limitations or risks when using LVM-Thin on a shared LUN accessed by multiple cluster nodes?
Click to expand...

Yes, lvm/thin is not cluster-aware so you will risk a loss of data. It's not supported for a reason, see also the table on https://pve.proxmox.com/wiki/Storage which clearly states that lvm/thin is NOT a shared storage.

pvpaulo said:
Can a thin pool shared between three nodes present any locking or concurrency issues?
Click to expand...
No for exactly the reasons you state (the mechanisms used by PVE for lvm/thick to ensure that no issues with locking and cocurency are occuring don't work there).

pvpaulo said:
Is there any official recommendation or established best practice for FC + multipath environments regarding the use of LVM-Thin versus traditional LVM?
Click to expand...

Yes: Don't use LVM/thin with them.
pvpaulo said:
For production-critical environments, is it generally safer to keep LVM thick and delegate snapshot functionality to the SAN instead?
Click to expand...
Yes or use the snapshot-as-chain feature introduced as technical preview in PVE9. It comes with some caveat though see following pieces by @bbgeek17 :
The PVE doc has also a section on this feature: https://pve.proxmox.com/pve-docs/chapter-pvesm.html#pvesm_lvm_config

An alternative which is often used is to utilice the snapshot-backups and live-restore of the Proxmox Backup Server for typical snapshot usecases (like ensuring that you can rollback a update gone wrong):
https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Alternatives_to_Snapshots
Since this native backup feature of ProxmoxVE uses an internal mechanism of the qemu/kvm hypervisor it doesn't need snapshot support in the storage.

pvpaulo said:
What specific precautions should be taken to avoid overprovisioning risks when using LVM-Thin on a shared Fibre Channel LUN?
Click to expand...

Just don't. Use lvm/thick instead.
 
  • Like
Reactions: bl1mp, bbgeek17, alexskysilk and 1 other person
Hi,
using the snapshot capability of your SAN provider, relies on the consistency capabilities of the SAN and
you will loose the VM hardware configuration of the VM at the time of the snapshot, because that one is only stored in the Proxmox Cluster Filesystem. The native snapshot/backup tools keep care of that, too.

BR, Lucas
 
Hi, just to clarify: the project does not call or depend on SAN-provider
snapshots.

Snapshots are initiated through the native Proxmox snapshot workflow. Proxmox
remains responsible for VM configuration and VM-state orchestration, while the
storage plugin implements the corresponding disk-volume snapshot operation
using LVM/device-mapper objects inside the shared VG.

Therefore, this is not an out-of-band SAN snapshot that captures only blocks
while bypassing the Proxmox cluster filesystem. Native Proxmox backup also
remains supported and is still the appropriate mechanism for independent,
restorable backups.

The SAN only provides the shared block device; it does not manage the VM
snapshot lifecycle in this design.
 
You must log in or register to reply here.
Top Bottom
Back