proxmox HA on shared SAN storage

Spiros Papageorgiou

Well-Known Member
Aug 1, 2017
82
0
46
43
Hi all,

I would like to build a 3node proxmox cluster that will use a 3par storage over FC. It is not clear to me, what are my options for setting up the storage. For now, I have setup multipathd on all 3 nodes and I have a LUN that all 3 nodes can see (each node has a dm-x device that corresponds to the same LUN).

1. What would be my best option for storage setup, in order to have HA, live migration,snapshots and backup ?

2. Does LVM over FC supports shared storage and snapshots? The proxmox Storage page says "snapshots"="no" . Is LVM a valid option for my case, with HA and live migration?

3. ZFS over ISCSI seems to support all the options I need. Does ZFS over FC, also supports all the options and works in my case?

Thank you,
Sp

PS: I think the wiki "storage" page should be populated with more info
 
Last edited:
There's no solution to this. I'm in the same situation (but with iSCSI instead of FC). The best you can do is LVM (not thin), but you loose thin provisioning and snapshots (you have live migration and backups though)
 
I gues you not loosing thin provisioning with storage, because the storage itself can do this. For snapshot, yes, that can be missing, but its depending on use case.
 
Most enterprise SAN's has the ability to make snapshots of LUN's so instead of making snapshots in proxmox you make the snapshots on the SAN.
 
Most enterprise SAN's has the ability to make snapshots of LUN's so instead of making snapshots in proxmox you make the snapshots on the SAN.
That's true, but for my case I will present one LUN, that is shared among many VMs (via LVM), so I don't want to snapshot all the VMs at the same time. Each VM should have its own snapshoting capability.

sp
 
ZFS over iSCSI (FC in my case) is mentioned in the "Storage" wiki, as supporting "shared"

ZFS is a non-cluster filesystem. It is constructed that way and it'll never change.

ZFS-over-iSCSI is a ZFS-backed single server which exports its zvols via iSCSI to multiple hosts, but you still have only one host with ZFS, nothing shared. ZFS is your SAN in this setup so you unfortunately cannot use your SAN.
 
I continue to look into the right answer on this. Is the info below still correct?

AmcAZf3WwTU3QTjoo2RSMBS11bVjL4


These would be the only Shared storage options for a HA cluster (aside from NFS and CIFS). If we have clients that are setting up a 3-5 nodes, I would like to settle on a stable and affordable recommendation.
 
These would be the only Shared storage options for a HA cluster (aside from NFS and CIFS). If we have clients that are setting up a 3-5 nodes, I would like to settle on a stable and affordable recommendation.

LizardFS is also working despite the fact that is not listed in your table. It could be used as shared block device .
 
I continue to look into the right answer on this. Is the info below still correct?

AmcAZf3WwTU3QTjoo2RSMBS11bVjL4


These would be the only Shared storage options for a HA cluster (aside from NFS and CIFS). If we have clients that are setting up a 3-5 nodes, I would like to settle on a stable and affordable recommendation.

You forgot LVM as PVE type LVM, level block, shared yes, snapshots no, stable yes
 
Another option is to use a cluster-aware file system like GFS2 or OCFS2, in which case you have all the features (including thin prov and snapshots, brought by the qcow2). OCFS2 is quite easy to setup, but you'll need to test performance and stability before going live with it.
 
Another option is to use a cluster-aware file system like GFS2 or OCFS2, in which case you have all the features (including thin prov and snapshots, brought by the qcow2). OCFS2 is quite easy to setup, but you'll need to test performance and stability before going live with it.
OCFS2 is abandon ware so hardly a candidate for customers running in production!
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!