Help with Proxmox Cluster Storage Configuration

abl

New Member
Jul 22, 2024
3
0
1
Hi guys,

I really like Proxmox and I kind of convinced my employer to move from RHEV to Proxmox. So now here I am in the middle of the configuration. So far everything went well and was simple to setup except of the storage configuration.

Environment:
We have two datacenters which are connected with 25GB WAN connection. The cluster is spanned over both datacenters and HA is enabled. There is one Storage System (Oceanstor from Huawei) in each datacenter which have 2 Controllers each (in total there are 4 IP addresses to connect to 2 Storage systems).

We really want to have the snapshot feature in order to be able to - of course - take snapshots but also to be able to create full (and incremental) backups without stopping or suspending the virtual machines.

I have read the proxmox documentation and walked through a lot of already existing forum entries, but I have not yet found a reliable solution.

This is my current understanding:
- LVM Thin provisioning is not working, because it cannot be shared accross multiple Proxmox Hosts
- LVM over iSCSI (+ dm-multipath) would work, but does not offer snapshots
- GlusterFS (on top of iSCSI + dm-multipath) is not recommended because GlusterFS support ends December 2024
- ZFS over iSCSI is not an option (unfortunately)
- Ceph would be good, but is not intended to be used on a storage box with RAID configured and actually used for independent disk on each Proxmox Host.

Possible solutions I discovered so far but couldn't make it work:
- SMB/CIFS multichannel
- NFS multipath
- NFS with a LoadBalancer in front (not yet confirmed if the Oceanstor offers this option)
- Directory solution on iSCSI multipath with any file system (in case one Proxmox Hosts is "down", but the VMs are still running and another Proxmox Host will start the missing VMs due to HA it could lead to corrupted VM disks because two Proxmox Hosts are writing in the same time on the same VM disk)

What would be the recommended solution to attach the shared storage to all hosts with the possibility to use snapshots? And if possible, how can I configure it?
 
What would be the recommended solution to attach the shared storage to all hosts with the possibility to use snapshots? And if possible, how can I configure it?
I concur with the previous poster, your most logical choice is OCFS2, if you can make it work.
If your existing SAN can provide NFS, thats also an option. However, if you have to build an external NFS cluster for this - its not worth the trouble.

Good luck


Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: abl
Thank you a lot for your replies!

We are awaiting an official statement, if our storage appliance supports a NFS Loadbalancer. If this is not the case, we will - for now - do LVM over iSCSI + multipath without the snapshot feature.
 
Hey guys,

I had a thought on this topic again and I wonder if I can just use iscsi+multipath with a ext4 formatted lv on it. I could then just share this as "directory storage" within the cluster. Is there any reason why this is not recommended to do?

Thanks in advance for your help!
 
Ext4 is not clusterable. Pve configure lvm cluster wide and will create e New subvolume for every vm. Inside of it u can use ext4.
 
  • Like
Reactions: UdoB

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!