[SOLVED] Disk format and configuration 2x Proxmox HV with NetApp SAN Storage via FC

[agolmar]

Hi all,

We are in the process of updating our production servers and plan to start with two HPE servers equipped with 32G 2P FC HBA adapters, connected to a NetApp MetroCluster SAN storage. The storage setup provides approximately 3TB on each of the two LUNs, which are configured with multipath.

My question is: Will I encounter any issues with storing ISO files, creating VMs, or using the snapshot function on a ZFS pool if I decide to format the LUNs with ZFS?

I don’t anticipate any hardware limitations, as the servers are relatively new. However, if additional information is needed, please don’t hesitate to ask, as I’m not deeply familiar with storage-related topics.

At the moment, I’m considering either LVM or ZFS, but I’m open to better alternatives if they provide improved performance or data security. Performance and loss prevention are critical factors for us. As far as I understand, the MetroCluster setup is mirrored, which already provides data loss prevention at that layer. Additionally, we have a separate backup system in place at another location.

Another topic later on might be the redundancy between the Hypervisors (later on...).

Thank you in advance for your insights and recommendations!

 

[bbgeek17]

Hi @agolmar , welcome to the forum.

We are in the process of updating our production servers and plan to start with two HPE servers equipped with 32G 2P FC HBA adapters, connected to a NetApp MetroCluster SAN storage

Does this ^ imply a PVE HA cluster? Based on your requirements of data availability and redundancy stated later in the post, I am going to assume the answer is "yes".

As such, a 2 node cluster is not a safe/approved production configuration for PVE. You are running a very significant risk of "split-brain" situation in case of single node or network outage. You must run an odd number of nodes in the cluster.
You can substitute a "qdevice" for a 3rd vote.

ZFS pool if I decide to format the LUNs with ZFS?

Again, assuming you are running PVE Cluster and your LUNs are "shared" across the nodes (i.e. available to all nodes at the same time), you can NOT use ZFS. ZFS is not a Cluster Aware Filesystem. Your only supported option is LVM.
You will not have PVE snapshots with LVM used in a "shared storage" application.

As far as I understand, the MetroCluster setup is mirrored

A MetroCluster is usually deployed across 2+ data centers. Are you planning to have the two aforementioned servers split across the data centers?
Presuming the answer is "yes":
- your 3rd PVE vote should be located in a 3rd data center
- For Hypervisor Live VM failover both PVE nodes (with qdevice) must be in the same PVE cluster.
- That means the data access from node2/DC2 to active storage in DC1 will traverse network back and forth.

Designing a highly available and DR ready infrastructure is an involved task, I'd recommend experimenting a lot and reviewing relevant documents and/or videos.
You can simulate what you are trying to build virtually in a single hypervisor.

Cheers.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[agolmar]

Hi @bbgeek17 , many thanks for the warm welcome and your help!

Sorry for my late reply – I took some time to get my first hands-on experience with Proxmox.

I feel like I understand it much better now after reading your post and studying Proxmox all day yesterday... haha.

You can substitute a "qdevice" for a 3rd vote

In that case, we might have some RPis and also the possibility to set up a VM as a QDevice. But my question would be: what happens if the QDevice fails? Will that impact the hosts?

Again, assuming you are running PVE Cluster and your LUNs are "shared" across the nodes (i.e. available to all nodes at the same time), you can NOT use ZFS. ZFS is not a Cluster Aware Filesystem. Your only supported option is LVM.
You will not have PVE snapshots with LVM used in a "shared storage" application.

That assumption is correct, and I totally missed that ZFS is not a CAF. That makes it really difficult to accept. I looked up a few posts about this, and I think in the worst case, we will have to accept the situation or migrate to VMware later on. But first, I will try to get my team’s input to see if we really need snapshots or if we can handle this with a good/fast backup infrastructure.

A MetroCluster is usually deployed across 2+ data centers.

From what I can tell, those are at least in two different buildings on one campus, but they have a backend mirror at another location. We are connected with each host to both buildings, similar to the standard NetApp setup.

Many thanks

 

[bbgeek17]

But my question would be: what happens if the QDevice fails? Will that impact the hosts?

Since, hopefully, you will still have 2 hosts left our of 3 - you will have the majority in the cluster. As such, the nodes should continue to function, given you time to address the failure.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[agolmar]

Hmm, just as I thought. That’s fine since it’s temporary until the third host arrives.

I have another question, which might be a bit off-topic: What are the best/recommended backup solutions for PVE HA Clusters, especially for Linux machines and some databases?

I’m leaning towards using the built-in backup function, but I don’t have much experience with it yet.

Many thanks in advance @bbgeek17

 

[bbgeek17]

I have another question, which might be a bit off-topic: What are the best/recommended backup solutions for PVE HA Clusters, especially for Linux machines and some databases?

I’m leaning towards using the built-in backup function, but I don’t have much experience with it yet.

You have 3 choices:
PVE built-in backup. It requires no additional software, however lacks in isolation that is required for many organizations. I.e. it runs on the hypervisor
PBS - a separate product from Proxmox team. It allows you to have backup redundancy: outside of hypervisor and on a remote site.
Veeam - a 3rd party commercial product.

There are few other Veeam-like products but I am not familiar with them.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[agolmar]

Hey @bbgeek17 ,

many thanks for your help. We got everything sorted well and come back if there are new questions.

For now, we can close this topic off.

Have a good weekend and take care!