Promox Design / Dell Powerstore / Pure Storage

A

Adelscott

New Member
May 30, 2025
2
0
1
Hello everyone,

Like everyone else, I'm looking to switch to Proxmox for my production environment.

I'm considering implementing the following architecture:

Photos_DJ1o0QQjTy.jpg


My question is, does the metro volume work with Proxmox under Dell PowerStore?

Same question, but with Pure Storage arrays and active cluster mode.

Best regards,
 
The more impactful factor would be how you would quorum and manage PVE across a stretch configuration. PVE doesn't have any provisions for this, and you would need to deploy some kind of stonith at the first layer cluster (with pve clusters being the second.) If you can have that addressed the storage part would just work. It can be done, but not as part of the PVE toolset.
 
Adelscott said:
My question is, does the metro volume work with Proxmox under Dell PowerStore?
Click to expand...
Looking at the documentation, the Metro Volume operates in active/active mode across both sites, allowing concurrent reads and writes. From PVE perspective, it has no awareness of this underlying behavior. If you stretch your PVE cluster across both sites, the Metro Volume will simply appear as a shared LVM volume to Proxmox.

In this setup, PVE treats the Metro Volume as any other block device - it's just an application using shared storage.

The primary concern, as @alexskysilk pointed out, is the risk of split-brain. While Dell does implement certain safeguards to mitigate this, their documentation mentions the possibility of requiring manual recovery in failure scenarios.

There are simply too many variables to predict how successful you will be in recovering from a worst-case situation.

Thoroughly test failure scenarios before moving to production. If you didn't test it - assume it is broken.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: Johannes S
Hello,

Yes, split brain, whether in Proxmox, VMware, or HyperV, will be there.
It must be as secure as possible, tested, validated, and fully understood.

Thank you all.
 
sorry in advance if not related to this topic, but - https://github.com/kolesa-team/pve-purestorage-plugin

passed through this "community plugin" to integrate purestorage to pve, as today pve do not have a proper way to setup the multipath storage directly without LVM / split-brain caveats on the mix.

For now we are using ovirt only because of its support of the bulk storages, and trying to migrate to proxmox as it offers more resources for our environment, but how to fully integrate the power and resources of purestorage is still a challenge as its way expensive to discard.

warding here to follow the discussion of this storage integrations
 
  • Like
Reactions: Johannes S
We're currently discussing a similar setup to this.

Adelscott said:
Yes, split brain, whether in Proxmox, VMware, or HyperV, will be there.
It must be as secure as possible, tested, validated, and fully understood.
Click to expand...
What did you conclude?


bbgeek17 said:
The primary concern, as @alexskysilk pointed out, is the risk of split-brain. While Dell does implement certain safeguards to mitigate this, their documentation mentions the possibility of requiring manual recovery in failure scenarios.
Click to expand...
AFAIK, the two PowerStores need a witness, as shown in the network topology diagram. This is linux based and could also host a QDevice for running the PVE cluster.

In case of a failure of the direct communication between the two metro nodes, the witness will give the prefered location of the metro volume a heads up and if it replies in time, the non-preferred site will be disconnected and will go offline, so if you're running a kind of streched cluster, it may be a good idea to have metro volumes, one preferred on each side and run the VMs on that site on the preferred metro volume. With this, you will have - if you stricktly split the domains - no outage if the connection will fail. This may be a bit strange, because this kind of not uses the active-active paradigm, but I fear that this is what you meant with the problems of this setup. In the time between the network outage occurs and the non-winning side is disconnected, you may already have a split brain of your data. This can IMHO only be mitigated by using the system as non-active-active.

(Another option would be to have a network ring configured between the three datacenters, if the latency is low enough. You could then route the traffic over the witness path.)

What are your thoughts on this?
 
  • Like
Reactions: Johannes S
Hi @Adelscott , I'm planning to set up a design similar to yours. May I ask if Proxmox is compatible and works well with Metronode and PowerStore?
 
Hello,

Has anyone implemented and tested this combination of Stretched Proxmox Cluster with Dell Powerstore Metro?

Regarding to this article - Metrovolume should work fine with FC or iSCSI (LVM thick):

https://infohub.delltechnologies.com/en-au/l/dell-powerstore-metro-volume-1/introduction-5117/

PowerStoreOS 4.0 adds the support of SCSI-3 reservations and registrations for Metro Volume, enabling it to be used in a standalone Linux host and clustered Linux hosts configuration.

PowerStore manages the synchronization of data and SCSI reservations across PowerStore clusters. In addition, PowerStoreOS 4.0 allows grouping several volumes into a single Metro Volume Group, allowing them to be managed in a logical unit.

Metro Volume is a storage-level feature that requires no modifications to the Linux hosts or applications. It is a stretched volume spanning across two PowerStore clusters. Each Metro Volume consists of two volumes, one on each cluster, synchronized bi-directionally over a replication network.

Important to consider for the 3rd site (Tiebreaker):

Unlike corosync itself, a QDevice connects to the cluster over TCP/IP. The daemon can also run outside the LAN of the cluster and isn’t limited to the low latencies requirements of corosync (<5ms)

If the QNet daemon itself fails, no other node may fail or the cluster immediately loses quorum. For example, in a cluster with 15 nodes, 7 could fail before the cluster becomes inquorate. But, if a QDevice is configured here and it itself fails, no single node of the 15 may fail. The QDevice acts almost as a single point of failure in this case

BR
 
Hi @PoschMox93 , welcome to the forum.

At a high level it appears that this might work with PVE. Keep in mind that for shared storage PVE does not employ either SCSI3 reservations or any of the items listed here: https://infohub.delltechnologies.co...ume-1/linux-applications-with-metro-volume-3/

PVE with shared storage is also not on the above supported list, so you may be on your own in case of an issue.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
LnxBil said:
but would you not consider (thick) LVM as supported shared storage in PVE?
Click to expand...
OPs storage is a not a simple block target. the comment was likely encompassing the entirety of the storage solution. If all you want is a block target it will work fine for that purpose, and is in the supported application matrix, but considering the cost of said solution that would be akin to buying a 747 to go to the grocery store.
 
  • Like
Reactions: Johannes S
alexskysilk said:
OPs storage is a not a simple block target. the comment was likely encompassing the entirety of the storage solution. If all you want is a block target it will work fine for that purpose, and is in the supported application matrix, but considering the cost of said solution that would be akin to buying a 747 to go to the grocery store.
Click to expand...
Sure, but what option is there for a metro volume that is cheaper? We use it heavily with oracle databases over fiberchannel with various vendors.
 
LnxBil said:
Maybe I don't get yout point, but would you not consider (thick) LVM as supported shared storage in PVE? That would be a match in my book.
Click to expand...
These are the items listed there:
  • Native Linux MPIO // yes, this one matches, but in PVE with shared storage it is used in combination with LVM
  • Regular Linux Logical Volume Manager on standalone host // does not match - PVE uses LVM but not as standalone host
  • ext4 and xfs file systems on a standalone host // does not match
  • Storage pool for KVM // unclear what exactly they mean here, but likely not PVE type storage pool
  • Clustered Linux Logical Volume Manager // CLVM is not used with PVE
  • GFS2 cluster file system // Not an officially supported PVE option
  • Red Hat High Availability Cluster // not a match
  • SUSE Linux Enterprise High Availability // not a match

When I mentioned it was not a supported option, I meant the PVE way of using storage is not a supported option for Dell/EMC

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
Last edited:
  • Like
Reactions: Johannes S
bbgeek17 said:
  • Native Linux MPIO // yes, this one matches, but in PVE with shared storage it is used in combination with LVM
  • Regular Linux Logical Volume Manager on standalone host // does not match - PVE uses LVM but not as standalone host
  • [...]

When I mentioned it was not a supported option, I meant the PVE way of using storage is not a supported option for Dell/EMC
Click to expand...
Oh, I see. Thank you. With that strict list in mind, you're right. This list would however apply to any storage vendor as the only viable PVE option is mpath+lvm in such a setup, yet it would still work without a problem as you already said.
 
  • Like
Reactions: PoschMox93, bbgeek17 and Johannes S
Hello everyone,

Thank you for the explanation.

As @LnxBil already mentioned, I would use LVM as shared storage on an iSCSI or FC LUN with native MPIO.

Dell Metrovolume is an active/active design. The physical block LUN is distributed between two data centres, which are readable and writable at both locations.

I am a DELL partner and will give you feedback how the PoC works.
 
You must log in or register to reply here.
Top Bottom