HA Enterpise

S

scontin

Renowned Member
Feb 24, 2009
51
3
73
Given the costs associated with competitors, more and more companies are considering Proxmox.

But as things stand - aside from Blockbridge and Ceph (the latter being hard to recommend nowadays due to prohibitive disk costs) - are there enterprise-grade solutions available for new infrastructure deployments that guarantee stability, High Availability (HA), and snapshots?

Everyone would like at least the same guarantees as the classic nodes+SAN model..
 
Hi @scontin,

Thanks for the mention! You're right, hardware cost is a real problem right now, and flash isn't getting cheaper anytime soon.

I know you asked about options beyond Blockbridge, but this seemed worth mentioning. What we're seeing is that for many shops, reusing the SAN they already own is a practical short-term strategy. That's exactly what we've been working on: improved support for running existing enterprise SANs behind Blockbridge, with native Proxmox snapshots, clones, and HA. This allows folks to migrate seamlessly to NVMe when prices come down, without changing their Proxmox workflows.

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
 
  • Like
Reactions: xybo, Johannes S and UdoB
scontin said:
Ceph (the latter being hard to recommend nowadays due to prohibitive disk costs)
Click to expand...
Is it?

I am running ZFS only. On mirrors, of course. Five nodes --> 10 disks. (Minimum, actually there are many more - but for simplicity...)

Every VM being replicated to all five nodes means that each and every byte is stored on 10 physical drives.

Utilizing Ceph with five nodes and the default rule "three copies, two minimum" only three copies would be stored --> three times the amount of space available = one third of costs.
 
Last edited:
You definitely don't want to make sweeping generalizations, especially about cost, because there are many ways to solve various problems.

For example, if you are a Windows shop you can get Windows Server 2025 Datacenter OEM licenses on eBay for under $100 (16/24/48 cores) which allows unlimited Windows VMs, and then you also have the free Windows Admin Center software which allows you manage it all via a single web GUI.

Enterprise-grade iSCSI SAN hardware is easy to find used on eBay and is a great value.

Older hardware does frequently have the problem of higher power utilization. But I think the 3-7 year age range is still reasonable.

There are also other costs involved such as balancing risk and downtime for workers and customers, and I think that is why there are still plenty of people paying a high price for proven solutions, such as vSphere, Nutanix, etc.
 
@J-Rod I have I made a simple request: purchasing NEW products, is there anything that can guarantee stability, High Availability (HA), and snapshots?
CEOs usually don't buy on eBay...

As far as I know today—October 2026—the only solution guaranteeing "stability, High Availability (HA), and snapshots" is the one offered by BlockBridge.
ZFS allows replication, which by definition is not real-time.
 
UdoB said:
Is it?
Click to expand...
I would say yes. At least of you want similar performance to local storage.
scontin said:
Everyone would like at least the same guarantees as the classic nodes+SAN model..
Click to expand...
scontin said:
guarantee stability, High Availability (HA), and snapshots?
Click to expand...
Really depends on your definiton of HA.
Real HA? Or, "we have a SAN, with a redunant PSU, but non redudant controller, and our nodes are connected through a none redudant way to the SAN" HA?

Since you say "classic model" I would assume the later :)
But yeah, CEPH can offer you that.

Of course, if you care about costs and performance, for many scenarios the way to go would be to make the application HA, not the hypervisor.
 
  • Like
Reactions: Johannes S
In the enterprise,
scontin said:
But as things stand - aside from Blockbridge and Ceph (the latter being hard to recommend nowadays due to prohibitive disk costs)
Click to expand...
Uhh.... you might want to have a conversation with Blockbridge before you make that assumption. Pretty sure they dont give out their product for free. The other thing you might want to consider is that disks is just a portion of the cost- with ceph you will need a minimum of 3 enclosures to house them. In a hyperconverged environment this may not be an issue.

UdoB said:
Every VM being replicated to all five nodes means that each and every byte is stored on 10 physical drives.
Click to expand...
I dont know if you are suggesting this sort of approach for business use; this seems like a nightmare to manage (how do you even know which copy is "master?")

J-Rod said:
you can get Windows Server 2025 Datacenter OEM licenses on eBay for under $100
Click to expand...
J-Rod said:
Enterprise-grade iSCSI SAN hardware is easy to find used on eBay and is a great value.
Click to expand...
You may have not noticed the title of ops post. I dont know about you but I wouldnt be picking grey licenses or decomissioned hardware for enterprise use.

scontin said:
So for now,, Blockbridge or Ceph.
Click to expand...
Note @bbgeek17 response. LVM thick on SAN or qcow over NFS are tried and true, and have been used for cluster stores for decades.

What I would suggest is putting together a RFQ which would include usable capacity and minimum performance criteria and shop around. I bet you'd find quite a few workable solutions to compare.
 
alexskysilk said:
I dont know if you are suggesting this sort of approach for business use;
Click to expand...
No! As so often my current situation is the result of an organically grown system. Restarting today would make me opt for Ceph, but changing a sufficiently well working system is... difficult. (And would require much more redundancy and speed on the network level.)
alexskysilk said:
this seems like a nightmare to manage
Click to expand...
No. Replication runs not very frequently (in my case) and works reliably.
alexskysilk said:
(how do you even know which copy is "master?")
Click to expand...
There is no master. Only once during multiple years I had to clean up manually - by removing stale / not-in-sync-anymore and "no common denominator available" snapshots on one of the multiple destinations.

Besides the architectural weakness my point was that I store everything 10 times on this construct - which is... expensive overkill.
 
Hi,

alexskysilk said:
I wouldnt be picking grey licenses or decomissioned hardware for enterprise use.
Click to expand...
Just to say that we do that for our customers is they ask for "better" price than new hardware and accept a refurbished one as a "short" term solution (at least until 2030/2031), but, of course, we don't buy it on eBay. For software like Microsoft, we use a refurbished certified reseller (EU law has this case cover).
And for hardware we buy from companies like bargainhardware/tonitrus.
But even refurbished hardware prices have gone UP and low stock so...

We see crazy pricing on new hardware that is just insane. A little example with a "simple" SSD like "Micron 5400 MAX 1.92TB SATA" went from 243€ (in Oct 2023) to 2 009€ (in Oct 2026) at the time of writing this message (price x8), and this pricing is for almost all hardware on a server.
So we completely understand was is the feeling of the customers when 5/6/7 years later their server pricing was multiplied by 10.

Best regards,
 
  • Like
Reactions: Johannes S
scontin said:
Given the costs associated with competitors, more and more companies are considering Proxmox.

But as things stand - aside from Blockbridge and Ceph (the latter being hard to recommend nowadays due to prohibitive disk costs) - are there enterprise-grade solutions available for new infrastructure deployments that guarantee stability, High Availability (HA), and snapshots?

Everyone would like at least the same guarantees as the classic nodes+SAN model..
Click to expand...
Hello. Im also new here. But I dont understand ur argument?
Why should the disk costs affect Ceph more than Blockbridge? The disk price is high in every country / vendor / branch?

I argue. The whole point of Proxmox + Ceph is that you can use old hardware (not to old). But as many here sad 3-7 years.
I left VMware with a whole infrastructure on 2019 servers and the fact that Proxmox doent care whats what on the front panel, makes it epic to save money, buying used stuff.

And for your question. Yes there is a third option. QNAP (using Cephfs code beneth) is now a new vendor with HA, Snapshots and "guaranteed stability".

https://www.qnap.com/en/operating-system/quts-mega - 3 Mega Nodes and your up.
 
@AkUsP

How many disks are necessary for Ceph? For NTS? for a SAN?
They're not the same number...

quts-mega has a native integration with Proxmox for snapshot?
 
janus57 said:
We see crazy pricing on new hardware that is just insane.
Click to expand...
Rampocalypse is universal. You either accept the new cost of doing business or hold on to your old equipment for as long as you can. I'm doing the latter where I can ;) Ironically, for the most part, customers only really need to upgrade because the assets depreciated to zero, not because there is additional load or performance requirement. There is a window between 3 years (typical depreciation cycle) and 5-ish years where a platform reaches EOS. beyond EOS many companies have criteria that require replacement.

AkUsP said:
Why should the disk costs affect Ceph more than Blockbridge? The disk price is high in every country / vendor / branch?
Click to expand...
Exactly correct :) hardware is hardware in both cases.

AkUsP said:
The whole point of Proxmox + Ceph is that you can use old hardware (not to old). But as many here sad 3-7 years.
Click to expand...
Yes, you can use generic hardware but that just means you dont get vendor locked. It doesnt mean the need to have compliant, supported hardware goes away. The other part of it is as drives age out, it is commonly (if we ignore 2026) a better choice to replace the whole solution with newer drive technology rather then replaced failed older drives.

scontin said:
How many disks are necessary for Ceph? For NTS? for a SAN?
Click to expand...
disk count isnt a function of storage technology. its a function of your capacity, performance, and fault tolerance requirements. begin by defining those.
 
Last edited:
I mean Ceph and (in part) ZFS need more disks than a traditional SAN.
And in these days disk's costs are insane.
 
scontin said:
I mean Ceph and (in part) ZFS need more disks than a traditional SAN.
And in these days disk's costs are insane.
Click to expand...
ZFS offers the same disk aggregation functionality as a SAN; edit zfs is a disk aggregation mechanism. SAN is a multi initiator storage router. You will need exactly the same number of drives at the same replication or parity levels. In fact, there are SAN solutions that use ZFS for disk aggregation.

ceph does need more drives for replication, so 50% more then ZFS/RAID1. I wouldnt suggest running EC or parity raid for virtualization- but if you are comparing those you'd end up with the same math either way for disk count vs capacity efficiency.
 
Last edited:
You must log in or register to reply here.
Top Bottom
Back