HA Enterpise

If you are scared of eBay, you can just contact companies directly. There are many hardware resellers out there that offer warranties and support for used/off-lease hardware. In the US, for example, I've used Vibrant for 20+ years, primarily for Dell hardware. I've even had them do ownership transfers then added or extended the warranty and support through Dell directly. Dell will typically provide hardware warranty support for seven years past the last sale date. You can also purchase technical support separately if you want to handle hardware repairs yourself. For example, they offer per-incident support.

Used versus new is a fine argument, but anytime someone is concerned about cost, I will suggest used. And with regard to reliability, there just isn't a difference between the two. You can have high failure rates with new hardware too. Redundancy is required to be built in regardless.

The sealed, OEM Windows licenses on eBay are legit. I've been buying them for decades. Far better than the online key sellers since you get a physical disc and genuine MS sticker with the activation key.

Ultimately, my reason for mentioning Windows Server 2025 Datacenter was due to Storage Spaces Direct, which can be used with Microsoft's virtualization platform to build a hyper-converged cluster. The haters will chime in, but the fact is there are plenty of people running this and it's a valid solution.
 
@janus57 and @alexskysilk

Perhaps I wasn't clear.
The question in this thread is: "Can I replicate the functionality offered by a SAN using Proxmox today, given that SAN vendors stubbornly refuse to provide direct support for Proxmox?"

While running VM disks over NFS on an external ZFS server is technically possible, it lacks the throughput and low IOPS latency required to compete with a SAN.
To get comparable performance and resiliency, ZFS pools must reside directly on the hypervisor hosts using local storage and cross-node replication. Consequently, achieving HA this way demands a much higher overall disk footprint than a traditional centralized array.

This is why, compared to a traditional SAN, ZFS-based setups generally require more disks.

Since you are compareing Ceph to a SAN. I feel that you dont realize the price of a good speed all flash SAN with full PSU redudancy, switch redudancy, controller redudancy a-s-o.
If you have 4+ hosts with 24x 2.5" in the front... then instead of buying for example 24 slot SAN, with a price of what? 25-50.000dollar ex disks? or something. Get 4x extra network cards and put 8 none-oem disks in each host and woops... you have the SAN for free?

How much storage and what speed of the SAN are you looking for?