This would be the most integrated method of providing storage within PVE. Proxmox wraps up Ceph installation and management into a neat GUI/CLI. Of course, under the pretty GUI is a complex system that is very different from all the other options listed. Ceph is a HyperConverged solution, in PVE case, and so you need to consider how sharing CPU between your Compute and Storage will influence your workload. You also need to consider whether you have expertise to deal with any arising problems, especially if you dont plan to pay for PVE subscription.
Additionally, two 10G interfaces is a somewhat bare minimum and your Client traffic will have to share with your Storage replication traffic, not to mention Corosync cluster traffic.
You'd be well served by adding additional network interfaces to isolate services. Your performance will depend on the disks that you can support/afford.
- Dell MD3620 SAN
- EMC VNXe3200 FC SAN
- HP MSA 2040
These solutions are essentially equivalent as far PVE is concerned. You called out VNX as FC, but not what type of transport you planned to use with the other two. IMHO FC is not a wise investment at this point, especially for a small environment like yours. You may get away without FC switches, depending on how many client ports each solution allows you to have. But, if you also have to buy FC switches - that would be overkill.
Regardless of which one of the 3 you pick, and whether you go with iSCSI or FC, you will have some limitations to deal with. Specifically:
- none integrate with PVE. Their involvement ends at "present raw disks to Debian".
- no direct snapshot support. Workarounds exist but are complicated
- no support from from Storage Vendor for PVE specifically (unlike ESXi and HyperV for example)
- limited support from PVE for storage issues
If you are looking at external SAN storage for your PVE - take a look at Blockbridge. Compared to the "Storage Cartel" options, we provide:
- native PVE integration via API
- full snapshot support
- integrated thin provisioning
- native PVE HA integration
and much more
https://www.blockbridge.com/proxmox/
Our solution is faster and lower latency than any of the above -
https://kb.blockbridge.com/technote/proxmox-tuning-low-latency-storage/.
We will support you in the entire compute and storage stack.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
