CEPH/RBD on FC SAN storage

[NikoBizTech]

Within our company, we want to establish a virtual infrastructure using Proxmox VE as the virtualization technology. We use Commvault as our backup solution.
After consulting the documentation listing the types of storage supported by Proxmox, we have chosen CEPH/RBD. We would like to deploy RBD on an FC SAN ( Hitachi VSP5600 or DELL storage solution).
Is it technically feasible to implement this solution? If so, how should we proceed? What kind of performance can we expect from such an implementation?

 

[bbgeek17]

Hi @NikoBizTech , welcome to the forum.

It is technically possible to do what you want. However, it's neither recommended nor was Ceph designed for it.
You have a Central SAN solution, Ceph is a distributed SDS that was designed for use with local hard disks.

You'd be going against best practices and design goals, so expectations are hard to predict.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[NikoBizTech]

Thanks @bbgeek17 !

OK for CEPH, it is not recommended and not designed to be implemented in this way.
I am a beginner in the field. And if we use RBD on FC SAN storage, would we face the same issue?

 

[LnxBil]

I am a beginner in the field. And if we use RBD on FC SAN storage, would we face the same issue?

If you mean the rados block device, that is what Ceph would expose to your PVE hosts, so yes, the same issues.

Your question has been discussed before (a lot) and your only Proxmox supported option is to run thick-lvm (no snapshot support). Just search the forums for SAN and PVE.

 

[Stefan Wagner]

Hi NikoBizTech,

The Hitachi storage supports iSCSI connectivity when the feature is installed and based on the Proxmox support list, iSCSI with ZFS for block storage supports shared storage and snapshots and it's stable.
https://pve.proxmox.com/wiki/Storage

Please follow the iSCSI multipath recommendation from Proxmox.
https://pve.proxmox.com/wiki/ISCSI_Multipath

 

[LnxBil]

The Hitachi storage supports iSCSI connectivity when the feature is installed and based on the Proxmox support list, iSCSI with ZFS for block storage supports shared storage and snapshots and it's stable.
https://pve.proxmox.com/wiki/Storage

Please follow the iSCSI multipath recommendation from Proxmox.
https://pve.proxmox.com/wiki/ISCSI_Multipath

This is true but can only be used on a single machine and is not shared in the same way as a cluster filesystem would be.

 

[bbgeek17]

iSCSI with ZFS for block storage supports shared storage and snapshots and it's stable.

iSCSI/ZFS requires the storage system to be based/using ZFS internally. I am reasonably confident that Hitachi is not.

Or, as @LnxBil said, you will need a front-end server to proxy FC/iSCSI connections from SAN. This will affect performance and, as mentioned, availability.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox