proxmox external scale up storage

[twixee]

Hello,

I am looking for a SAN storage solution in scale-up configuration if possible in HA for VMs and lxc containers under proxmox.

The ceph solution is very attractive in tandem with proxmox in a scale-out configuration but the hardware I have does not allow me to do it.

For proxmox cluster i have 3 Dell PeR710 servers with plenty of CPU and RAM and just enough disks for the system.

On the other hand for the SAN part I have 2 Dell PeR510 servers equipped with 16TB of SAS disks each with RAID controllers 64GB of RAM with Intel Xeon CPUs.

To connect everything in ISCSI I have Intel dual 10 Gigabit x520-DA2 cards, a 10 Gigabit switch and 10 Gigabit DAC cables.

There are many solutions under linux, BSD and even Solaris like OpenFiler, Freenas, OmniOS there are also more or less free solutions like nexenta, quantastor, zstor, and others.

Are any of the above solutions working well with proxmox for vm and container storage I have seen that most of them use ZFS over ISCSI technologies while others do LVM over ISCSI.


Best regards,

Twixee.

 

[bbgeek17]

"Working well with proxmox", I think, means that you can create/delete/attach/etc storage using Proxmox methods (CLI/API/UI). There is a list of specific storage technologies that work that way, most are listed here : https://pve.proxmox.com/wiki/Storage. These are the storage types supported by Proxmox developers/support. However Ceph is the only type that is fully integrated for HA and end-to-end management.

At Blockbridge we developed a proxmox plugin integration that we maintain and support for our Software Defined Storage solution.

Other storage types, ie ZFS/iSCSI are supported for end-user management, i.e. volume create/delete/attach/snapshot. However the HA needs to be implemented by you and is out of scope of Proxmox support. Same thing for LVM/iSCSI - how you implement iSCSI HA is up to you.

If storage HA is your requirement then you have to research each one of the solutions you have listed (Freenas, openfiler, etc) to determine if you can create HA solution on the hardware you have. It doesnt really matter if you plan to connect Proxmox to it or HyperV.

Again, if you want API/CLI/UI integration with Proxmox - consult the table in the wiki and look at all storage options that are "shared" capable". Keep an eye on "shared&snapshot" capable if thats important to you.

Good luck

---

Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[twixee]

Thank you for your feedback, I looked carefully at the content of the page https://pve.proxmox.com/wiki/Storage.

Indeed I am looking to use proxmox in high availability from end to end to use vm and containers.

I did not know your blockbridge solution, however if I understood correctly it is not a free and open source solution.

Is it possible under proxmox to use nodes only for ceph storage and other nodes for vm and containers ?

In this way I could use for example 3 Dell R510 for ceph storage (my+osb+mds) and at the same time several Dell R710 to run vm and containers.

This article talks about it --> https://www.jamescoyle.net/how-to/1213-ceph-storage-on-proxmox.

Best regards,

Twixee.

 

[bbgeek17]

I did not personally try to run it this way. But theoretically it should be possible. And you found an excellent blog describing how to do it, so I'd say yes - you can design your system in that way.

---

Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox