Dears, I'm preparing to setup 3 node Proxmox Cluster using Dell R740 for our production systems. I am trying to decide between using CEPH storage for the Cluster / Shared storage using iSCSI. Which is the best option for Shared Storage in case of 3 node Proxmox cluster? I need a reliable solution to support live VM migration from one host to another in case of host failure.
Shared Storage Recommendation for Proxmox Cluster
- Thread starter abdulwahab
- Start date
Hi @abdulwahab , welcome to the forum.
Both Ceph and, practically any, iSCSI storage will provide reliable live VM migration.
However, neither will provide live migration in case of host failure. That functionality is not available in PVE/QEMU yet.
The choice between Ceph, iSCSI, or NVMe/TCP comes down to finer details of your use case, budget, skill, location, high availability needs, capacity, etc.
There is no one right answer. As with many things in IT - it depends.
Good luck.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
Both Ceph and, practically any, iSCSI storage will provide reliable live VM migration.
However, neither will provide live migration in case of host failure. That functionality is not available in PVE/QEMU yet.
The choice between Ceph, iSCSI, or NVMe/TCP comes down to finer details of your use case, budget, skill, location, high availability needs, capacity, etc.
There is no one right answer. As with many things in IT - it depends.
Good luck.
Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox
We need the VMs to be highly available. That's the reason looking for a reliable storage solution that can be shared between hosts. For the VMs, storage capacity planned is 12 TB.
I did initial research and I found that iSCSI doesn't support snapshots. Requires the advice from experts who is running small proxmox clusters with budget friendly hardware but without losing reliability.
I did initial research and I found that iSCSI doesn't support snapshots. Requires the advice from experts who is running small proxmox clusters with budget friendly hardware but without losing reliability.
That's not problem as in "if it crashes, it'll be started on another node". The next better option is fault tolerance, which would be to have a standby VM running and syncing all the time on another node and immediately taking over if a host fails. This is currently not possible in PVE and a very restrictive and expensive additional cost on VMware.
As always: "it depends". If you have a storage that is capable of doing this, you can surely have snapshots, e.g. ZFS-over-iSCSI offers snapshots.
If you go with iSCSI, buy a dual-controller box with PVE storage support, e.g. the blockbridge hardware.
The integration of CEPH into Proxmox makes it really easy to setup shared storage. We use a 3-node-Ceph-Cluster only for storage, no virtualization but I would recommend at least 5 nodes for Ceph in terms of availability and resilience.
Another option, like blockbride, would be linstor/drbd. It's also a commercial product and the support experience has been really good. But like it has been said before:
If you have no experience at all, I would recommend to build a cheap lab environment to test out the different possibilities. Three Intel NUCs are a cheap way to setup a small testing cluster.
Another option, like blockbride, would be linstor/drbd. It's also a commercial product and the support experience has been really good. But like it has been said before:
If you have no experience at all, I would recommend to build a cheap lab environment to test out the different possibilities. Three Intel NUCs are a cheap way to setup a small testing cluster.