Proxmox shared storage with snapshoots

[MateuszK]

Hello,

Like many others now, I’ve been considering migrating from VMware. I’ve reviewed the documentation, but I’m looking for a straightforward answer. I have a storage array that only supports iSCSI. What options do I have for achieving functionality similar to VMFS— a shared disk resource accessible by multiple hosts with snapshot capabilities ?

 

[LnxBil]

Like many others now, I’ve been considering migrating from VMware. I’ve reviewed the documentation, but I’m looking for a straightforward answer. I have a storage array that only supports iSCSI. What options do I have for achieving functionality similar to VMFS— a shared disk resource accessible by multiple hosts with snapshot capabilities ?

If you only look at supported options according to the documentation and your current hardware: you have none.

 

[Johannes S]

The native ISCSI support doesn't support snapshots at the moment, see
https://pve.proxmox.com/wiki/Storage

To get a shared storage you would create a LVM on the ISCSI:
https://pve.proxmox.com/wiki/Storage:_LVM

LVM doesn't supports snapshots either though.
Depending on your usecase you might not need it though. The migration guideline suggests to use Proxmox Backup Server for backups and live-restore to achieve a similiar result:

Alternatives to Snapshots
If an existing iSCSI or FC storage needs to be repurposed for a Proxmox VE cluster, the following alternative approaches might fulfil the same goal.

Use a network share (NFS or CIFS) instead of iSCSI. When using qcow2 files for VM disk images, snapshots can be used.
Rethink the overall strategy; if you plan to use a Proxmox Backup Server, then you could use backups and live restore of VMs instead of snapshots.
Backups of running VMs will be quick thanks to dirty bitmap (aka changed block tracking) and the downtime of a VM on restore can also be minimized if the live-restore option is used, where the VM is powered on while the backup is restored.
-- https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE#Alternatives_to_Snapshots

Please note that ISCSI and LVM storage are used as block storage for raw images, not filesystem. The benefit is (at least in theory, maybe somebody has a benchmark? I never did one myself ) a higher performance since the overhead of the filesystem is avoided.
If you want to use a filesystem the only supported option at the moment is ZFS over ISCSI, but this is supported just for few specific ISCSI targets since several requirements need to be met (which most existing implementations don't have):
https://pve.proxmox.com/wiki/Storage:_ZFS_over_ISCSI

However since Proxmox VE in the end is a Debian Linux you could use any filesystem supported by Debian including Oracles cluster file system ocfs2. With it you can then use the qcow2 image format which supports snapshots.
This is however not supported offically nontheless people in this forum are using it. There are some caveats though so I would carefully think about whether the potential benefit is actually worth it. Search in the forum for ocfs2 to get an idea what's possible or not.
Some service providers like Heinlein Support uses ocfs2 for some of their customers so it's a possible way but not without potential problem. Proxmox Server Solutions will still support you in case of non-storage-related issues though.

In any case you could setup a minimal Proxmox cluster in your existing vmware with enabled nested virtualization as virtual machines to try out the diffent options with your storage array. The guest performance obviouvsly won't be great but for a simple test or proof of concet it should be enough.

 

[MateuszK]

Thanks a lot for your in-depth answers.

 

[bbgeek17]

Depending on your usecase you might not need it though. The migration guideline suggests to use Proxmox Backup Server for backups and live-restore to achieve a similiar result:

Note that fast backup/restore based in dirty bitmap depends on VM staying powered on 100% of the time. There are plans to preserve bitmap to survive the shutdown, however, afaik, they are not implemented yet.

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[kellogs]

ceph

 

[bbgeek17]

ceph

Hi @kellogs,

Op said they have an existing iSCSI appliance they would like to re-use. Ceph is designed to be used with local server storage. Its a great tool but not a solution for every problem.

Best

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox