Storage concepts and Best practices

[javieitez]

Hi there,

I'm testing a small Proxmox cluster with 2 nodes (+1 qdevice for quorum) and I'm trying to decide on which would be the best option for storage.

I have access to iSCSI fibre storage, which will be shared between the two nodes. I already know how to configure LVM and the ABC of Linux partitioning, but I feel overwhelmed with the amount of options available: Ceph, LVM-Thin, ZFS, and the like... The more I search and read, the more I get undecided.

I'd need to put to practice

• 2 or 3 running nodes which will run Linux and Windows guests.
• Shared iSCSI storage for VMs
• Snapshot support

backup will be decided after the cluster is running, I don't know if it's relevant at this point.

Which would be the recommended storage configuration for this setup? I'm just looking for some general guidance, not precise step-by-step instructions.

Thank you in advance for your time & Knowledge

PS: I'll accept it if you send me to RTFM, but please point me to a precise manual section (or a tutorial) to start with

 

[Hubguru]

https://pve.proxmox.com/pve-docs/chapter-pvesm.html

For a simple cluster setup with shared storage for HA, NFS server, super simple setup and reliable.

 

[bbgeek17]

The table of features located here https://pve.proxmox.com/wiki/Storage is your primary reference guide.
It tells you which storage can be used as Shared storage and what features it has, or missing.

Ceph, LVM-Thin, ZFS,

None of these can be used with iSCSI based array because they were not designed for it. All 3 are for storage that is local to the node.

• 2 or 3 running nodes which will run Linux and Windows guests.
• Shared iSCSI storage for VMs
• Snapshot support

1) it doesnt matter how many nodes you have or what guests you run
2) you will likely be able to use your existing storage. However, since you did not name it - there is no way to say for sure.
3) yits very likely that you will NOT have snapshot support with your shared iSCSI storage

You may want to investigate ZFS/iSCSI option. Whether you are able to use it will depend on your storage head. In 95% of cases - you will NOT.

Good luck

---

Blockbridge : Ultra low latency all-NVME shared storage for Proxmox - https://www.blockbridge.com/proxmox

 

[javieitez]

https://pve.proxmox.com/pve-docs/chapter-pvesm.html

For a simple cluster setup with shared storage for HA, NFS server, super simple setup and reliable.

Thank you for the feedback... It's definitely the easier option, but the manual says this

Storage Features​

NFS does not support snapshots, but the backend uses qcow2 features to implement snapshots and cloning.

I'm not sure what this means... Am I able to take snapshots if a VM disk is hosted at a NFS share? Does it work, but with limitations?

 

[Dunuin]

I'm not sure what this means... Am I able to take snapshots if a VM disk is hosted at a NFS share? Does it work, but with limitations?

It depends on the format of the image files you are storing your virtual disks on that NFS share. "raw" images won't support snapshots. "qcow2" does.

 

[javieitez]

You may want to investigate ZFS/iSCSI option. Whether you are able to use it will depend on your storage head. In 95% of cases - you will NOT.

I'm currently testing this solution and it seems to work: storage is shared, snapshots work... the configuration is a bit of a pain and it doesn't come back immediately after a reboot, so I'm not sure if it's ideal for a production system. A lot of manual config is made under the hood, and I'm not sure if future updates can easily break this.

Still investigating, thank you for your help