Recommendations for Proxmox local Filesystems (2 standalone nodes maybe moving to cluster later)?

[n1nj4888]

Hi All,

Proxmox seems to offer a multitude of options for filesystems on the local disk(s). I started off by installing the default options on my 1 Node standalone server and noticed it configured the local 500GB SSD as follows:

local - Directory - 94GB: /dev/mapper/pve-root (according to "df -h", GUI states mounted as /var/lib/vz)
local-lvm LVM-Thin: 338GB

I’ve also configured an NFS mount to my NAS for ISO and vzdump backups which are scheduled to be backed up locally and to the NAS.

I read (https://pve.proxmox.com/wiki/Storage) that the local-Directory and LVM-Thin storage types (A) Support qcow2 (File) and raw (Block) formats respectively and (B) neither are shared storage (although there is a "Shared" checkbox (not currently enabled) for the "local - Directory" storage type in the GUI?) and (C) Snapshots are only supported on qcow2 (local-Directory) and LVM-Thin (block) formats

Therefore I have a few questions please:

(1) If the VMs are running locally on the SSD for speed, would local-Directory (qcow2) or LVM-Thin (raw) be preferable?

(2) For local VMs, I'd like to be able to snapshot the VMs and possibly backup to local storage before shipping to the NAS NFS mount (rather than backing up to the NAS direct which I assume would be slower)? Therefore would local-Directory (qcow2) or LVM-Thin (raw) be a better choice for this, assuming that the backup file contains the previous local snapshots of that VM?

(3) Is there a way to re-partition / change the storage types from the GUI once Proxmox is installed or does it require a reinstall of Proxmox?

(4) If I’m considering experimenting with a cluster by adding 1-2 nodes in the future, would it be better to start with a shared filesystem type (such as CephFS or Ceph/RDB) at the outset and, if so, which is better?

Appreciate any pointers / insight!

Thanks!

 

[Binary Bandit]

Hi,

I can't speak to all of your questions but I can to your future cluster.

3) Yes, there are plenty of options in the GUI.

4) We're running a three node cluster (3 x Dell r510s) with Ceph for the file system. Ceph is very nicely integrated into Proxmox right out of the box. There are multiple tutorials (Google away) explaining how to do this. If your hardware is capable you will have no issues running Ceph right on the same three nodes that are running your VMs. Ceph will let you get away from monolithic / big-up-front-cost storage arrays. It's well worth the learning curve.

Note that Proxmox requires a minimum of three nodes to form a cluster. Google a bit here to see why / what your options are.



James

 

[n1nj4888]

Thanks for the Reply...

I now have two physical (standalone) nodes both with a local 500GB SSD and with the default install storage options of the following:

local - Directory - 94GB: /dev/mapper/pve-root (according to "df -h", GUI states mounted as /var/lib/vz)
local-lvm LVM-Thin: 338GB

While Ceph sounds interesting, it seems to be based on multiple network disks and so I assume performance would be impacted over using local SSD storage (with backups going to an NFS share)?

I'm starting to think that moving to local ZFS would be a good idea since this would seem to allow (A) Running the VMs off the local SSD storage for speed (B) ZFS Replication from node1 to node2 ... but have a couple of questions:

(1) To change the current local Dir/LVM-Thin partitions to ZFS I assume I will have to reinstall PVE on both nodes? Or is there a way to migrate to ZFS post PVE install?
(2) ZFS Replication in the GUI seems to suggest a minimum of 2 nodes are required - I have two nodes but they are not clustered so I assume I would have to join the two nodes to a cluster to use local ZFS <-> local ZFS replication across the nodes? Or is there a way of getting the two nodes to see each other for ZFS replication without being part of a cluster?
(3) If I have to join the two nodes to a cluster, I'm assuming I would also need to add a 3rd node (or a corosync qouorum device such as a Raspberry Pi) to ensure the cluster remains stable?

Thanks!