Proxmox Disc Structure - Philosophy and Flexibility

[99projects]

Greetings.

I have been aware of ProxMox for some years but only recently has my need for virtualization pushed me over the edge and into "lets do this."

I have read along and followed some basic tutorials, but all include some caveats regarding discs, and particularly SSDs. While I'm not averse to spending some money, it is also a limited resource and I would like to understand what constraints truly exist on these discs. This is the official install reference document I'm pulling most my operational understanding from : https://youtu.be/I-e1_CTa4s0

This tutorial lays out 3 disc groups as fundamental:

1. The disc(s) for installing ProxMox
2. The disc(s) for storing ISOs on
3. The disc(s) for storing your Virtual Machines

While I can throw money at some discs, even a small amount, I would prefer to understand why things are required rather than just following. I'm a slow learner but strong doer, so please bear with me.

My core questions revolve around what can be replaced or migrated in a ProxMox install.

Regarding the OS/Hypervisor Install Disc:

• If my ProxMox install disc failed, could I replace it, reinstall ProxMox and import all the existing arrays and appliances from my other 2 sets of discs?

Some people seem to put the ProxMox install on a RAID 1, and if losing this disc means losing everything else, then that makes a lot of sense to me.​

Regarding the ISO Disc:

• How frequently does the OS refer to the ISO storage? Only during VM creation or during VM operation as well?

If they are only used during install, and I currently only use 2 OS's, then would using a 32Gb USB for ISO storage really be that bad??​

• Can I replace the ISO drive upon failure without losing my VMs?

If I can replace this directory without much disturbance, and if it did incur heavy read-write wear, then perhaps using the USB for a month until I drop some money on a better ISO drive would seem pretty feasible.​

​

Regarding the OS Discs:

I dont have many questions on this, but what I understand is that this is where the bulk of the read/write disc durability matters I assume. ZFS is a reputable redundant stable method for this cluster, and while I have some questions of expand-ability, I think those are more fitting for a ZFS Forum/thread.

I'm sorry for the pile of questions, but really I want to learn this system by putting it to use, but I dont have a lot of time to make uncalculated risks with data and setup. I'm hoping to learn from the shared wisdom here and really appreciate your help and insight. I'm excited about getting ProxMox going and hope to learn enough to start contributing to the solutions.

Kind regards

 

[Stefan_R]

This is the official install reference document I'm pulling most my operational understanding from : https://youtu.be/I-e1_CTa4s0

This is not the official documentation. You can find that here, as well as our own Youtube channel.

This tutorial lays out 3 disc groups as fundamental:

1. The disc(s) for installing ProxMox
2. The disc(s) for storing ISOs on
3. The disc(s) for storing your Virtual Machines

While I can throw money at some discs, even a small amount, I would prefer to understand why things are required rather than just following. I'm a slow learner but strong doer, so please bear with me.

While it does make some sense to have seperate OS and VM disks, having a seperate disk for ISO images is a waste (unless you have many many ISOs). You can just store these on the OS disk.

Depending on the size of your deployment I'd personally even recommend against seperating the OS and the VM disks. If you have enough money available to buy multiple disks, simply join them together in a RAID or ZFS array and use the single logical disk with thin-provisioning (i.e. VMs and the OS simply share the available space). Our installer can set up ZFS RAIDZ/mirror arrays natively for example.

If my ProxMox install disc failed, could I replace it, reinstall ProxMox and import all the existing arrays and appliances from my other 2 sets of discs?

Of course, if your VMs are seperate you can reimport them on a reinstall. However, if you use RAID or ZFS mirroring a full data-loss is improbable, and you should always have a full backup available off-site (or at least on a seperate machine) anyway if data availability is of any concern. Besides, if you have two disks, one for VMs and one for the OS, who's to say the VM disk won't fail first? By mirroring them, if one disk fails, both OS and VM data will still be available.

Regarding the OS Discs:

I dont have many questions on this, but what I understand is that this is where the bulk of the read/write disc durability matters I assume. ZFS is a reputable redundant stable method for this cluster, and while I have some questions of expand-ability, I think those are more fitting for a ZFS Forum/thread.

PVE OS disks have enough writes that it should be on an SSD and not, say, on a USB drive, as is common with some other hypervisor solutions. The bulk of the disk IO will probably happen wherever you put your VMs though.