1-SSD/4-HDD with 2 VM config

[Zoxfee]

Recently got a decommissioned appliance that I hope to deploy for personal use. Comfortable with Debian but new to Proxmox and I'm looking for advice. The end-goal is to have

• Ubuntu Server (under LXC?), for around-the-clock services (uPNP, bit torrent, backups over network). Using SSH for access or management.
• Windows 7 Home, for occasional use of some windows-only apps. Using VNC (or RDP but only one user will be created on this machine). Not always running.

The appliance storage has been upgraded and currently has the following:

• 1x 60GB SSD (internal, no hot-swap)
• potential second 60GB SSD but requires case modification to accommodate the second drive (UNAS NSC-400 mini-ITX case)
• 4x 2TB 7200 HDD (hot-swappable bays)

The plan is to have the HDDs in raid10 or raidz1 as I need a minimum of ~4TB storage capacity, and have the internal SSD as the host/ISO/VM&LXC source. Second SSD could be used to mirror the first but not exactly a plan or need. There is nothing mission critical planned for this appliance but I will have regular backups on a cold storage device (compressed images for host OS & direct copy of the storage array).
Most of what I have seen via typical installs requires a disk per VM and the host disk could not be used. However, reading of other setups on here I noticed the host disk can be used to mount/serve VMs or, at the very least, store the ISO files. Hosting VMs on the storage array does not seem like a proper config in my mind.

In terms of best use of resources: how could I best configure this system? As planned above, or should I keep a 2TB drive separated from the array for running VM&LXC from it? To keep all the HDDs for storage only would I have to load the second SSD to achieve the end-goal?


Thanks!

 

[aaron]

If you want to run VMs on the disks, don't use raidz. You will have a surprising overhead of parity data due to the fixed block size of ZVOLs, plus the performance of a RAID10 is better in terms of IOPS, which is what you want if VMs access it.

I would configure the 4x 2TB drives into a RAID 10 (2x mirror vdevs) and then depending on how much I trust the SSD either use it as single disk or in a mirror as well.

What kind of SSDs are they? If they should be used to install Proxmox VE on, they will be taxed with the writes of the logs. Cheap consumer SSDs tend to wear out faster.

 

[Zoxfee]

Thanks aaron

I was thinking of running the VMs on an SSD (for performance) but would be on the same disk Proxmox is installed on. If that is a no go I'll bite the bullet and install the second SSD for the VMs. That way the HDD array would be for storage only. Unless I am missing something blatently obvious I feel running the VMs on the disks would be a "shoot yourself in the foot" type of config.

The warning you gave about VMs on raidz: does that only apply when they run from the array or even just having access to it? I hoped the LXC use it for the media serving/storage. If I should keep a separate storage for the Win7 VM then I'll just dedicate an external usb storage to it as the Win7 VM will be used sparingly.

The stock SSD is a Kingston V300, healthy SMART report & zero bad blocks but a couple years of power-on time. The spare I have is a Crucial BX500 with probably a few days of use max as it was cold storage for phone backups.

Is Proxmox host really taxing? If that is the case then I assume it would be ill-advised to run a VM or even a LXC from the same disk as it..

 

[aaron]

Running VMs on the disk PVE is running on is perfectly fine. The only issue you could run into is that it might not be as fast. If the system is important you should obviously have some redundancy in the storage layer.

The warning you gave about VMs on raidz: does that only apply when they run from the array or even just having access to it?

VMs on a raidz suffer from the problem. LXC containers should be ok. There will be a new section in our documentation regarding ZFS performance and sizing factors. But since it is not yet published I can only link to the patch on our mailing list [0]

The stock SSD is a Kingston V300, healthy SMART report & zero bad blocks but a couple years of power-on time. The spare I have is a Crucial BX500 with probably a few days of use max as it was cold storage for phone backups.

Is Proxmox host really taxing? If that is the case then I assume it would be ill-advised to run a VM or even a LXC from the same disk as it..

Since PVE writes the logs to the install disk, these disks need to handle quite some writes. Experience shows that consumer SSDs just cannot handle it that well over a longer period and wear out but YMMV.

You can go ahead and use it, but keep an eye on the smart values.


[0] https://lists.proxmox.com/pipermail/pve-devel/2020-July/044457.html

 

[Zoxfee]

Cheers, aaron, I really appreciate the help with this!

I believe I'll move forward with the second SSD install just to ease the load on the older/cheaper drives

• 1x 60GB SSD for PVE & ISO directory
• 1x 60GB SSD for running VM/LXC
• HDDs in Raid10 with both VM & LXC allowed access for storage
• external 4TB for backups of SSDs & some critical data of Raid10, then cold storage