Converting machine to Proxmox VE server

[DotUniverse]

Hello,

Wanted to ask for suggestions. I'm planning to convert machine with following specs to Proxmox VE server:

CPU: AMD Ryzen 9 5900X
RAM: DDR 4, 32GB
Drives:

• nVme - 500GB
• SSD - 500GB
• HDD 2.5' - 1TB
• HDD 3.5' - 2TB
• HDD 3.5' - 4TB

Initial ideas was to try and use ZFS as I'm really interested to learn how it works (self-healing, deduplication) but going through forum I learned that it's not great idea if you don't have similar drive's size to make any kind of RAID.
Also, for nVme and SSD it's technically possible to make them a ZFS cache but both of them are consumer SSD's so it's also not the best idea.

To be fair this server is not intended to run 24\7, more like an occasional test lab's for: Windows 11, Plex, some Linux distro's.

So, my current plan is to install Proxmox on part of nVme (like 200GB; using LVM) and use all other space / drives for VM's / containers.

 

[LnxBil]

So, my current plan is to install Proxmox on part of nVme (like 200GB; using LVM) and use all other space / drives for VM's / containers.

Why waste fast space with something that does not need speed? Also consumer SSDs are never good for PVE, due to the heavy writing of the PVE internal filesystem /etc/pve and metrics.

Your disks make it hard to play around with ZFS meaningfully as you already laid out. Can't you just buy another disks that has one of your current capacities? If so, put a ZFS mirror and PVE on it.

 

[DotUniverse]

Why waste fast space with something that does not need speed? Also consumer SSDs are never good for PVE, due to the heavy writing of the PVE internal filesystem /etc/pve and metrics.

Your disks make it hard to play around with ZFS meaningfully as you already laid out. Can't you just buy another disks that has one of your current capacities? If so, put a ZFS mirror and PVE on it.

Thanks!

Two more questions:

• If I install PVE on ZFS raid0 will I be able to use free space for VM?
• When I buy same model new HDD should I aim for 2 new drives (as current one's are 2+ years old)?

 

[louie1961]

Here's what I would do, if it was my hardware:

Install Proxmox onto the 1TB spinner, and let Proxmox partition it as per the normal install. The extra space on that disk could be used for storing ISOs, etc. I would set up the nvme and the ssd as independent pools for storing VMs, after the initial install. If you are not running in a cluster, disable the corosync, pve-ha-crm, and pve-ha-lrm services. It will help your consumer drives last longer. I have a number of consumer drives that have been in service for 2 years now that have less than 2% wear out. VMs will make the best use of the faster storage. You can decide to use ZFS even on a single drive and still have snapshots, or you can go with LVM thin with EXT4, its your call.

I would pass through the bigger spinners to a VM running something like openmediavault. You could mash them together into one giant pool and share it out as SMB, NFS or iSCSI shares for data storage in the VMs you arre running. For all my VMs I mount an NFS share for the persistent data I want to store. For docker containers, I do the same thing basically, mounting my volumes on NFS. The beauty shot is that if your VMs/containers are all on the same VLAN as your VM tyhat serves up the shares, the data never has to leave the proxmox host, making it super fast. Running TrueNAS or OMV on the same box as my VMs I have been able to achieve 20-30 GB/second on iperf testing. Granted your actual transfer speeds will be limited by the speeds of the disks, but you don't lose anything by having the data go out to to your router and back Putting your data on the spinners also helps preserve your SSD or NVME drives.

 

[LnxBil]

If I install PVE on ZFS raid0 will I be able to use free space for VM?

Yes and it is a very good setup (besides the raid 0 of course)

When I buy same model new HDD should I aim for 2 new drives (as current one's are 2+ years old)?

No, I'd just buy the same drive once more and add it to the already present zfs pool.

 

[LnxBil]

Install Proxmox onto the 1TB spinner, and let Proxmox partition it as per the normal install. The extra space on that disk could be used for storing ISOs, etc. I would set up the nvme and the ssd as independent pools for storing VMs, after the initial install.

As an alternative, I'd setup both disk types in one pool:

• create the pool with the spinning disk in the installer
• add the SSD as a special device (maybe go for two used enterprise SSDs) and set special_small_blocks