Disk configuration recommendations please

louie1961

New Member
Jan 25, 2023
5
1
3
I am running Proxmox VE 7.3-3 on an old repurposed HP Z640 workstation (Xeon E5-2690v3 chip, 48 GB of ram, and three drives all connected by SATA III: a 256GB SSD currently used for the boot drive/proxmox root, a 1TB SSD drive used for VM and CT storage, and 1TB HDD used only for ISOs CT templates currently-all formatted as EXT4). This is not a production box, and I have no issue wiping the drives and starting over. I am using this box for home lab and tinkering purposes only at the moment. I only run it a couple of hours a day and it is not doing anything super important. My NAS runs on different hardware at the moment which is running 24x7. I have 1 TB of storage on the NAS served up via SMB/CIFS and backed up locally every 15 minutes and offsite to AWS S3 every night. I may at some point migrate the NAS to this box and also spin up VMs or docker images to run pfsense, openVPN, and a few other services on a more permanent basis.

SO, with all that being said, what are your suggestions for how to configure the storage on the HP Z640? I am guessing some kind of LVM would be better than what I am doing. I am not sure I am ready to take the leap into ZFS yet, but maybe, who knows? I am still pretty new to Proxmox. This box has been running for only a couple of weeks now, and I have installed a few VMs (windows 11, Linux Mint 11, Kali, and Ubuntu server). I have also installed docker a couple of different ways (in Ubuntu and in an LXC container), and wordpress in an LXC container. SAS is not currently an option (as far as I know) since there isn't an SAS adapter in the machine (that I am aware of)
 

LnxBil

Famous Member
Feb 21, 2015
6,780
899
173
Saarland, Germany
SO, with all that being said, what are your suggestions for how to configure the storage on the HP Z640?
You are in the optimal position to try out every storage option there is including ZFS and compare if it fits your requirements. There is no one solution for all, but generally ZFS is a VERY good option to play around with and get familiarized. It may not fit your requirements, but you have to check that.

Running Docker inside of LX(C) containers does not offer much better isolation than just running it directly on your PVE host (for security always run it in a VM - has been discussed a lot on the forums). So with that beeing said, try out Docker directly on your PVE host WITH ZFS, so that you can benefit from all ZFS glory while running Docker. This is a game changer with respect to quota, file isolation, snapshots, compression etc. - althrough I would not run this in production, but you already said, that this would be a test system.

Having PVE on its own disk has the advantage, that you can play around with different storage types on your other disks and beeing able to wipe / reinstall it easily. Again, this may not be a good production setup, but that depends on a lot of other factors and again ... just a test system, so this is totally fine.
 
  • Like
Reactions: louie1961

louie1961

New Member
Jan 25, 2023
5
1
3
Great info and recommendation, thanks. But one follow up. Would you install ZFS on only one of the disks (say for example the 1TB SSD) or would it get installed across all three disks, or something else? I am kind of wondering how to best take advantage of the mix of hard drives and mix of SSD vs HDD disks. The HDD disk is a 7200 RPM Seagate Baracuda, with the 64MB cache. So not as fast as the SSD drives, but not terribly slow either.
 

Dunuin

Famous Member
Jun 30, 2020
8,990
2,310
156
Germany
Great info and recommendation, thanks. But one follow up. Would you install ZFS on only one of the disks (say for example the 1TB SSD) or would it get installed across all three disks, or something else? I am kind of wondering how to best take advantage of the mix of hard drives and mix of SSD vs HDD disks. The HDD disk is a 7200 RPM Seagate Baracuda, with the 64MB cache. So not as fast as the SSD drives, but not terribly slow either.
Depends on what you call slow. For a VM/LXC storage you most care about IOPS and here HDDs are terrible slow and a SSD can easily be 100 or even 1000 times faster.

Mixing SSDs and HDDs is generally not a good idea. I would use those disks individually. The 1TB SSD for VMs/LXCs and the HDD for cold storage or backups.

I personally would replacethe 250GB SSD with another 1TB one, then create a mirror with those 2x 1TB SSDs and use them for both PVE system disk and VM/LXC storage and use the HDD for a PBS datastore.
 
Last edited:
  • Like
Reactions: louie1961

louie1961

New Member
Jan 25, 2023
5
1
3
I personally would replacethe 250GB SSD with another 1TB one,
That's just not in cards right now. I am going to have to work with what I have. I currently using the 1TB SSD for VMs/LXCs and the HDD for ISOs and backups as suggested.

Is it worth trying to capture the open space on the 256 GB SSD and making it into an LVM with the 1TB SSD?
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get your own in 60 seconds.

Buy now!