Should I install Proxmox on NVMe SSD or HDD?

[akcapak]

I have 2 NVMe SSDs and 2 HDDs in my Ugreen NAS. I will be using:
- the HDDs for my cold data (Media files) and VM Backups
- and SSDs for my Proxmox VMs and LXCs.

Both in RAID1 and ZFS.

The Question is, where should i install Proxmox VE? in HDD or NVMe?

Thank you for your help in advance.

 

[guruevi]

Your OS should be on SSD.

 

[leesteken]

Proxmox VE itself runs fine from a HDD (but don't use SMR) as it writes much more (logs, graphs) than it reads. Put (most of) your VMs and containers on (enterprise) SSDs as they compete for IOPS

 

[zodiac]

…but don't use not SMR) as it writes much more (logs, graphs) than it reads

It bears stating that for most users in a homelab, all these logs are pretty unimportant. There unfortunately isn't any control in the UI to turn off persistent logging, but it's not incredibly difficult to do so from the command line. That might make this decision easier for a home user. Where a stock install of ProxmoxVE would destroy a consumer-grade SSD in a matter of months, disabling the logs can extend this time to many years (depending on what other things you run on your hypervisor).

Of course, if you are a large enterprise user and rely on these logs, then you'd plan your hardware purchases around these requirements.

 

[UdoB]

Both in RAID1 and ZFS.

What does that mean? In ZFS context we call it "mirrored" ;-)

The Question is, where should i install Proxmox VE? in HDD or NVMe?

Install on HDD only. After initial setup add both NVMe as a mirrored "Special Device". Search for that term here in the forum, it has been mentioned several times...

Have fun

 

[Johannes S]

It bears stating that for most users in a homelab, all these logs are pretty unimportant.

I beg to differ: Home users often rely on community support (like here in the forum) for that it's important to have logs to post. For this reason I think it's a good thing people needs to use the command line because that ensures that they think about the pros- and cons before.

Install on HDD only. After initial setup add both NVMe as a mirrored "Special Device". Search for that term here in the forum, it has been mentioned several times...

I wouldn't do this to be honest because the mirror will still be quite slow for VM access.

OP has following hardware constraints:

I have 2 NVMe SSDs and 2 HDDs in my Ugreen NAS. I will be using:
- the HDDs for my cold data (Media files) and VM Backups
- and SSDs for my Proxmox VMs and LXCs.

So he has only four slots so he can't add another SSD pair for VM/LXcs. What should be possible though is to partiton the SSDs in two parts so OP could have VM storage and a special device on the same storage media:
@akcapak You could do something like this
- Install the OS to the HDDs, use the HDDs also for cold data (ISO images, LXC-Templates, bulk data ). You don't want to store the backups there though because in case of a broken device you would loose your backups too.
- After installation partition your SSDs in two partitions (one very small, between 0,05 to 2 % of your HDD capacity, the rest is the second partition).
- Add the small partions as a mirrored special device to the HDD pool
- Create another mirrored ZFS pool out of the large SSD paritions for VM and LXC storage.

The benefit of this setup would be that your bulk data access will still profit from your SSDs while maintaining the performance of a SSD mirror for the VMs and LXCs. Since most data from the OS will still be written to the HDD the endurance of your SSDs will still be great. The downside is that your setup is mroe complex, less flexible due to the partioning and might be more error-prone. For this reason I would just go with two pools:
- Install the OS on mirrored HDD (aka RAID1)
- Build another mirrored ZFS pool after installation out of the SSDs.

 

[UdoB]

I wouldn't do this to be honest because the mirror will still be quite slow for VM access.

Everything is a compromise with some tradeoff. And for a lonely general purpose system with only one pair of fast devices and one pair or rotating rust that would be my way to go.

- After installation partition your SSDs in two partitions ...
- Add the small partions as a mirrored ...

This sounds good. But this time I would not recommend it. It complicates the storage layout massively - from my point of view. And some people here in the forum might not have the knowledge to handle a device failure reliably and with a safe hand.

As usual: there is more than one way to skin a cat! ;-)