[SOLVED] New Proxmox Server setup - SATAIII SSD and NVME. Which to use for OS and which for VMs?

[andyn]

Hi all,


I'm getting better with Proxmox, but still have some confusion over performance and storage options.

I have some questions for you good people today, and I hope you can save me a lot of testing time.

I'm setting up a new standalone Proxmox node. Won't be clustered. Not for production.
Size and noise are important, so it's a Lenovo M720q tiny.

Intel I5-8500T (6c/6t @2.10GHz)
16GB RAM
Two drives. One a SATA-III SSD, one a PCI-e NVME (about 4 times as fast as the SATA).

Both drives are the same size. It will run a couple of VMs, and both VMs will fit on either disk with room to spare. Both will have about the same I/O requirements and both will have bursts of I/O at the same time. All the I/O is file-based, not database updates or any other kind of many-many-of-them-but-tiny reads and writes.

I could install Proxmox on the SSD and have the NVME as storage, or the other way round.


Question 1)
My guess is that the best performance for the VMs would be to install Proxmox on the SATA drive and use the NVME for VM storage. Can someone confirm this would be the right way round, or is it more important for VM performance that Proxmox and the KVM engine has the faster disk?

Question 2)
What storage would give the best performance?
It think the options are LVM/LVM-Thin/ZFS/Directory (XFS or EXT?)
Speed is the primary consideration, with Snapshots a nice to have.

Thanks in advance.
Andy.

 

[Dunuin]

Hi all,


I'm getting better with Proxmox, but still have some confusion over performance and storage options.

I have some questions for you good people today, and I hope you can save me a lot of testing time.

I'm setting up a new standalone Proxmox node. Won't be clustered. Not for production.
Size and noise are important, so it's a Lenovo M720q tiny.

Intel I5-8500T (6c/6t @2.10GHz)
16GB RAM
Two drives. One a SATA-III SSD, one a PCI-e NVME (about 4 times as fast as the SATA).

Both drives are the same size. It will run a couple of VMs, and both VMs will fit on either disk with room to spare. Both will have about the same I/O requirements and both will have bursts of I/O at the same time. All the I/O is file-based, not database updates or any other kind of many-many-of-them-but-tiny reads and writes.

I could install Proxmox on the SSD and have the NVME as storage, or the other way round.


Question 1)
My guess is that the best performance for the VMs would be to install Proxmox on the SATA drive and use the NVME for VM storage. Can someone confirm this would be the right way round, or is it more important for VM performance that Proxmox and the KVM engine has the faster disk?

Question 2)
What storage would give the best performance?
It think the options are LVM/LVM-Thin/ZFS/Directory (XFS or EXT?)
Speed is the primary consideration, with Snapshots a nice to have.

Thanks in advance.
Andy.

If you care about data integrity and availability I would use ZFS and mirror them. Then you only get 40% of the raw capacity and the NVMe would be slowed down to SATA performance but atleast you won't loose data and the server will continue running when a disks dies. And your data would be protected against bit rot, so your data won't silently corrupt over time without you noticing it until its too late and don't got any healthy backup because all healthy backups were overwriten by corrupted versions of the data meanwhile.

If you are not interested in data integrity or availability and just want good performance and a long SSD life expectation I would use the SATA SSD with LVM + xfs for the OS+backups+ISOs and the NVMe SSD with LVM-Thin as the VM/LXC storage.

 

[andyn]

MAny thanks, XFS on the SATA disk for OS and NVME for the VMs it is. Availability isn't an issue, as this is a lab, so I'm always sat next to it if it's running.

 

[mrpbennett]

I am in the same boat here, waiting for my M720q 9500T to arrive. What NVMe drive did you go with? I have done a ton of research and everyone's opinion is different. This is just for a homelab situation so nothing enterprise-level.

I will more than likely go along the lines of

> If you are not interested in data integrity or availability and just want good performance and a long SSD life expectation I would use the SATA SSD with LVM + xfs for the OS+backups+ISOs and the NVMe SSD with LVM-Thin as the VM/LXC storage.

For the backups how much would we need?

 

[Dunuin]

For the backups how much would we need?

Really depends on how much backups you want to keep and if you want to use vzdump or PBS. And you would need to backup host configs on another drive.

 

[mrpbennett]

For me i will more than likely use vzdump as I don't "currently" have PBS...I do have a WD Cloud on the network could i send the backups to that? I want to build a TrueNAS box eventually and have that as a source of truth for all backup and file store

 

[tonycav]

Hi, I was wondering if it made sense to use nvme disk as ext4 directory for data only.
Thank you

 

[Matthias.]

Considering we don't know anything about your setup, sure, setups where you want to do that are conceivable.

 

[tonycav]

Considering we don't know anything about your setup, sure, setups where you want to do that are conceivable.

Sorry, you are right
SSD 250GB (Proxmox + HA + Turnkey File Server)
NVME 2TB (Directory ext4 for only file sharing)

 

[Dunuin]

In case you don't got a 10+ Gbit NIC that is wasted money, as the disk performance of a NVMe is bottlenecked by the NIC throughput. Small NVMe SSD for PVE+guest storage and big SATA SSD for network shares would make more sense.

 

[andyn]

What NVMe drive did you go with?

Nothing special - just whatever I found cheapest online from a brand I'd heard off and a reputable supplier. I've noticed that both NVME and 2.5" SATA SSDs seem to have come down hugely in price over the last few months, so if I was doing it again I would likely make a different choice and have a larger, faster NVME for the same price.