Disk Layout - Optimal Configuration and Performance Concerns?

[ldavis]

Hello, I'm planning to transition from a TrueNAS Scale system to a Proxmox setup, incorporating a virtualized TrueNAS system. This shift is primarily to overcome some virtualization limitations in TrueNAS, and I intend to use PCI passthrough for this setup.

Currently, my TrueNAS configuration is as follows, and I'm wondering if a similar arrangement is desirable with Proxmox:

TrueNAS Configuration:

• Boot pool: ZFS mirrored pool using small, basic SSDs.
• ZVOL Pools: Utilized for VMs, consisting of NVMe drives.
• Data Pools: Designed for data and similar needs, using spinning disks.

Proposed Proxmox Configuration:

• Boot/Install: ZFS mirrored setup with small, basic SSDs.
• VM Storage: 2-3 separate NVMe pools.
• Data Storage: Spinning disks, with HBAs ultilized by TrueNAS VM via PCI passthrough.

My concern arises from noticing that most guides suggest installing directly onto the primary VM drives, which in my case are NVMe drives. However, the Proxmox Installation Wiki doesn't specify a recommended or best practice setup for this scenario. I'm hesitant to install Proxmox on the slower SSDs if it might negatively impact performance. Additionally, I'm considering the need for separation of duties; for instance, if I ever need to reinstall Proxmox and restore my configuration. This process is straightforward with TrueNAS, and I'm assuming Proxmox should offer similar ease, but I'm unsure about the best approach.

I would greatly appreciate any advice or insights on this matter.

 

[leesteken]

Proxmox itself is best installed on (slow) HDDs because it does not determine the performance of the VMs but it does write a lot of logs and data for graphs.
For VMs/CTs, (second-hand) enterprise SSD/NVMe with PLP are the best (in which case Proxmox can also be installed on that because of the high write endurance).

 

[ldavis]

Perfect thanks @leesteken !