Hard drive configuration - what do you do?

[dust_bro]

I'm looking on recommendations on how you guys have your drives setup. What configuration would I use to enable: redundant drives for VMs that allow snapshots and have fast transfer rates?
I currently have 4 SSDs in RaidZ2 for my VMs and then a couple other random SSDs for ISOs and data. Some machines (windows and mac) are unable to take snapshots as their drive format is RAW and the option to convert is greyed out, but linux machines on the same drive CAN take snapshots. Migrating disks takes FOREVER. Like 4 hours to move a 256GB drive from one ssd to another.
What's the proper way to do this?

 

[Maximiliano]

Hello,

What kind of disks are you using, what exact model? We highly recommend enterprise grade SSDs. Consumer disks generally slow to to a crawl after filling their cache, which can happen rather quickly when working with block devices.

On another note, we recommend 2 mirror vdevs [1] when using four drives and performance is important. Another thing that you might want to check is whether your VM images are stored in the qcow format, as doing so would add quite a bit of overhead.

[1] https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_zfs_raid_considerations

 

[dust_bro]

Thanks for the reply! So, 4 disks in Raid10 for VMs. That gives a performance boost + parity.

 

[dust_bro]

Hello,

What kind of disks are you using, what exact model? We highly recommend enterprise grade SSDs. Consumer disks generally slow to to a crawl after filling their cache, which can happen rather quickly when working with block devices.

On another note, we recommend 2 mirror vdevs [1] when using four drives and performance is important. Another thing that you might want to check is whether your VM images are stored in the qcow format, as doing so would add quite a bit of overhead.

[1] https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#sysadmin_zfs_raid_considerations

Could you also enlighten me on the connection between snapshots and a Windows VM TPM storage location? For example, I have an NFS share for VM disks. If the TPM storage is located on the share, I can't take a snapshot. But if I move the TPM storage to a local disk, snapshots become available.

 

[dust_bro]

Also, it's it normal to have a higher i/o delay (6-10%) when moving data or restoring backups?