VM on ZFS RAID on HDDs or on SSD, help?

[b0n3v]

Hello everyone, i have question about my Proxmox VE.
First details about my system:
OS - Proxmox VE 6.0 updated
Hardware:
CPU - Ryzen 1700x, RAM - 32GB 3200 nonECC,
SDD - nvme 512(system is on it +20% overprovision)
SSD - SATA 3 256GB
HDD - 2x2TB ZFS mirror lz4(for storage),
HDD - 2x1TB ZFS mirror lz4 for VMs

What is the right way to do my setup for VMs, and CTs storage?
What I'm thinking:
1.All of my VMs/CTs go to SATA SDD, and use 2x1TB for weekly fully backup - and i will have speed and redundun on ZFS mirror.
2.All of my VMs/CTs go to 2x1TB ZFS raid1 and use SATA SSD for cache?
3.May be something more clever?
And last question, what wiil be best option for file system on my SSD(SATA) for VMs/CTS - ZFS(did have already trim support?), ext4 ... ?
Thanks!

 

[Alwin]

1.All of my VMs/CTs go to SATA SDD, and use 2x1TB for weekly fully backup - and i will have speed and redundun on ZFS mirror.
2.All of my VMs/CTs go to 2x1TB ZFS raid1 and use SATA SSD for cache?

You can use both setups, its up to you. Just test it.

And last question, what wiil be best option for file system on my SSD(SATA) for VMs/CTS - ZFS(did have already trim support?), ext4 ... ?

ZFS has trim support. If you want snapshots, checksums and send/receive (pve-zsync) to transfer snapshot to other disks or offsite, than use ZFS. Besides RAID support for ZFS.

 

[guletz]

Hi,

If your data/vm is important, and you can not lose x hours (like for the case when vm are using ssd as storage) I will do like this:
- create 2 partition on 2 Tb hdd (1 Tb/parttion)
- create a zfs pool like raid 10 using 2 x 1 Tb disks + 2x 1 b partitions from 2 Tb hdd

On this pool run your VMs. The rest of 1 Tb partition from 2 Tb hdd use it for backup (another zfs pool if you like, mirror).


Use nvme for PMX Os and the ssd for zfs cache and log.

Good luck / Bafta

 

[jaceqp]

Just wondering if that platform is for production environment or just testing? Non-ECC RAM might matter on ZFS because of RAM dependency on ZFS in general.

 

[Alwin]

Non-ECC RAM might matter on ZFS because of RAM dependency on ZFS in general.

For long running systems ECC memory is always a good idea.

 

[b0n3v]

Just wondering if that platform is for production environment or just testing? Non-ECC RAM might matter on ZFS because of RAM dependency on ZFS in general.

Yes, is almost in production(moveing from dev), yes i know the negative side of using non-ECC RAM, but for this price and my old experience with ECC RAM isn't so important. I clearly understand how ZFS work, and i can say same thing ALWAYS bеtter with ECC RAM modules.
Don't you think ext4 is better with ECC RAM or any other file system... and etc.


Thanks for answer on @Alwin, @guletz, now i decide - will use SATA SSD for VMs/CTs, formatted in ext4(better performance than ZFS i think?), and ZFS mirror 2x1TB for weekly backup. IF this SSD die, i can restore backup and will continuous use my server. All of my VMs/CTs use another ZFS mirror for storage, which give me batter survival of important data.