ZFS With Raid 1

[Dakater]

Enterprise SSDs aren't that expensive anymore. You can get some nice second-hand eMLC Enterprise SSDs for 60€ per TB. Or even new NVMe TLC Enterprise SSDs starting at 70€ per TB

Checked all the params for eMLC and TLC again.
No. TLC is not fine for ZFS while 1.000-3.000 cycles is just not enough.

But those enterprise MLC is a try with 20.000 to 30.000 cycles. Thanks for this. No idea why I never saw or read about before. Great hint.

 

[Dunuin]

Again, there are no SLC, eMLC or MLC anymore unless you want to buy second-hand hardware or new old stock. Only SLC you see these days are industrial USB sticks and industial SD cards and only MLC are SMMs and DOMs. Last SLC SSD product lines were killed in 2022 (Intel Optane). and last MLC SSD product lines in 2018 (except for some Transcent SMMs).

If you got lots of writes, many sync writes or a high write amplification your best bet these days is to buy a very big TLC Enterprise SSD with PLP, don't partition the whole disk and use the unallocated space as additional spare area for wear leveling.

 

[mow]

Again, there are no SLC, eMLC or MLC anymore unless you want to buy second-hand hardware or new old stock. Only SLC you see these days are industrial USB sticks and industial SD cards and only MLC are SMMs and DOMs. Last SLC SSD product lines were killed in 2022 (Intel Optane). and last MLC SSD product lines in 2018 (except for some Transcent SMMs).

I thought WD still had some Ultrastar SS300 models? (Starting with HUSM, TLC would be HUST)
And there's https://www.solidigm.de/products/data-center/d7/p5810.html, 144L SLC 3D NAND with 50 DWPD, I assume that's the new Optane.
Edit: Kioxia has something similar with 60 DWPD: https://europe.kioxia.com/en-europe/business/ssd/enterprise-ssd.html#fl-series

If you got lots of writes, many sync writes or a high write amplification your best bet these days is to buy a very big TLC Enterprise SSD with PLP, don't partition the whole disk and use the unallocated space as additional spare area for wear leveling.

That's not necessarily optimal. The really big SSDs optimized for size tend to have less TBW than smaller but mixed-workload ones like the P4610.

Another thing is data retention. QLC SSDs guarantee only 3 months, so better don't keep those offline for long.

 

[qhash]

Just want to add some usefull informations on the raid1 informations.

1. Yes, it is recommended to install Proxmox with ZFS as raid.

2. Yes, you better use mechanic disks (hdd) and no ssd! Or buy yourself special industrie ssd. (None Multi-Layer!)

3. If you use it for home improvement you can turn of a lot of ZFS logging which will be a bit better for a "standard ssd". Myself using 2x 120gb soho ssd (12$ each), BUT ...

4. It is NOT recommended to use any md raid, BUT you can. First install Debian 12.2 or higher. Create a normal raid1 with mdadm in the shell or right away in the GUI installer.

5. (u)EFI and a new BIOS can kill your plans. Old mainboards support a "Legacy mode". Then it is the old easy way.
New BIOS are now the first days without this option. You aew forced to EFI (or more harder boot-security).
That leads to an EPS partition, a swap partition and a root partition (at least).

6. There is no way to directly raid1 with EPS, BUT ... After installing with 2x SSD you can create md0 for swap and md1 for root and just ignore the EPS partition. That leads to a raid1 without any EPS failover.
After that just ADD in the active EPS partition in the boot order the 2nd inactive EPS partition. Copy the active boot partition to the inactive one.
-> If sda fails then sdb has an active EPS partition. While 1st is down 2nd will be used.

7. Replacing on ZFS and with mdadm on EXT4 is something total different.

8. LVM containering WILL not prevent you against the EFI limitations. Your EPS will not be "directed" correctly.

9. mdadm with Proxmox CAN lead to a major problem where you could destroy all your data.
ZFS will destroy your hardware very fast, if its not the right one!

10. My toughts: Put your Proxmox on cheap ssd. With raid1 and zfs or "single" and always hold a 1:1 copy. So if you change elementary setup on proxmox then copy again 1:1 to your 2nd ssd. In case of failure you just have to replace. I use a 6x SSD cage in a 5,25" slot. 2x cheap ssd for proxmox (right now hard testing it with md-raid and irs working awesome). 2x 2TB SSD for Data (raid1) and of course your fast NVMe. Put them in a raid1 and use it for the base install of your guest OS (Linux, Windows, Hackintosh)

And always keep in mind:
A raid is never a backup.
A NAS or SAN is never a backup even in HA mode.
Even a cluster spreaded on several actice-active SAN's with HA-iSCSI connection of each guest is not a backup or high security.
One controller error and ...

Wish you luck and I suggest ZFS with GUI install flr easy start plus logging off plus learn to replace a disk before installing active (needed) guests.

I have installed PVE on ZFS RAID1 using standard graphical PVE installer ISO on a N100 miniPC with legacy BIOS. Trying to figure out why after adding an ISO of my Untangle instance to PVE ISO repository only space on PVE local is occupied 0,1%, whereas local-zfs is 0.0%... I only have 2 x 1TB SSDs and I have installed on them. I am worried my install went wrong and my drives do not work in RAID1. There are ZFS partition on both drives, I have checked that. Should I replicate any of the steps you have advised and also should I disable ZFS logging if those drives are rated for 1000TBW?