Questions regarding, storage mediums, software RAID, etc.

[nebyneb2357]

I'll be starting college this fall, so I'll have to leave my current homelab setup behind. It includes two Ubuntu servers, one of which has a ZFS mirror with two 4TB WD Red drives in a Mediasonic Probox USB JBOD. Since I'll be 16 hours away, I want to minimize the need for in-person maintenance. You can see my current setup though process below. Note: I understand that mirrored drives aren't a backup, so I'll use PBS for that. While I trust the people at my house to replace a drive that I can rebuild remotely, I don't trust them to set up a full Proxmox install by themselves.

Questions​

1. I want to do things cheaply and efficiently, while maintaining redundancy. My machines are SFF, so adding more than 2 SATA drives will require a [PCIE to SATA card](https://a.co/d/4rjbvih) and a [SATA power splitter](https://a.co/d/9o4EJ80), and I'd rather not do this unless it's absolutely necessary. I have seen that these cards can be problematic to boot off of. Does anyone know if they work in Linux (Proxmox and TrueNAS)?
2. Should I get some enterprise SSDs to run in both machines? I will need to buy the PCIE SATA card and SATA splitter. This is because I currently don't have the funds to invest in two 1TB enterprise SSDs to store both the Proxmox OS and virtual machines on the same mirror. Does anyone have a link or datasheet where I can see actual model numbers and specs of popular used enterprise SSDs? I can't find this information online and would like to explore all the options myself.
3. I have heard that it is not okay to run consumer SSDs in a ZFS boot mirror, especially in Proxmox. I don't want boot drives failing while I'm away. Is it okay to run VMs off of some ZFS mirrored HDDs? Is there an alternative to ZFS that doesn't consume drives quickly but still provides redundancy and allows for a quick enough rebuild? The last thing I want is for both drives to fail before they can rebuild.

Future setup structure​

Dell Precision t3430 SFF​

• Proxmox Virtual Environment
• Debian
• Bare-metal
• DDNS updater
• Docker
• Jellyfin
• AMP
• Nginx Proxy Manager
• Authentik
• Immich
• Syncthing
• PiHole
• WireGuard
• VaultWarden/BitWarden
• Frigate NVR (uses TrueNAS mirrored NFS share for recordings)
• Annatar Stremio add-on
• Home Assistant
• Proxmox Backup Server

Dell Precision t3420 SFF​

• TrueNAS Scale
• Proxmox Backup Server
• NFS and Samba share
• DDNS updater

Currently owned storage devices available​

Note: I have omitted the 2tb and 2x4tb drives. I will be using them for TrueNAS Scale storage.
• Kingston a400 256gb 2.5 SSD
• Seagate 500gb 3.5 HDD
• Toshiba 1tb 2.5 HDD
• Toshiba 500gb 2.5 HDD (super old)

Networking​

• 100mbps/10mbps incomming ADSL connection
• TP Link Archer AX1500 with Zyxel C3000Z (transparent bridging mode)
• TP Link 5-port Gigabit switch

 

[Dunuin]

Is it okay to run VMs off of some ZFS mirrored HDDs?

VMs usually need IOPS performance and 100 IOPS of HDDs vs for example 1,000,000 IOPS of a SSD is huge.
Running a single OS on one HDD is already slow as hell. Running multiple OSs from a single HDD won't make it better.

Is there an alternative to ZFS that doesn't consume drives quickly but still provides redundancy and allows for a quick enough rebuild?

LVM-Thin on top of mdadm SW raid if you install PVE8 on top of Debian 12.
But not officially supported (but works) and you will lose your bit rot protection.

 

[nebyneb2357]

VMs usually need IOPS performance and 100 IOPS of HDDs vs for example 1,000,000 IOPS of a SSD is huge.
Running a single OS on one HDD is already slow as hell. Running multiple OSs from a single HDD won't make it better.

I'm thinking of keeping it simple and just using 2 used enterprise SSDs in a mirrored setup for both the OS and virtual machines. Would this be okay? Also, any recommendations for SSD models suitable for this workload? Maybe even good eBay sellers? I don't plan on running more than the mentioned services, so I think two 1TB SSDs should suffice, but I want to know your opinion on this.

LVM-Thin on top of mdadm SW raid if you install PVE8 on top of Debian 12.
But not officially supported (but works) and you will lose your bit rot protection.

Thanks, but I'd rather not do a hacky solution. I haven't even had a chance to install Proxmox yet. Quick question, would it work if I buy small capacity enterprise SSDs for a TrueNAS boot mirror? I know this might be beyond this forum's scope, but you seem quite knowledgeable.

Thanks so much for replying. I've been trying to find help on r/proxmox and couldn't really get any.

 

[Dunuin]

I'm thinking of keeping it simple and just using 2 used enterprise SSDs in a mirrored setup for both the OS and virtual machines. Would this be okay?

Yes.

Quick question, would it work if I buy small capacity enterprise SSDs for a TrueNAS boot mirror?

Yes, even 100GB disks are way too big for boot disks.

 

[nebyneb2357]

Yes.

Yes, even 100GB disks are way too big for boot disks.

Do you have any recommendations for drive models? I found a 1.2TB enterprise SSD priced at $44.95 (for the Proxmox machine). The seller claims it has 81% or more of its lifespan remaining. I have never bought used storage, especially enterprise grade storage. Would you consider this a good deal?

 

[Dunuin]

Depends on the model. But sounds like a good deal.

 

[nebyneb2357]

Depends on the model. But sounds like a good deal.

The seller says it is an Intel DC S3500. How long do these things last? Here's the listing.

 

[PikkonMG]

The seller says it is an Intel DC S3500. How long do these things last? Here's the listing.

1.2TB model has a 660 TBW Endurance Rating so its not bad if you can get them at a good price. I have 5 dc s3500 600GB models I run they are lower TBW of 330 I think. I switched logging to use "folder2ram" and not much gets written to them.

 

[nebyneb2357]

1.2TB model has a 660 TBW Endurance Rating so its not bad if you can get them at a good price. I have 5 dc s3500 600GB models I run they are lower TBW of 330 I think. I switched logging to use "folder2ram" and not much gets written to them.

Thanks! Is folder2ram any better than log2ram?

 

[PikkonMG]

Thanks! Is folder2ram any better than log2ram?

I like folder2ram just because im use to it and It sync logs back to disk when system is shutdown, You can also do manual sync do disk. Can probably be done with log2disk also never used it.