LVM / ZFS possible safe and sensible disk layout

[mafel]

Hi guys, since I'm still at the beginning with proxmox (coming from unrai*) I'm wondering about the right disk layout. I have quite a wild growth concerning my hard disk landscape and wanted to ask if a ZFS makes sense or rather LVM?

How could a possible ZFS make sense with the disks?
I also have an ioDrive2 with 1.2 tb that I could install instead of the NVME, for example.

I would be grateful for your ideas

Translated with DeepL.com (free version)

 

[justinclift]

Hmmm, providing a screenshot like that is a bit of a pain.

Taking the time to translate (heh) those model numbers into more human readable terms, it's these:

• Samsung 980 Pro 2TB M.2 NVMe drive
  • Doesn't have power loss protection. Uh oh.
• Western Digital Red NAS 10TB 3.5" SAS HDD (WD101EFBX)
• The model string is truncated , but it looks like it's probably a Seagate IronWolf 125 500GB SSD
  • The data sheet for that model also has no mention of power loss protection. Hmmm.
• Western Digital Red NAS SSD 2TB
  • Amazingly, this one's data sheet also doesn't mention power loss protection. This seems to be a theme?
• Samsung 850 Pro SSD 512GB
  • Data sheet here. No power loss protection.

Next question is... what are you wanting to do with your system? That's kind of a critical thing.

Anyway, without knowing your intended use case my first thought would be this:

1. Mirrored boot drives (using ZFS RAID1) with both the 2TB Samsung 980 Pro & the 2TB Western Digital Red NAS SSD
   • That'll give you resilience to failure of either, and all of the excess space will be made available for storing both ISOs and VMs on.
2. "Do something" with the 2x ~500GB SSDs. You could put them in a RAID 1 pair for mirroring (giving about 500GB of space), or pair them up in RAID 0 (no redundancy) for ~1TB of usable space
3. Use the 10TB drive for bulk storage. I'd go with ZFS for this again, just coz I like ZFS.

 

[Kingneutron]

I would recommend using the 2x500GB SSDs, respectively, for LVM+ext4 root partition and thin-lvm. LVM+ext4 is "easy" to backup and restore, and for homelab you don't really need a mirror - you need backups. Always Have Something To Restore From.

With a mirror yes, you get redundancy and uptime - but you lose 1/2 your storage space. Backup root + VMs to the 10TB as ZFS or separate NAS.

Then mirror the 2x2TB SSDs as a ZFS pool.

Apart from that, I like the above recommendation

 

[justinclift]

for homelab you don't really need a mirror - you need backups

Good point. I'm so used to having separate backup systems (even in a homelab environment) that it just didn't occur to me here.

 

[mafel]

Hello justinclift and Kingneutron,

first of all, thank you for your thoughts and ideas.

To add a few things:
- will be used as a homelab with lxc containers openvpn, npm, bitwarden, wordpress, etc and VMs like nextcloud, homeassistant, a gameserver, access win10 from outside etc.

so I guess ATP would be really nice but probably not worth the extra cost in this environment. To compensate for this, I have a UPS in between, which sends mega alarms (mails, sms etc.) in the event of a power failure and shuts down the system.

In fact, I agree with Kingneutron. I have also worked a lot with backups so far (also externally via another NAS) and get along with the procedure, so I guess this is the way to go.

Since 1x 2 TB is an nvme and the 2TB an SSD, does it make sense to use 2x SSD? (I wouldn't be able to get another NVME in as there is only 1 pcie slot in the microserver)

I also like the idea of completely outsourcing the backup from Kingneutron. I had actually used the 10TB HDD as a backup and then outsourced it again (simply mirrored externally)

Would it speed up the ZFS if the layout looked like this?

ZFS CACHE - nessasary?
- 2TB NVME or Fusion ioDrive2 HP ioDrive II ? (got it to run stable under proxmox) or another alternative would be a DC P4510 with 2 TB that I still have lying around

ZFS POOL
- 2TB SSD
- 2TB SSD
*-2TB SSD

BACKUP (as single LVM ext4 - outside the ZFS) ?
- 10TB HDD

then the 2x 500 gb SSDs would be thrown out and 1 slot would still be open, what would be useful additions? one more 2TB SSD?

 

[Kingneutron]

Never throw out a perfectly good ssd, you never know when you may need to stand up another Linux (or cluster) and they're already paid for. Sell them if you really think you'll never need them

> Since 1x 2 TB is an nvme and the 2TB an SSD, does it make sense to use 2x SSD

If you have a speed discrepancy between 2 drives, you don't want to mirror them bc the pool will be limited to the SLOWEST drive and it would be a waste of money for the extra speed. Just like you wouldn't want to mirror an SSD with a spinning drive.

I would use the 10TB as a single-disk zfs pool for backup. XFS (as a standard, native filesystem) is better for larger drives but you get benefits with zfs, especially if you can mirror it down the road.