Best ZFS configuration for 4 SSDs

marcelovvm

Member
Feb 20, 2020
11
0
21
48
Rio de Janeiro - Brazil
Dear colleagues, I'm setting up my first server with Proxmox in a test environment to later go to production. I've already used Proxmox at home in a simpler environment (only one XFS disk for the system and for storing the VMs). But this new scenario is very different and I want to try ZFS.

The server has a flash drive (32GB) with Proxmox 8 installed. I have 4 Kingston SDEC450R SSDs (natively 512 bytes per sector) of 480GB connected to an LSI 9211-8i controller. When we used RAID this server was configured in RAID 10 having 899GB RAW for the Linux and Windows VMs (we used ESXi). The server has 72GB of RAM.

So for ZFS what would be the best configuration for good performance? Two pools + mirror? A RAID-Z1 or Z2? For the ashift parameter, would 12/13 be better?

Conditions:
- losing the space of two disks (960GB) is acceptable;
- for protection, losing a disk and keeping the environment working is quite acceptable (even knowing that in RAID 10 you could lose up to two disks)

Live long and prosper,
Marcelo Magalhães
 
Usually it is really(REALLY) not recommended to run proxmox on usb drives.Enough on that.
For ZFS raid10 would be recommended way, for performance,and ideally you can lose two nodes(not in all cases).
 
Usually it is really(REALLY) not recommended to run proxmox on usb drives.Enough on that.
For ZFS raid10 would be recommended way, for performance,and ideally you can lose two nodes(not in all cases).
OK. So just in case you opt for a RAID 10-like structure. Would I have to create two VDEVs with two disks each and a mirror between those two VDEVs? Stripe + mirror.
 
Good point! Tks.

So... the sequence of commands would be:

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb

root@hostname:~# zpool create -o ashift=12 mypool mirror /dev/sdc /dev/sdd
In ZFS a raid10 i called a "striped mirror", not a mirrored stripe ;)

And no, that would be wrong and would try to create two mirrors (aka raid1). Striped mirror (aka raid10) would be zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb mirror /dev/sdc /dev/sdd. And don't use "/dev/sda", "/dev/db" and so on. Use "/dev/disk/by-id/YourFirstDisk", "/dev/disk/by-id/YourSecondDisk" and so on instead.
 
In ZFS a raid10 i called a "striped mirror", not a mirrored stripe ;)

And no, that would be wrong and would try to create two mirrors (aka raid1). Striped mirror (aka raid10) would be zpool create -o ashift=12 mypool mirror /dev/sda /dev/sdb mirror /dev/sdc /dev/sdd. And don't use "/dev/sda", "/dev/db" and so on. Use "/dev/disk/by-id/YourFirstDisk", "/dev/disk/by-id/YourSecondDisk" and so on instead.
TKS!!! Thank you for your help! I hadn't noticed the "subtlety" of the difference.
 

About

The Proxmox community has been around for many years and offers help and support for Proxmox VE, Proxmox Backup Server, and Proxmox Mail Gateway.
We think our community is one of the best thanks to people like you!

Get your subscription!

The Proxmox team works very hard to make sure you are running the best software and getting stable updates and security enhancements, as well as quick enterprise support. Tens of thousands of happy customers have a Proxmox subscription. Get yours easily in our online shop.

Buy now!