Space available on Raidz2 on 4 disks

[yena]

Hello to all, I have a doubt about a raidz2 consisting of 4 SSDs.
I don't understand how I can get 6.98TB Size with two disks that can fails:

root@nodo2:~# zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
rpool 232G 22.0G 210G - - 50% 9% 1.00x ONLINE -
storage 6.98T 850G 6.15T - - 4% 11% 1.00x ONLINE -

-------------------------------------------------------------------------------------------------------------------------------------

root@nodo2:~# zpool status -v
pool: storage
state: ONLINE
scan: scrub repaired 0B in 0 days 00:29:31 with 0 errors on Sun Nov 8 00:53:33 2020
config:

NAME STATE READ WRITE CKSUM
storage ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ata-SAMSUNG_MZ7LH1T9HMLT-00005_S455NA0N412328 ONLINE 0 0 0
ata-SAMSUNG_MZ7LH1T9HMLT-00005_S455NA0N412311 ONLINE 0 0 0
ata-SAMSUNG_MZ7LH1T9HMLT-00005_S455NA0N911583 ONLINE 0 0 0
ata-SAMSUNG_MZ7LH1T9HMLT-00005_S455NA0N412329 ONLINE 0 0 0

errors: No known data errors


---------------------------------------------------------------------------------------------------------

Disk /dev/sdd: 1.8 TiB, 1920383410176 bytes, 3750748848 sectors
Disk model: SAMSUNG MZ7LH1T9
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D2425F3C-A1D4-F24F-9BC1-E54F06C37E2C

Device Start End Sectors Size Type
/dev/sdd1 2048 3750731775 3750729728 1.8T Solaris /usr & Apple ZFS
---------------------------------------------------------------------------------------------------------

On my calculation if fails 2 disks i will have ~3.8Tb available...

Thank!

/dev/sdd9 3750731776 3750748159 16384 8M Solaris reserved

-----------------------------------------------------------------------------------------------------------

 

[Alwin]

Hello to all, I have a doubt about a raidz2 consisting of 4 SSDs.
I don't understand how I can get 6.98TB Size with two disks that can fails:

z2 is two parity chunks for the written data, not for the empty space.

 

[yena]

z2 is two parity chunks for the written data, not for the empty space.

So, if i fill more than about 3.8TB, i don't have two spare disks anymore? Or do you mean that 6.98 Tb is not real .. thanks!

 

[H25E]

I think that what Alwin tries to say is that you have 6.98Tb for all your data, where data is actual content, parity and padding. The actual real space that you will have available for your content will vary (decrease) depending on the ashift/sector size of your disks and the recordsize of the dataset. But as maximum you will have the half of this value.

In the other hand, you could reconsider your scheme. A pair of mirrors of two disks each one could be better.

 

[yena]

I think that waht Alwin tries to say is that you have 6.98Tb for all your data, where data is actual data, parity and padding. The actual real space that you will have available for your content will vary depending on the ashift/sector size of your disks and the recordsize of the dataset.

I think that waht Alwin tries to say is that you have 6.98Tb for all your data, where data is actual data, parity and padding. The actual real space that you will have available for your content will vary depending on the ashift/sector size of your disks and the recordsize of the dataset.

So it's super hard to Know the "real" space available for contents

 

[H25E]

Take a look here

 

[chamseddine]

with 4x4Tb in raidz2 :​

ZFS usable storage capacity:

6.810606

Practical usable storage capacity:

5.448485

ZFS / RAIDZ Capacity Calculator (beta) https://wintelguy.com/zfs-calc.pl ​

ZFS / RAIDZ Capacity Calculator (beta)​

 

[chamseddine]

https://nocin.eu/proxmox-installing-cockpit-with-zfs-manager-extension/

 

[aaron]

We have a section in the documentation about the different pro and cons regarding pool design [0].

The TL;DR for a VM workload is to use a pool made of mirror vdevs (RAID 10 like) for the best performance and to avoid surprises regarding higher space usage due to unexpected parity data.

[0] https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_zfs_raid_considerations