[SOLVED] ZFS Eating Storage like nothing

[Broehl]

New here so bear with me. I have a zfs pool setup with 8 4tb ssd's. That was done in zfs raid 1. I am told i have 26.81tb of usable space. I then go to create the server vm, and i have the OS drive on a separate pool. I create another SCSI drive from the 26.81tb pool.

The problem is, I cant create it until I have put the drive size all the way down to 20tb. Why am i losing that much space?

I really wanted to buy this as I am migrating from VMware, but this is leaving a sour taste in my mouth about the loss of that much drive space. Any help would be greatly appreciated.

 

[VictorSTS]

ZFS RADIz's padding in action. There are dozens of forum posts about this [1]. You will have to find the right balance between volblocksize and write amplification (which depends on you applications).

For me RAIDz is a no-go for VM storage, unless the performance requirement is low. AFAIK VMware has no raid5 software equivalent, is this the same hardware you used with VMWare?

EDIT: official docs also mention this [2]

[1] https://forum.proxmox.com/search/7905224/?q=raidz+padding&o=date
[2] https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_raid_considerations

 

[Broehl]

I am using my old VM server.
Also, I used to have only 6 - 4tb drives and it had just as much space as adding the two more 4tbs. It has me baffled and I am not wondering if this is a bug. Cause there is no reason why I start with 32tb, run raidz1 and then I am down to 26.81tb... And then setup a partition inside that and only use 19.04 of that. I will have lost more drive space than gained. Almost like i didn't even install them.

 

[leesteken]

Hardware RAID5 with a BBU has much better performance and space than raidz1 with (less than a dozen) drives without PLP. The former can handle most VMs fine but the latter is terrible for VMs. They are completely different in all aspects except that they can lose one drive without breaking. This has indeed been discussed before but keeps surprising new users. Maybe Proxmox VE should warn about this in the installer and web GUI.