Storage capacity problems - ZFS RAIDZ-1

[Saphirim]

Hi everyone,

I recently switched from a Hyper-V based home server to proxmox...out of intereset and to engange a little bit with technologies I was not really firm with in the past (Linux, LXC, ZFS, ...)

So far the migration went quite fine, my usual setup includes:
- 4 Windows 10 VMs
- Some linux hosts, future LXC containers (Pi-hole, nextcloud, ...) (Still on the old environment)

When I started copying the biggest part of the media to the new proxmox-solution, I encountered issues I certainly cannot explain to myself right now.

I am using an ZFS RaidZ-1 consisting of 3 x 2TB HDDs as a (slower) storage for data.
On this storage I have just on VM disk, which is mounted to a Windows 10 VM.


When I started copying files (mostly movies and photos) to this storage, everything seemed fine first, but after aprox. 2.5 TiB, the machine crashed, the copy was interrupted and the storage summary in proxmox shows that im using 3,87T of the available 3.52T




My storage.cfg looks like this:

dir: local
        path /var/lib/vz
        content backup,vztmpl,iso

zfspool: local-zfs
        pool rpool/data
        content rootdir,images
        sparse 1

zfspool: fast-storage
        pool performance-data
        content rootdir,images
        nodes proxmox
        sparse 1

zfspool: slow-data01
        pool slow-data01
        content images,rootdir
        nodes proxmox

zfspool: slow-data02
        pool slow-data02
        content images,rootdir
        nodes proxmox
        sparse 0

I'm happy to provide more information about the system, if you could point me to the needed bits.

Also happy to learn something new about my missunderstandings or missconfiguration, if that happened at any point.

Regards,
Saphirim

 

[fabian]

it looks like your pool is full? raidz has a certain (space) overhead that can be higher than expected, especially in combination with zvols, small volblocksize, and ashift=12.

 

[Saphirim]

Hi,

thank you for the input.
Is there a good reading you could suggest to get more information about a pool configuration more fitting to my use case?
Waht would be recommended values for volblocksize and ashift in this scenario?

Can I provide some more information to make the solution more obvious?

Regards,
Saphirim

 

[fabian]

https://github.com/zfsonlinux/zfs/issues/548
https://github.com/zfsonlinux/zfs/issues/1807
https://github.com/zfsonlinux/zfs/wiki/FAQ#ZVOL-used-space-larger-than-expected

modifying either ashift or volblocksize both come with their own drawbacks - I suggest reading the documentation and the linked issues and deciding which variant you prefer (you could also switch to 4 disks and striped mirrors, which don't suffer from this issue).

 

[guletz]

Hi

As your vdisk name suggests, your usage is to store some movies/filme00 who are large files and mostly uncompressed . I would do like this:

- install as usual a w10 or whatever on a small vdisk
- create a new vdisk using volblocksize=64k
- attach this new vdisk to your vm and format this new drive using the same 64k size
- then copy your data to this new vdisk

Good luck/Bafta