new drive setup, considering RAIDZ1

Z

zenowl77

Active Member
Feb 22, 2024
248
45
33
finally managed to get some new drives for proxmox, ended up with 4x8tb SAS 7200RPM drives, thinking about setting them up as RAIDZ1, they will just be for VMs, AI models, VM data drives (large vm drives full of extra random files i dont want on the OS drive to keep it small when using a VM for media files, encoding, ai models, etc they all go on a second virtual disk)

i have 2x10tb 1x12tb 2x8tb drives for backups, data, media, etc so it isn't critical this raid be perfect or anything, although i am worried about increasing hard drive costs and not being able to afford a new 8tb drive to rebuild the raid if one fails, that is a worry.

i am thinking with this if i can offload all my VM disks and everything to this raid i can then use that space from my current drives for backups and other things, i want to try my hand at setting up proxmox backup as a container in proxmox too.

curious, what would you do? would like to hear some opinions, esp if this idea doesnt sound good.
 
Last edited:
UdoB said:
This gives the IOPS of a single spindle. Recommended is to use (two) mirrors --> double IOPS for writing data and four times the IOPS for reading data.

And... for rotating rust I highly recommend to add two fast - but small - SSD/NVMe as a so called "Special Device". It really speeds things up.

Also: https://forum.proxmox.com/threads/fabu-can-i-use-zfs-raidz-for-my-vms.159923/
Click to expand...
that is good to know, just write right?

i am weary of using special devices, if one of those fails the pool goes down too right? so its best to use mirrors there too?

i planned to use l2arc from an NVME but i wanted to go low risk routes that wont cause the pool to fail if anything happens, i need to get an enterprise nvme still
 
zenowl77 said:
i am weary of using special devices, if one of those fails the pool goes down too right? so its best to use mirrors there too?
Click to expand...
UdoB said:
I highly recommend to add two fast - but small - SSD/NVMe as a so called "Special Device". It really speeds things up.
Click to expand...
so yes :)

zenowl77 said:
i planned to use l2arc from an NVME
Click to expand...
l2arc almost never yields useful results. you're better off just using the drive seperately.

More to the point- what is your usecase? in a homelab, its common that your bulk storage can be slow without any real impact. put your vms/cts on the nvme and keep your raidz1 for your "iso" library.
 
alexskysilk said:
so yes :)


l2arc almost never yields useful results. you're better off just using the drive seperately.

More to the point- what is your usecase? in a homelab, its common that your bulk storage can be slow without any real impact. put your vms/cts on the nvme and keep your raidz1 for your "iso" library.
Click to expand...

yeah i think id probably skip the special devices unless i end up getting some more NVMEs or something

it has seemed to at least improve results slightly from what i have noticed although barely noticeable, i have also just been using it as a file on the nvme with other files not a dedicated device, so its not a big deal a few GB goes to that just for the heck of it.

mostly the usecase for the ZFS raid will just be file storage and not often used VMs/some CTs, most CTs that need low latency and handle many small files are on SSD/NVME and the main VM i use is on an SSD, just a few other VMs arent on SSD, currently they are running off of a spare 1tb 2.5in sata CMR HDD with compression and dedup and its fine so i doubt the raid will be an issue performance wise.

i might end up making a dataset for podcasts or other media on it for use with audibookshelf, just to free up more space from other drives. most these the main OS will be backed up and the data is just random downloads, temp files, models, etc nothing i cant just redownload even if it would take some time.

the most performance demanding thing will be reading AI models off the drives probably, as those can be tens of GB, most other things will really see relatively zero impact from low performance.
 
alexskysilk said:
keep those on the nvme. they dont need any resilience, and you're likely to be replacing them quite often anyway.
Click to expand...
right now i have about 4.4TB+ of ai models between all the different text/image versions, loras and all the different things related to them all and for ollama since it cant just use the files as other options can. which is too much really, but they all do slightly different things better than others and i randomly play around with different models. (although i will probably end up deleting half of them at least)

its just too much to waste my SSDs/NVMEs on
 
You must log in or register to reply here.
Top Bottom
Back