i am wanting to take advantage of the ZSTD compression feature in ZFS for backups, possibly even deduplication but i am not certain i want to enable that on a single disk setup and risk multiple backups relying on a single file.
but in previous attempts i have found the performance loss on ZFS is massive for a single disk, while in other filesystems i will see 250mb/s ZFS drops that to 50-80 maybe 120mb/s plus uses a lot of memory and seems to increase delays and hang for short bursts since it is trying to handle tasks on a single disk so it doesn't really work for VM storage when used in a single disk scenario.
i am guessing this is mostly because of compression? and i should probably set compression on a per zvol/directory level
i have experimented with BTRFS but i feel like it is a bit unstable to trust to backups, twice i had it lose all data when doing many small writes to files as a vm storage disk. (which wouldnt happen as a backup disk but still i am hesitent of it, why add something that has specifically caused problems before. )
the disk i am considering converting to ZFS for backups and ZSTD compression is a 12TB HGST Sata Enterprise HDD, currently i am using a 10tb HGST enterprise disk for VM storage disks, so i do not use the drive directly for much of anything except storage. so it is also not the end of the world of performance loss cannot be mitigated somehow, just wondering if there is anything i can do really to tweak ZFS to better fit my setup.
i am considering a sas controller with HBA firmware so i can offload drives from the on board sata controller to hopefully get slightly better i/o along with sas support, this will also free up extra sata ports allowing me to use the 2nd NVME slot on the board (the board disables sata port 1 if i plug in a second nvme, so right now i have to choose between a 1tb sata ssd or a 256gb nvme with what i have on hand), then with the extra nvme i can add a slog, etc for ZFS. (i also have no ups, so thats a bit of an issue there for write caching but one step at a time right haha)
any tweaks, suggestions, resources, etc, would be greatly appreiciated, i have been reading up on zfs a lot since beginning my use of proxmox but i feel there is still so much to it in little advanced settings behind the scenes and beyond basic commands.
but in previous attempts i have found the performance loss on ZFS is massive for a single disk, while in other filesystems i will see 250mb/s ZFS drops that to 50-80 maybe 120mb/s plus uses a lot of memory and seems to increase delays and hang for short bursts since it is trying to handle tasks on a single disk so it doesn't really work for VM storage when used in a single disk scenario.
i am guessing this is mostly because of compression? and i should probably set compression on a per zvol/directory level
i have experimented with BTRFS but i feel like it is a bit unstable to trust to backups, twice i had it lose all data when doing many small writes to files as a vm storage disk. (which wouldnt happen as a backup disk but still i am hesitent of it, why add something that has specifically caused problems before. )
the disk i am considering converting to ZFS for backups and ZSTD compression is a 12TB HGST Sata Enterprise HDD, currently i am using a 10tb HGST enterprise disk for VM storage disks, so i do not use the drive directly for much of anything except storage. so it is also not the end of the world of performance loss cannot be mitigated somehow, just wondering if there is anything i can do really to tweak ZFS to better fit my setup.
i am considering a sas controller with HBA firmware so i can offload drives from the on board sata controller to hopefully get slightly better i/o along with sas support, this will also free up extra sata ports allowing me to use the 2nd NVME slot on the board (the board disables sata port 1 if i plug in a second nvme, so right now i have to choose between a 1tb sata ssd or a 256gb nvme with what i have on hand), then with the extra nvme i can add a slog, etc for ZFS. (i also have no ups, so thats a bit of an issue there for write caching but one step at a time right haha)
Code:
Hardware setup:
CPU: i7-7820X @ 3.3ghz / 120w + 20% throttle (to 96w) after 2 min of 100% usage.
RAM: 96GB DDR4 2133mhz
GPU 1: GTX 1060 3GB (limited to 60w)
GPU 2: Intel Arc A310 4GB
GPU 3: Tesla P4 8GB
MAIN Drive: 1TB WD Blue SN550 NVME
2ND Drive: 1TB WD Blue SA510
3RD/4TH: Drive: 2x10TB HGST Sata enterprise HDD
5TH: 12TB HGST Sata Enterprise HDDany tweaks, suggestions, resources, etc, would be greatly appreiciated, i have been reading up on zfs a lot since beginning my use of proxmox but i feel there is still so much to it in little advanced settings behind the scenes and beyond basic commands.